vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #495
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.Xrefh11TxL', fmt=raw size=107374182414worker2: QEMU running (pid 52)15worker1 # Disk image does not exist, creating the virtualisation disk image...16worker2 # mke2fs 1.47.4 (6-Mar-2025)17worker1 # Formatting '/build/vm-state-worker1/tmp.1ERVoeI7OA', fmt=raw size=107374182418worker2 # Discarding device blocks: 0/262144 done19worker1 # mke2fs 1.47.4 (6-Mar-2025)20worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes21worker1 # Discarding device blocks: 0/262144 done22worker2 # Filesystem UUID: 71f9c82e-e231-4bc1-bed1-273f5f2ac2a623worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes24worker2 # Superblock backups stored on blocks:25worker1 # Filesystem UUID: 0da89c8e-3218-4e07-b6f2-6c5adc41e4d826worker2 # 32768, 98304, 163840, 22937627client: QEMU running (pid 49)28worker1 # Superblock backups stored on blocks:29worker2 # 30client # Disk image does not exist, creating the virtualisation disk image...31worker1: QEMU running (pid 50)32worker1 # 32768, 98304, 163840, 22937633worker2 # Allocating group tables: 0/8 done34worker1 # 35lb # Disk image does not exist, creating the virtualisation disk image...36lb: QEMU running (pid 51)37worker1 # Allocating group tables: 0/8 done38lb # Formatting '/build/vm-state-lb/tmp.LUN1SsED7L', fmt=raw size=107374182439worker2 # Writing inode tables: 0/8 done40client # Formatting '/build/vm-state-client/tmp.k5Mkxk6yf0', fmt=raw size=107374182441worker1 # Writing inode tables: 0/8 done42worker2 # Creating journal (8192 blocks): done43client # mke2fs 1.47.4 (6-Mar-2025)44worker1 # Creating journal (8192 blocks): done45worker2 # Writing superblocks and filesystem accounting information: 0/8 done46client # Discarding device blocks: 0/262144 done47worker1 # Writing superblocks and filesystem accounting information: 0/8 done48worker2 # 49client # Creating filesystem with 262144 4k blocks and 65536 inodes50worker1 # 51worker2 # Virtualisation disk image created.52client # Filesystem UUID: 5364948e-6f97-49bf-be67-41f7f7f6fc5453worker1 # Virtualisation disk image created.54worker2 # Starting virtiofs daemons...55client # Superblock backups stored on blocks:56worker1 # Starting virtiofs daemons...57worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)58client # 32768, 98304, 163840, 22937659worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether61client # 62worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether63worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...64client # Allocating group tables: 0/8 done65worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...66worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67client # Writing inode tables: 0/8 done68worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70client # Creating journal (8192 blocks): done71worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...73client # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...75worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76client # 77worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78worker2 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # Virtualisation disk image created.80worker1 # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...82client # Starting virtiofs daemons...83worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...84worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests85client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests87worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests88client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests90worker2 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests91client # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker1 # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests93(finished: start all VMs, in 0.22 seconds)94lb # mke2fs 1.47.4 (6-Mar-2025)95client: waiting for unit network-addresses-eth1.service96lb # Discarding device blocks: 0/262144 done97client: waiting for the VM to finish booting98lb # Creating filesystem with 262144 4k blocks and 65536 inodes99lb # Filesystem UUID: 9691f527-99d9-4fca-88fd-8047024c3083100lb # Superblock backups stored on blocks:101lb # 32768, 98304, 163840, 229376102lb # 103client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)104client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105client # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...106client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)107client # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether108client # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...109client # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests110client # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests111client # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-22T11:24:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-22T11:24:35Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-22T11:24:35Z INFO virtiofsd] Client connected, servicing requests131worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker2 # 136worker2 # 137worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # 139client # 140client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142worker2 # 143worker2 # 144worker2 # 145worker2 # 146lb # 147worker1 # 148lb # 149worker1 # 150client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151client # 152client # 153client # 154client # 155lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00156worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00157worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00158client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00159worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...160worker1 # 161worker1 # 162worker1 # 163worker1 # 164lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...165lb # 166lb # 167lb # 168lb # 169worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...170worker2 # 171worker2 # 172worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00173lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00174client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175client # 176client # 177worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178worker1 # 179worker1 # 180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183worker2 # Booting from ROM...184client # Booting from ROM...185lb # Booting from ROM...186worker1 # Booting from ROM...187lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g4s62ig20603prhr6rpvm83lvwh8ggjh-nixos-system-lb-test/init regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration console=ttyS0,115200n8 console=tty0190lb # [ 0.000000] BIOS-provided physical RAM map:191client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test/init regInfo=/nix/store/pp49lp3mrkpf817wwl4f76r0y0512f6i-closure-info/registration console=ttyS0,115200n8 console=tty0192lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable193client # [ 0.000000] BIOS-provided physical RAM map:194lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable196lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved198lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable199client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved200lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved201client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable202lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved203client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved204lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved205client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved206lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved207client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved208lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved209client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved210lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved211client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved212lb # [ 0.000000] NX (Execute Disable) protection: active213lb # [ 0.000000] APIC: Static calls initialized214worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026215client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved216lb # [ 0.000000] SMBIOS 2.8 present.217worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026218client # [ 0.000000] NX (Execute Disable) protection: active219client # [ 0.000000] APIC: Static calls initialized220lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014221client # [ 0.000000] SMBIOS 2.8 present.222lb # [ 0.000000] DMI: Memory slots populated: 1/1223lb # [ 0.000000] Hypervisor detected: KVM224client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014225lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000226client # [ 0.000000] DMI: Memory slots populated: 1/1227lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00228client # [ 0.000000] Hypervisor detected: KVM229worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7p3x1fr20awv6rqp4jyn18vhmavwjmq9-nixos-system-worker2-test/init regInfo=/nix/store/gn50nvbwg10sg3igwi322f48c4wpkc2v-closure-info/registration console=ttyS0,115200n8 console=tty0230worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a69286jdpp14g4cl1d6rckvg3a912f09-nixos-system-worker1-test/init regInfo=/nix/store/pxx1wbycjhhl6ikgpkzncw5i5x2n0awy-closure-info/registration console=ttyS0,115200n8 console=tty0231lb # [ 0.000000] kvm-clock: using sched offset of 446020489 cycles232worker2 # [ 0.000000] BIOS-provided physical RAM map:233worker1 # [ 0.000000] BIOS-provided physical RAM map:234client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000235client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00236worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable237worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable238lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns239client # [ 0.000000] kvm-clock: using sched offset of 447648601 cycles240lb # [ 0.000005] tsc: Detected 2400.012 MHz processor241worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved242worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved243lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000244worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved245client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns246worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved247client # [ 0.000005] tsc: Detected 2400.012 MHz processor248worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable249lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs250worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable251client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000252lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT253worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved254worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved255lb # [ 0.002772] found SMP MP-table at [mem 0x000f5450-0x000f545f]256client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs257worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved258worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved259lb # [ 0.002783] Using GB pages for direct mapping260client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT261worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved262lb # [ 0.002870] RAMDISK: [mem 0x5e36e000-0x5ffcffff]263worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved264client # [ 0.002756] found SMP MP-table at [mem 0x000f5450-0x000f545f]265lb # [ 0.002878] ACPI: Early table checksum verification disabled266worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved267worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved268client # [ 0.002767] Using GB pages for direct mapping269lb # [ 0.002881] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )270worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved271client # [ 0.002926] RAMDISK: [mem 0x5e370000-0x5ffcffff]272worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved273lb # [ 0.002886] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)274client # [ 0.002934] ACPI: Early table checksum verification disabled275worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved276worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved277worker1 # [ 0.000000] NX (Execute Disable) protection: active278client # [ 0.002937] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )279lb # [ 0.002890] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)280worker2 # [ 0.000000] NX (Execute Disable) protection: active281worker1 # [ 0.000000] APIC: Static calls initialized282worker1 # [ 0.000000] SMBIOS 2.8 present.283worker2 # [ 0.000000] APIC: Static calls initialized284client # [ 0.002941] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)285lb # [ 0.002898] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)286worker2 # [ 0.000000] SMBIOS 2.8 present.287lb # [ 0.002900] ACPI: FACS 0x000000005FFE0000 000040288worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014289client # [ 0.002946] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)290worker1 # [ 0.000000] DMI: Memory slots populated: 1/1291worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014292lb # [ 0.002901] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)293worker1 # [ 0.000000] Hypervisor detected: KVM294client # [ 0.002953] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)295worker2 # [ 0.000000] DMI: Memory slots populated: 1/1296worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000297client # [ 0.002955] ACPI: FACS 0x000000005FFE0000 000040298worker2 # [ 0.000000] Hypervisor detected: KVM299lb # [ 0.002903] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)300worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00301worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000302client # [ 0.002956] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)303lb # [ 0.002904] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)304worker1 # [ 0.000000] kvm-clock: using sched offset of 457726921 cycles305worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00306worker2 # [ 0.000001] kvm-clock: using sched offset of 462400349 cycles307client # [ 0.002958] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)308lb # [ 0.002906] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)309worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns310lb # [ 0.002907] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]311worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor312client # [ 0.002959] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)313worker2 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns314worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000315lb # [ 0.002908] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]316worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor317client # [ 0.002961] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)318worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs319lb # [ 0.002909] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]320worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000321client # [ 0.002962] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]322lb # [ 0.002909] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]323worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT324worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs325client # [ 0.002963] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]326worker1 # [ 0.002723] found SMP MP-table at [mem 0x000f5450-0x000f545f]327lb # [ 0.002910] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]328worker1 # [ 0.002734] Using GB pages for direct mapping329worker2 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT330client # [ 0.002964] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]331lb # [ 0.002910] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]332worker1 # [ 0.002822] RAMDISK: [mem 0x5e370000-0x5ffcffff]333worker2 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]334client # [ 0.002964] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]335worker1 # [ 0.002832] ACPI: Early table checksum verification disabled336lb # [ 0.002911] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]337worker2 # [ 0.002742] Using GB pages for direct mapping338lb # [ 0.003139] No NUMA configuration found339client # [ 0.002965] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]340worker2 # [ 0.002845] RAMDISK: [mem 0x5e36f000-0x5ffcffff]341worker1 # [ 0.002836] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )342lb # [ 0.003140] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]343worker2 # [ 0.002859] ACPI: Early table checksum verification disabled344client # [ 0.002965] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]345worker1 # [ 0.002840] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)346lb # [ 0.003142] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]347worker2 # [ 0.002864] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )348lb # [ 0.005312] Zone ranges:349client # [ 0.002966] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]350worker1 # [ 0.002844] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)351client # [ 0.003186] No NUMA configuration found352lb # [ 0.005312] DMA [mem 0x0000000000001000-0x0000000000ffffff]353worker2 # [ 0.002868] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)354client # [ 0.003187] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]355worker1 # [ 0.002851] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)356lb # [ 0.005314] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]357lb # [ 0.005315] Normal empty358worker2 # [ 0.002872] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)359worker1 # [ 0.002853] ACPI: FACS 0x000000005FFE0000 000040360client # [ 0.003189] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]361lb # [ 0.005316] Device empty362client # [ 0.005436] Zone ranges:363lb # [ 0.005316] Movable zone start for each node364worker1 # [ 0.002855] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)365lb # [ 0.005317] Early memory node ranges366worker2 # [ 0.002880] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)367client # [ 0.005437] DMA [mem 0x0000000000001000-0x0000000000ffffff]368worker2 # [ 0.002882] ACPI: FACS 0x000000005FFE0000 000040369lb # [ 0.005317] node 0: [mem 0x0000000000001000-0x000000000009efff]370client # [ 0.005439] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]371worker1 # [ 0.002856] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)372client # [ 0.005440] Normal empty373lb # [ 0.005318] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]374client # [ 0.005441] Device empty375worker2 # [ 0.002883] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)376worker1 # [ 0.002858] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)377client # [ 0.005441] Movable zone start for each node378lb # [ 0.005320] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]379client # [ 0.005442] Early memory node ranges380worker2 # [ 0.002885] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)381worker1 # [ 0.002859] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)382lb # [ 0.005339] On node 0, zone DMA: 1 pages in unavailable ranges383client # [ 0.005442] node 0: [mem 0x0000000000001000-0x000000000009efff]384lb # [ 0.005590] On node 0, zone DMA: 97 pages in unavailable ranges385worker1 # [ 0.002861] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]386worker2 # [ 0.002886] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)387client # [ 0.005443] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]388lb # [ 0.031453] On node 0, zone DMA32: 40 pages in unavailable ranges389worker1 # [ 0.002862] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]390lb # [ 0.031911] ACPI: PM-Timer IO Port: 0x608391worker2 # [ 0.002888] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)392client # [ 0.005444] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]393worker1 # [ 0.002862] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]394lb # [ 0.031922] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])395client # [ 0.005470] On node 0, zone DMA: 1 pages in unavailable ranges396worker2 # [ 0.002889] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]397worker1 # [ 0.002863] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]398lb # [ 0.031950] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23399client # [ 0.005730] On node 0, zone DMA: 97 pages in unavailable ranges400worker2 # [ 0.002890] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]401worker1 # [ 0.002863] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]402lb # [ 0.031952] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)403client # [ 0.031412] On node 0, zone DMA32: 40 pages in unavailable ranges404worker2 # [ 0.002891] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]405client # [ 0.031873] ACPI: PM-Timer IO Port: 0x608406worker1 # [ 0.002864] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]407lb # [ 0.031954] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)408client # [ 0.031885] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])409worker2 # [ 0.002891] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]410lb # [ 0.031955] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)411worker1 # [ 0.002864] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]412worker1 # [ 0.003090] No NUMA configuration found413client # [ 0.031909] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23414worker2 # [ 0.002892] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]415lb # [ 0.031956] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)416worker1 # [ 0.003091] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]417client # [ 0.031912] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)418worker2 # [ 0.002892] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]419lb # [ 0.031957] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)420worker1 # [ 0.003093] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]421client # [ 0.031913] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)422worker1 # [ 0.005397] Zone ranges:423lb # [ 0.031959] ACPI: Using ACPI (MADT) for SMP configuration information424worker2 # [ 0.002892] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]425client # [ 0.031915] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)426lb # [ 0.031960] ACPI: HPET id: 0x8086a201 base: 0xfed00000427worker1 # [ 0.005398] DMA [mem 0x0000000000001000-0x0000000000ffffff]428worker2 # [ 0.003142] No NUMA configuration found429lb # [ 0.031964] TSC deadline timer available430worker1 # [ 0.005399] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]431client # [ 0.031916] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)432worker2 # [ 0.003142] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]433lb # [ 0.031967] CPU topo: Max. logical packages: 1434worker1 # [ 0.005401] Normal empty435worker1 # [ 0.005402] Device empty436lb # [ 0.031968] CPU topo: Max. logical dies: 1437client # [ 0.031916] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)438worker2 # [ 0.003145] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]439worker1 # [ 0.005402] Movable zone start for each node440worker2 # [ 0.006391] Zone ranges:441lb # [ 0.031968] CPU topo: Max. dies per package: 1442worker1 # [ 0.005403] Early memory node ranges443client # [ 0.031919] ACPI: Using ACPI (MADT) for SMP configuration information444lb # [ 0.031972] CPU topo: Max. threads per core: 1445worker2 # [ 0.006392] DMA [mem 0x0000000000001000-0x0000000000ffffff]446client # [ 0.031920] ACPI: HPET id: 0x8086a201 base: 0xfed00000447worker1 # [ 0.005403] node 0: [mem 0x0000000000001000-0x000000000009efff]448lb # [ 0.031972] CPU topo: Num. cores per package: 1449worker2 # [ 0.006394] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]450client # [ 0.031924] TSC deadline timer available451lb # [ 0.031973] CPU topo: Num. threads per package: 1452worker1 # [ 0.005404] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]453worker2 # [ 0.006395] Normal empty454client # [ 0.031928] CPU topo: Max. logical packages: 1455worker2 # [ 0.006396] Device empty456lb # [ 0.031973] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs457worker1 # [ 0.005405] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]458client # [ 0.031928] CPU topo: Max. logical dies: 1459worker2 # [ 0.006397] Movable zone start for each node460worker2 # [ 0.006398] Early memory node ranges461client # [ 0.031929] CPU topo: Max. dies per package: 1462lb # [ 0.031994] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()463worker1 # [ 0.005425] On node 0, zone DMA: 1 pages in unavailable ranges464client # [ 0.031932] CPU topo: Max. threads per core: 1465worker1 # [ 0.005696] On node 0, zone DMA: 97 pages in unavailable ranges466worker2 # [ 0.006398] node 0: [mem 0x0000000000001000-0x000000000009efff]467lb # [ 0.032025] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]468client # [ 0.031933] CPU topo: Num. cores per package: 1469worker1 # [ 0.034947] On node 0, zone DMA32: 40 pages in unavailable ranges470worker2 # [ 0.006399] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]471client # [ 0.031933] CPU topo: Num. threads per package: 1472lb # [ 0.032027] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]473worker1 # [ 0.035413] ACPI: PM-Timer IO Port: 0x608474worker2 # [ 0.006400] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]475client # [ 0.031933] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs476lb # [ 0.032028] [mem 0x60000000-0xafffffff] available for PCI devices477worker1 # [ 0.035425] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])478lb # [ 0.032030] Booting paravirtualized kernel on KVM479worker2 # [ 0.006423] On node 0, zone DMA: 1 pages in unavailable ranges480client # [ 0.031953] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()481worker1 # [ 0.035449] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23482worker2 # [ 0.006775] On node 0, zone DMA: 97 pages in unavailable ranges483worker1 # [ 0.035452] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)484client # [ 0.031983] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]485lb # [ 0.032033] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns486worker2 # [ 0.040190] On node 0, zone DMA32: 40 pages in unavailable ranges487worker1 # [ 0.035453] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)488client # [ 0.031985] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]489worker2 # [ 0.040716] ACPI: PM-Timer IO Port: 0x608490lb # [ 0.036504] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1491worker1 # [ 0.035454] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)492client # [ 0.031987] [mem 0x60000000-0xafffffff] available for PCI devices493lb # [ 0.038582] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152494worker2 # [ 0.040729] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])495worker1 # [ 0.035456] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)496client # [ 0.031988] Booting paravirtualized kernel on KVM497lb # [ 0.038625] kvm-guest: PV spinlocks disabled, single CPU498worker2 # [ 0.040759] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23499worker1 # [ 0.035456] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)500worker2 # [ 0.040762] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)501client # [ 0.031991] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns502worker1 # [ 0.035459] ACPI: Using ACPI (MADT) for SMP configuration information503worker2 # [ 0.040763] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)504worker1 # [ 0.035460] ACPI: HPET id: 0x8086a201 base: 0xfed00000505client # [ 0.036478] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1506worker1 # [ 0.035464] TSC deadline timer available507worker2 # [ 0.040764] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)508client # [ 0.038569] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152509worker1 # [ 0.035468] CPU topo: Max. logical packages: 1510worker2 # [ 0.040765] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)511worker1 # [ 0.035469] CPU topo: Max. logical dies: 1512lb # [ 0.038627] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g4s62ig20603prhr6rpvm83lvwh8ggjh-nixos-system-lb-test/init regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration console=ttyS0,115200n8 console=tty0513client # [ 0.038608] kvm-guest: PV spinlocks disabled, single CPU514worker1 # [ 0.035469] CPU topo: Max. dies per package: 1515worker2 # [ 0.040766] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)516worker1 # [ 0.035472] CPU topo: Max. threads per core: 1517worker1 # [ 0.035473] CPU topo: Num. cores per package: 1518worker2 # [ 0.040768] ACPI: Using ACPI (MADT) for SMP configuration information519lb # [ 0.038720] Unknown kernel command line parameters "regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration", will be passed to user space.520lb # [ 0.038733] random: crng init done521worker1 # [ 0.035473] CPU topo: Num. threads per package: 1522worker2 # [ 0.040770] ACPI: HPET id: 0x8086a201 base: 0xfed00000523worker2 # [ 0.040773] TSC deadline timer available524worker1 # [ 0.035473] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs525lb # [ 0.038734] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes526worker2 # [ 0.040778] CPU topo: Max. logical packages: 1527worker1 # [ 0.035494] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()528client # [ 0.038609] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test/init regInfo=/nix/store/pp49lp3mrkpf817wwl4f76r0y0512f6i-closure-info/registration console=ttyS0,115200n8 console=tty0529lb # [ 0.040778] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)530worker2 # [ 0.040778] CPU topo: Max. logical dies: 1531worker2 # [ 0.040779] CPU topo: Max. dies per package: 1532worker1 # [ 0.035524] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]533lb # [ 0.042094] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)534worker2 # [ 0.040782] CPU topo: Max. threads per core: 1535lb # [ 0.042127] Fallback order for Node 0: 0536worker1 # [ 0.035526] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]537client # [ 0.038704] Unknown kernel command line parameters "regInfo=/nix/store/pp49lp3mrkpf817wwl4f76r0y0512f6i-closure-info/registration", will be passed to user space.538worker2 # [ 0.040782] CPU topo: Num. cores per package: 1539client # [ 0.038717] random: crng init done540lb # [ 0.042130] Built 1 zonelists, mobility grouping on. Total pages: 393078541worker1 # [ 0.035527] [mem 0x60000000-0xafffffff] available for PCI devices542worker2 # [ 0.040783] CPU topo: Num. threads per package: 1543lb # [ 0.042130] Policy zone: DMA32544worker1 # [ 0.035529] Booting paravirtualized kernel on KVM545client # [ 0.038717] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes546worker2 # [ 0.040783] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs547lb # [ 0.044658] mem auto-init: stack:all(zero), heap alloc:on, heap free:off548client # [ 0.040775] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549lb # [ 0.047911] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1550worker1 # [ 0.035532] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns551worker2 # [ 0.040802] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()552lb # [ 0.051249] allocated 3145728 bytes of page_ext553client # [ 0.041973] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)554worker1 # [ 0.040007] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1555worker2 # [ 0.040832] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]556lb # [ 0.060865] ftrace: allocating 48787 entries in 192 pages557client # [ 0.042004] Fallback order for Node 0: 0558lb # [ 0.060867] ftrace: allocated 192 pages with 2 groups559worker1 # [ 0.042147] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152560worker2 # [ 0.040833] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]561client # [ 0.042006] Built 1 zonelists, mobility grouping on. Total pages: 393078562lb # [ 0.061668] Dynamic Preempt: lazy563worker1 # [ 0.042188] kvm-guest: PV spinlocks disabled, single CPU564client # [ 0.042007] Policy zone: DMA32565worker2 # [ 0.040835] [mem 0x60000000-0xafffffff] available for PCI devices566lb # [ 0.061789] rcu: Preemptible hierarchical RCU implementation.567worker2 # [ 0.040836] Booting paravirtualized kernel on KVM568client # [ 0.044448] mem auto-init: stack:all(zero), heap alloc:on, heap free:off569lb # [ 0.061789] rcu: RCU event tracing is enabled.570client # [ 0.047667] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1571lb # [ 0.061790] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.572worker2 # [ 0.040840] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns573client # [ 0.050886] allocated 3145728 bytes of page_ext574lb # [ 0.061791] Trampoline variant of Tasks RCU enabled.575client # [ 0.060548] ftrace: allocating 48787 entries in 192 pages576lb # [ 0.061792] Rude variant of Tasks RCU enabled.577worker2 # [ 0.045320] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1578worker1 # [ 0.042190] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a69286jdpp14g4cl1d6rckvg3a912f09-nixos-system-worker1-test/init regInfo=/nix/store/pxx1wbycjhhl6ikgpkzncw5i5x2n0awy-closure-info/registration console=ttyS0,115200n8 console=tty0579client # [ 0.060550] ftrace: allocated 192 pages with 2 groups580lb # [ 0.061792] Tracing variant of Tasks RCU enabled.581client # [ 0.061351] Dynamic Preempt: lazy582worker2 # [ 0.047607] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152583lb # [ 0.061793] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.584client # [ 0.061540] rcu: Preemptible hierarchical RCU implementation.585worker2 # [ 0.047652] kvm-guest: PV spinlocks disabled, single CPU586worker1 # [ 0.042286] Unknown kernel command line parameters "regInfo=/nix/store/pxx1wbycjhhl6ikgpkzncw5i5x2n0awy-closure-info/registration", will be passed to user space.587client # [ 0.061540] rcu: RCU event tracing is enabled.588lb # [ 0.061793] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1589worker1 # [ 0.042299] random: crng init done590client # [ 0.061541] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.591lb # [ 0.061844] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.592worker1 # [ 0.042300] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes593client # [ 0.061543] Trampoline variant of Tasks RCU enabled.594worker1 # [ 0.044402] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)595client # [ 0.061543] Rude variant of Tasks RCU enabled.596lb # [ 0.061846] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.597client # [ 0.061543] Tracing variant of Tasks RCU enabled.598worker1 # [ 0.045654] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)599worker2 # [ 0.047654] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7p3x1fr20awv6rqp4jyn18vhmavwjmq9-nixos-system-worker2-test/init regInfo=/nix/store/gn50nvbwg10sg3igwi322f48c4wpkc2v-closure-info/registration console=ttyS0,115200n8 console=tty0600worker1 # [ 0.045693] Fallback order for Node 0: 0601lb # [ 0.061847] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.602client # [ 0.061544] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.603lb # [ 0.066388] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16604worker1 # [ 0.045696] Built 1 zonelists, mobility grouping on. Total pages: 393078605client # [ 0.061545] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1606worker1 # [ 0.045697] Policy zone: DMA32607lb # [ 0.066664] rcu: srcu_init: Setting srcu_struct sizes based on contention.608worker2 # [ 0.047749] Unknown kernel command line parameters "regInfo=/nix/store/gn50nvbwg10sg3igwi322f48c4wpkc2v-closure-info/registration", will be passed to user space.609worker1 # [ 0.048174] mem auto-init: stack:all(zero), heap alloc:on, heap free:off610client # [ 0.061559] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.611worker2 # [ 0.047761] random: crng init done612worker1 # [ 0.051403] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1613lb # [ 0.066671] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns614worker2 # [ 0.047762] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes615worker1 # [ 0.054711] allocated 3145728 bytes of page_ext616client # [ 0.061561] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617worker1 # [ 0.064415] ftrace: allocating 48787 entries in 192 pages618lb # [ 0.066774] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)619worker2 # [ 0.049894] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)620worker1 # [ 0.064416] ftrace: allocated 192 pages with 2 groups621lb # [ 0.070327] Console: colour VGA+ 80x25622client # [ 0.061562] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker1 # [ 0.065249] Dynamic Preempt: lazy624lb # [ 0.070330] printk: legacy console [tty0] enabled625worker2 # [ 0.050946] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)626client # [ 0.066071] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16627worker1 # [ 0.065376] rcu: Preemptible hierarchical RCU implementation.628lb # [ 0.099900] printk: legacy console [ttyS0] enabled629worker2 # [ 0.050977] Fallback order for Node 0: 0630worker1 # [ 0.065377] rcu: RCU event tracing is enabled.631client # [ 0.066327] rcu: srcu_init: Setting srcu_struct sizes based on contention.632lb # [ 0.196610] ACPI: Core revision 20250807633worker2 # [ 0.050980] Built 1 zonelists, mobility grouping on. Total pages: 393078634worker1 # [ 0.065378] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.635worker2 # [ 0.050981] Policy zone: DMA32636client # [ 0.066333] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns637worker1 # [ 0.065379] Trampoline variant of Tasks RCU enabled.638worker2 # [ 0.053742] mem auto-init: stack:all(zero), heap alloc:on, heap free:off639worker1 # [ 0.065379] Rude variant of Tasks RCU enabled.640lb # [ 0.197492] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns641worker1 # [ 0.065379] Tracing variant of Tasks RCU enabled.642worker2 # [ 0.057024] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1643client # [ 0.066433] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)644lb # [ 0.199056] APIC: Switch to symmetric I/O mode setup645worker2 # [ 0.060359] allocated 3145728 bytes of page_ext646client # [ 0.069980] Console: colour VGA+ 80x25647worker1 # [ 0.065380] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.648lb # [ 0.200107] x2apic enabled649client # [ 0.069982] printk: legacy console [tty0] enabled650worker2 # [ 0.070168] ftrace: allocating 48787 entries in 192 pages651worker1 # [ 0.065381] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1652client # [ 0.100375] printk: legacy console [ttyS0] enabled653worker2 # [ 0.070170] ftrace: allocated 192 pages with 2 groups654worker2 # [ 0.071014] Dynamic Preempt: lazy655lb # [ 0.200861] APIC: Switched APIC routing to: physical x2apic656worker1 # [ 0.065429] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657client # [ 0.200774] ACPI: Core revision 20250807658worker2 # [ 0.071201] rcu: Preemptible hierarchical RCU implementation.659worker2 # [ 0.071202] rcu: RCU event tracing is enabled.660worker1 # [ 0.065431] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.661lb # [ 0.202849] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1662worker2 # [ 0.071202] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.663client # [ 0.201756] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns664worker1 # [ 0.065432] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.665worker2 # [ 0.071204] Trampoline variant of Tasks RCU enabled.666client # [ 0.203383] APIC: Switch to symmetric I/O mode setup667worker1 # [ 0.070012] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16668worker2 # [ 0.071205] Rude variant of Tasks RCU enabled.669client # [ 0.204411] x2apic enabled670lb # [ 0.203869] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns671worker1 # [ 0.070289] rcu: srcu_init: Setting srcu_struct sizes based on contention.672worker2 # [ 0.071205] Tracing variant of Tasks RCU enabled.673worker2 # [ 0.071206] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.674client # [ 0.205194] APIC: Switched APIC routing to: physical x2apic675worker1 # [ 0.070295] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns676lb # [ 0.205576] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)677worker2 # [ 0.071206] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1678worker1 # [ 0.070402] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)679lb # [ 0.206666] x86/cpu: User Mode Instruction Prevention (UMIP) activated680worker1 # [ 0.073951] Console: colour VGA+ 80x25681worker2 # [ 0.071216] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.682client # [ 0.207158] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1683worker1 # [ 0.073954] printk: legacy console [tty0] enabled684lb # [ 0.208709] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127685worker1 # [ 0.103699] printk: legacy console [ttyS0] enabled686worker2 # [ 0.071217] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.687lb # [ 0.209574] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0688worker1 # [ 0.199289] ACPI: Core revision 20250807689client # [ 0.208211] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns690worker2 # [ 0.071218] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.691worker2 # [ 0.075623] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16692client # [ 0.209951] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)693worker1 # [ 0.200173] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns694lb # [ 0.210577] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto695worker2 # [ 0.075891] rcu: srcu_init: Setting srcu_struct sizes based on contention.696worker1 # [ 0.201723] APIC: Switch to symmetric I/O mode setup697client # [ 0.212048] x86/cpu: User Mode Instruction Prevention (UMIP) activated698worker1 # [ 0.202653] x2apic enabled699lb # [ 0.211574] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl700worker2 # [ 0.075899] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns701client # [ 0.213104] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127702lb # [ 0.212573] Transient Scheduler Attacks: Vulnerable: No microcode703worker1 # [ 0.203402] APIC: Switched APIC routing to: physical x2apic704worker2 # [ 0.076000] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)705client # [ 0.213949] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0706lb # [ 0.213573] Spectre V2 : Mitigation: Enhanced / Automatic IBRS707worker2 # [ 0.079926] Console: colour VGA+ 80x25708worker2 # [ 0.079928] printk: legacy console [tty0] enabled709worker1 # [ 0.205167] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1710lb # [ 0.214573] Speculative Return Stack Overflow: Mitigation: Safe RET711worker2 # [ 0.110700] printk: legacy console [ttyS0] enabled712client # [ 0.214952] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto713worker2 # [ 0.214151] ACPI: Core revision 20250807714worker1 # [ 0.206122] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns715client # [ 0.215949] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl716worker2 # [ 0.215089] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns717worker1 # [ 0.207776] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)718client # [ 0.216948] Transient Scheduler Attacks: Vulnerable: No microcode719worker2 # [ 0.216696] APIC: Switch to symmetric I/O mode setup720worker1 # [ 0.208861] x86/cpu: User Mode Instruction Prevention (UMIP) activated721client # [ 0.217948] Spectre V2 : Mitigation: Enhanced / Automatic IBRS722lb # [ 0.215573] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization723worker2 # [ 0.217738] x2apic enabled724worker1 # [ 0.209902] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127725lb # [ 0.217580] Spectre V2 : Enabling IBPB for BPF726client # [ 0.218948] Speculative Return Stack Overflow: Mitigation: Safe RET727worker2 # [ 0.218507] APIC: Switched APIC routing to: physical x2apic728worker1 # [ 0.210773] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0729lb # [ 0.219112] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier730lb # [ 0.219574] active return thunk: srso_alias_return_thunk731worker2 # [ 0.220441] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1732worker1 # [ 0.211776] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto733lb # [ 0.220582] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'734worker1 # [ 0.212773] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl735client # [ 0.219948] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization736worker2 # [ 0.221501] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns737lb # [ 0.221573] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'738client # [ 0.221957] Spectre V2 : Enabling IBPB for BPF739worker1 # [ 0.213773] Transient Scheduler Attacks: Vulnerable: No microcode740worker2 # [ 0.223231] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)741lb # [ 0.222573] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'742worker1 # [ 0.214772] Spectre V2 : Mitigation: Enhanced / Automatic IBRS743client # [ 0.223505] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier744lb # [ 0.223573] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'745worker2 # [ 0.225276] x86/cpu: User Mode Instruction Prevention (UMIP) activated746worker1 # [ 0.215773] Speculative Return Stack Overflow: Mitigation: Safe RET747client # [ 0.224950] active return thunk: srso_alias_return_thunk748lb # [ 0.224573] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'749worker2 # [ 0.226358] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127750client # [ 0.225978] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'751lb # [ 0.225573] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752worker2 # [ 0.227228] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0753client # [ 0.226949] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'754lb # [ 0.226573] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'755client # [ 0.227948] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'756worker1 # [ 0.216773] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization757worker2 # [ 0.228232] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto758lb # [ 0.227574] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256759worker1 # [ 0.218781] Spectre V2 : Enabling IBPB for BPF760client # [ 0.228948] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'761worker2 # [ 0.229228] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl762lb # [ 0.228573] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64763client # [ 0.229948] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'764worker1 # [ 0.219774] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier765worker2 # [ 0.230228] Transient Scheduler Attacks: Vulnerable: No microcode766lb # [ 0.229573] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512767worker1 # [ 0.220774] active return thunk: srso_alias_return_thunk768client # [ 0.230949] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'769worker2 # [ 0.231227] Spectre V2 : Mitigation: Enhanced / Automatic IBRS770lb # [ 0.230573] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024771worker1 # [ 0.221794] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'772client # [ 0.231948] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'773lb # [ 0.231573] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8774worker2 # [ 0.232228] Speculative Return Stack Overflow: Mitigation: Safe RET775worker1 # [ 0.222773] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'776client # [ 0.233949] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256777lb # [ 0.232573] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.778worker1 # [ 0.223772] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'779client # [ 0.234948] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64780worker1 # [ 0.224773] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'781client # [ 0.235948] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512782worker1 # [ 0.225772] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'783client # [ 0.236948] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024784worker2 # [ 0.233227] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization785worker1 # [ 0.226773] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'786client # [ 0.237948] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8787worker2 # [ 0.235235] Spectre V2 : Enabling IBPB for BPF788worker1 # [ 0.227772] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'789client # [ 0.238905] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.790worker2 # [ 0.236817] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier791worker1 # [ 0.229682] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256792worker2 # [ 0.238228] active return thunk: srso_alias_return_thunk793worker1 # [ 0.230500] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64794worker2 # [ 0.239251] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'795worker1 # [ 0.231506] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512796worker1 # [ 0.232508] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024797worker2 # [ 0.240228] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'798worker1 # [ 0.233495] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8799worker2 # [ 0.241227] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'800worker2 # [ 0.242228] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'801worker1 # [ 0.234496] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.802worker2 # [ 0.243227] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803worker2 # [ 0.244227] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804worker2 # [ 0.245227] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805worker2 # [ 0.247228] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806worker2 # [ 0.248227] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker2 # [ 0.249227] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker2 # [ 0.250227] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker2 # [ 0.251228] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker2 # [ 0.252227] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811lb # [ 0.266180] Freeing SMP alternatives memory: 44K812lb # [ 0.266576] pid_max: default: 32768 minimum: 301813lb # [ 0.267657] LSM: initializing lsm=capability,landlock,yama,bpf,ima814lb # [ 0.268680] landlock: Up and running.815lb # [ 0.269291] Yama: becoming mindful.816lb # [ 0.270188] LSM support for eBPF active817client # [ 0.272783] Freeing SMP alternatives memory: 44K818lb # [ 0.271134] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819client # [ 0.273531] pid_max: default: 32768 minimum: 301820lb # [ 0.271615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)821client # [ 0.274035] LSM: initializing lsm=capability,landlock,yama,bpf,ima822client # [ 0.275048] landlock: Up and running.823client # [ 0.275949] Yama: becoming mindful.824client # [ 0.277072] LSM support for eBPF active825lb # [ 0.274897] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)826client # [ 0.277826] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)827lb # [ 0.276176] Performance Events: Fam17h+ core perfctr, AMD PMU driver.828worker1 # [ 0.268774] Freeing SMP alternatives memory: 44K829lb # [ 0.276582] ... version: 2830client # [ 0.278987] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)831worker1 # [ 0.269553] pid_max: default: 32768 minimum: 301832lb # [ 0.277328] ... bit width: 48833worker1 # [ 0.269883] LSM: initializing lsm=capability,landlock,yama,bpf,ima834lb # [ 0.277576] ... generic counters: 6835worker1 # [ 0.270882] landlock: Up and running.836lb # [ 0.278288] ... generic bitmap: 000000000000003f837client # [ 0.281396] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)838worker1 # [ 0.271773] Yama: becoming mindful.839lb # [ 0.278576] ... fixed-purpose counters: 0840worker1 # [ 0.272984] LSM support for eBPF active841lb # [ 0.279331] ... fixed-purpose bitmap: 0000000000000000842client # [ 0.282529] Performance Events: Fam17h+ core perfctr, AMD PMU driver.843client # [ 0.282959] ... version: 2844worker1 # [ 0.273711] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)845lb # [ 0.279576] ... value mask: 0000ffffffffffff846client # [ 0.283687] ... bit width: 48847lb # [ 0.280494] ... max period: 00007fffffffffff848worker1 # [ 0.274814] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849client # [ 0.283951] ... generic counters: 6850lb # [ 0.281271] ... global_ctrl mask: 000000000000003f851client # [ 0.284668] ... generic bitmap: 000000000000003f852lb # [ 0.281680] signal: max sigframe size: 3376853client # [ 0.284951] ... fixed-purpose counters: 0854lb # [ 0.282514] rcu: Hierarchical SRCU implementation.855worker1 # [ 0.277244] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)856client # [ 0.285679] ... fixed-purpose bitmap: 0000000000000000857lb # [ 0.283154] rcu: Max phase no-delay instances is 400.858worker2 # [ 0.286752] Freeing SMP alternatives memory: 44K859client # [ 0.285951] ... value mask: 0000ffffffffffff860worker1 # [ 0.278343] Performance Events: Fam17h+ core perfctr, AMD PMU driver.861worker2 # [ 0.287231] pid_max: default: 32768 minimum: 301862client # [ 0.286887] ... max period: 00007fffffffffff863worker1 # [ 0.278778] ... version: 2864worker2 # [ 0.288332] LSM: initializing lsm=capability,landlock,yama,bpf,ima865client # [ 0.287624] ... global_ctrl mask: 000000000000003f866worker1 # [ 0.279506] ... bit width: 48867worker2 # [ 0.289333] landlock: Up and running.868worker1 # [ 0.279775] ... generic counters: 6869client # [ 0.288064] signal: max sigframe size: 3376870worker2 # [ 0.290228] Yama: becoming mindful.871client # [ 0.288847] rcu: Hierarchical SRCU implementation.872worker1 # [ 0.280480] ... generic bitmap: 000000000000003f873worker2 # [ 0.291383] LSM support for eBPF active874worker1 # [ 0.280782] ... fixed-purpose counters: 0875client # [ 0.289517] rcu: Max phase no-delay instances is 400.876lb # [ 0.288176] smp: Bringing up secondary CPUs ...877worker1 # [ 0.281506] ... fixed-purpose bitmap: 0000000000000000878worker2 # [ 0.292210] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)879lb # [ 0.288589] smp: Brought up 1 node, 1 CPU880worker1 # [ 0.281775] ... value mask: 0000ffffffffffff881lb # [ 0.289234] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)882worker1 # [ 0.282683] ... max period: 00007fffffffffff883worker2 # [ 0.293265] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)884worker1 # [ 0.283426] ... global_ctrl mask: 000000000000003f885worker1 # [ 0.283878] signal: max sigframe size: 3376886lb # [ 0.289784] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)887worker1 # [ 0.284689] rcu: Hierarchical SRCU implementation.888worker2 # [ 0.295963] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)889client # [ 0.294524] smp: Bringing up secondary CPUs ...890lb # [ 0.290824] devtmpfs: initialized891worker1 # [ 0.285333] rcu: Max phase no-delay instances is 400.892client # [ 0.294965] smp: Brought up 1 node, 1 CPU893lb # [ 0.291670] x86/mm: Memory block size: 128MB894worker2 # [ 0.296790] Performance Events: Fam17h+ core perfctr, AMD PMU driver.895client # [ 0.295670] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)896worker2 # [ 0.297232] ... version: 2897worker2 # [ 0.298025] ... bit width: 48898lb # [ 0.293487] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)899worker2 # [ 0.298230] ... generic counters: 6900client # [ 0.296168] Memory: 1454608K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)901lb # [ 0.294488] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).902worker2 # [ 0.299021] ... generic bitmap: 000000000000003f903client # [ 0.297173] devtmpfs: initialized904worker2 # [ 0.299991] ... fixed-purpose counters: 0905lb # [ 0.295673] pinctrl core: initialized pinctrl subsystem906worker1 # [ 0.290335] smp: Bringing up secondary CPUs ...907client # [ 0.298089] x86/mm: Memory block size: 128MB908worker1 # [ 0.290788] smp: Brought up 1 node, 1 CPU909worker2 # [ 0.300230] ... fixed-purpose bitmap: 0000000000000000910lb # [ 0.296842] PM: RTC time: 11:24:35, date: 2026-09-22911worker2 # [ 0.301194] ... value mask: 0000ffffffffffff912worker1 # [ 0.291481] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)913worker2 # [ 0.301988] ... max period: 00007fffffffffff914client # [ 0.299979] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)915worker2 # [ 0.302230] ... global_ctrl mask: 000000000000003f916worker1 # [ 0.291969] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)917client # [ 0.300976] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).918worker2 # [ 0.303350] signal: max sigframe size: 3376919worker1 # [ 0.293041] devtmpfs: initialized920lb # [ 0.300553] NET: Registered PF_NETLINK/PF_ROUTE protocol family921client # [ 0.302034] pinctrl core: initialized pinctrl subsystem922worker2 # [ 0.304182] rcu: Hierarchical SRCU implementation.923worker1 # [ 0.293865] x86/mm: Memory block size: 128MB924worker2 # [ 0.304843] rcu: Max phase no-delay instances is 400.925client # [ 0.303168] PM: RTC time: 11:24:35, date: 2026-09-22926lb # [ 0.301804] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations927lb # [ 0.302602] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations928worker1 # [ 0.295605] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)929worker1 # [ 0.296667] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).930lb # [ 0.303860] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations931client # [ 0.306683] NET: Registered PF_NETLINK/PF_ROUTE protocol family932worker1 # [ 0.297858] pinctrl core: initialized pinctrl subsystem933lb # [ 0.304587] audit: initializing netlink subsys (disabled)934worker1 # [ 0.299020] PM: RTC time: 11:24:35, date: 2026-09-22935lb # [ 0.305719] thermal_sys: Registered thermal governor 'fair_share'936worker2 # [ 0.309870] smp: Bringing up secondary CPUs ...937client # [ 0.307445] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations938worker2 # [ 0.310244] smp: Brought up 1 node, 1 CPU939lb # [ 0.305721] thermal_sys: Registered thermal governor 'bang_bang'940client # [ 0.307975] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations941worker2 # [ 0.310960] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)942lb # [ 0.306577] thermal_sys: Registered thermal governor 'step_wise'943lb # [ 0.307558] thermal_sys: Registered thermal governor 'user_space'944client # [ 0.309241] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations945worker1 # [ 0.302514] NET: Registered PF_NETLINK/PF_ROUTE protocol family946client # [ 0.309961] audit: initializing netlink subsys (disabled)947worker2 # [ 0.311439] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)948lb # [ 0.308364] audit: type=2000 audit(1790076276.208:1): state=initialized audit_enabled=0 res=1949worker1 # [ 0.303367] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations950client # [ 0.311195] thermal_sys: Registered thermal governor 'fair_share'951lb # [ 0.310579] thermal_sys: Registered thermal governor 'power_allocator'952worker2 # [ 0.312765] devtmpfs: initialized953lb # [ 0.310593] cpuidle: using governor menu954worker1 # [ 0.303799] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations955client # [ 0.311197] thermal_sys: Registered thermal governor 'bang_bang'956worker2 # [ 0.313472] x86/mm: Memory block size: 128MB957client # [ 0.311953] thermal_sys: Registered thermal governor 'step_wise'958lb # [ 0.312844] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5959worker1 # [ 0.305043] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations960worker1 # [ 0.305785] audit: initializing netlink subsys (disabled)961worker2 # [ 0.315264] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)962client # [ 0.312954] audit: type=2000 audit(1790076276.210:1): state=initialized audit_enabled=0 res=1963lb # [ 0.313837] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]964client # [ 0.314952] thermal_sys: Registered thermal governor 'user_space'965worker1 # [ 0.306938] thermal_sys: Registered thermal governor 'fair_share'966worker2 # [ 0.316261] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).967lb # [ 0.314579] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry968worker1 # [ 0.306940] thermal_sys: Registered thermal governor 'bang_bang'969client # [ 0.314954] thermal_sys: Registered thermal governor 'power_allocator'970worker2 # [ 0.317320] pinctrl core: initialized pinctrl subsystem971lb # [ 0.315587] PCI: Using configuration type 1 for base access972client # [ 0.315978] cpuidle: using governor menu973worker1 # [ 0.307776] thermal_sys: Registered thermal governor 'step_wise'974worker2 # [ 0.318477] PM: RTC time: 11:24:35, date: 2026-09-22975worker1 # [ 0.308717] thermal_sys: Registered thermal governor 'user_space'976lb # [ 0.316698] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.977client # [ 0.318273] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5978worker1 # [ 0.309526] audit: type=2000 audit(1790076276.220:1): state=initialized audit_enabled=0 res=1979client # [ 0.319200] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]980worker1 # [ 0.310778] thermal_sys: Registered thermal governor 'power_allocator'981worker1 # [ 0.310791] cpuidle: using governor menu982client # [ 0.319954] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry983worker2 # [ 0.321977] NET: Registered PF_NETLINK/PF_ROUTE protocol family984client # [ 0.320962] PCI: Using configuration type 1 for base access985worker1 # [ 0.312959] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5986worker2 # [ 0.322728] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations987lb # [ 0.321819] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages988client # [ 0.322114] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.989worker1 # [ 0.314001] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]990worker2 # [ 0.323253] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations991lb # [ 0.322577] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page992worker1 # [ 0.314778] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry993worker2 # [ 0.324500] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations994worker1 # [ 0.315787] PCI: Using configuration type 1 for base access995worker2 # [ 0.325240] audit: initializing netlink subsys (disabled)996worker1 # [ 0.316908] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.997worker2 # [ 0.326428] thermal_sys: Registered thermal governor 'fair_share'998client # [ 0.327209] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages999worker2 # [ 0.326430] thermal_sys: Registered thermal governor 'bang_bang'1000lb # [ 0.327577] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1001worker2 # [ 0.327231] thermal_sys: Registered thermal governor 'step_wise'1002client # [ 0.327952] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1003lb # [ 0.328576] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1004worker2 # [ 0.328233] audit: type=2000 audit(1790076276.224:1): state=initialized audit_enabled=0 res=11005worker2 # [ 0.330234] thermal_sys: Registered thermal governor 'user_space'1006worker2 # [ 0.330235] thermal_sys: Registered thermal governor 'power_allocator'1007worker2 # [ 0.331259] cpuidle: using governor menu1008worker1 # [ 0.323918] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1009client # [ 0.332953] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1010worker1 # [ 0.324776] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1011worker2 # [ 0.333538] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51012client # [ 0.333951] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1013worker2 # [ 0.334484] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1014worker2 # [ 0.335233] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1015worker2 # [ 0.336242] PCI: Using configuration type 1 for base access1016lb # [ 0.338939] ACPI: Added _OSI(Module Device)1017worker1 # [ 0.329777] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1018lb # [ 0.339577] ACPI: Added _OSI(Processor Device)1019worker2 # [ 0.337394] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1020worker1 # [ 0.330776] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1021lb # [ 0.340311] ACPI: Added _OSI(Processor Aggregator Device)1022client # [ 0.344381] ACPI: Added _OSI(Module Device)1023client # [ 0.344953] ACPI: Added _OSI(Processor Device)1024client # [ 0.345734] ACPI: Added _OSI(Processor Aggregator Device)1025worker2 # [ 0.346407] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1026lb # [ 0.346566] ACPI: 1 ACPI AML tables successfully acquired and loaded1027worker2 # [ 0.347231] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1028worker1 # [ 0.341216] ACPI: Added _OSI(Module Device)1029worker1 # [ 0.341777] ACPI: Added _OSI(Processor Device)1030worker1 # [ 0.342501] ACPI: Added _OSI(Processor Aggregator Device)1031lb # [ 0.352981] ACPI: Interpreter enabled1032lb # [ 0.353591] ACPI: PM: (supports S0 S3 S4 S5)1033lb # [ 0.354266] ACPI: Using IOAPIC for interrupt routing1034worker2 # [ 0.352231] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1035client # [ 0.354488] ACPI: 1 ACPI AML tables successfully acquired and loaded1036worker2 # [ 0.353230] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1037lb # [ 0.356624] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1038worker1 # [ 0.349346] ACPI: 1 ACPI AML tables successfully acquired and loaded1039lb # [ 0.357576] PCI: Using E820 reservations for host bridge windows1040client # [ 0.358199] ACPI: Interpreter enabled1041client # [ 0.358865] ACPI: PM: (supports S0 S3 S4 S5)1042client # [ 0.359512] ACPI: Using IOAPIC for interrupt routing1043lb # [ 0.360668] ACPI: Enabled 2 GPEs in block 00 to 3F1044client # [ 0.359987] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1045worker1 # [ 0.353180] ACPI: Interpreter enabled1046client # [ 0.360951] PCI: Using E820 reservations for host bridge windows1047worker1 # [ 0.353792] ACPI: PM: (supports S0 S3 S4 S5)1048worker1 # [ 0.354479] ACPI: Using IOAPIC for interrupt routing1049worker2 # [ 0.362576] ACPI: Added _OSI(Module Device)1050worker2 # [ 0.363232] ACPI: Added _OSI(Processor Device)1051client # [ 0.364058] ACPI: Enabled 2 GPEs in block 00 to 3F1052worker2 # [ 0.364015] ACPI: Added _OSI(Processor Aggregator Device)1053worker1 # [ 0.356846] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1054worker1 # [ 0.357777] PCI: Using E820 reservations for host bridge windows1055lb # [ 0.368113] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1056lb # [ 0.368582] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1057worker1 # [ 0.360914] ACPI: Enabled 2 GPEs in block 00 to 3F1058lb # [ 0.369655] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1059lb # [ 0.370697] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1060worker2 # [ 0.371747] ACPI: 1 ACPI AML tables successfully acquired and loaded1061lb # [ 0.371970] PCI host bridge to bus 0000:001062client # [ 0.372725] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1063lb # [ 0.372580] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1064lb # [ 0.373576] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1065client # [ 0.373782] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1066lb # [ 0.374576] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1067client # [ 0.375209] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1068worker2 # [ 0.375648] ACPI: Interpreter enabled1069worker1 # [ 0.367848] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1070lb # [ 0.375576] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1071worker2 # [ 0.376247] ACPI: PM: (supports S0 S3 S4 S5)1072client # [ 0.376074] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1073worker2 # [ 0.377014] ACPI: Using IOAPIC for interrupt routing1074lb # [ 0.376581] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1075worker1 # [ 0.368781] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1076client # [ 0.377323] PCI host bridge to bus 0000:001077lb # [ 0.377576] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1078worker1 # [ 0.369852] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1079client # [ 0.377956] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1080lb # [ 0.378613] pci_bus 0000:00: root bus resource [bus 00-ff]1081worker1 # [ 0.370898] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1082client # [ 0.378952] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1083worker2 # [ 0.379267] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1084lb # [ 0.379578] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1085worker1 # [ 0.372121] PCI host bridge to bus 0000:001086client # [ 0.379952] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1087worker2 # [ 0.380230] PCI: Using E820 reservations for host bridge windows1088worker1 # [ 0.372780] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1089client # [ 0.380952] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1090lb # [ 0.381036] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1091worker1 # [ 0.373775] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1092client # [ 0.381951] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093worker1 # [ 0.374775] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1094worker2 # [ 0.383370] ACPI: Enabled 2 GPEs in block 00 to 3F1095client # [ 0.382951] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1096worker1 # [ 0.375776] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1097client # [ 0.383952] pci_bus 0000:00: root bus resource [bus 00-ff]1098worker1 # [ 0.376775] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1099client # [ 0.384953] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1100worker1 # [ 0.377775] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1101worker1 # [ 0.378775] pci_bus 0000:00: root bus resource [bus 00-ff]1102client # [ 0.386391] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1103worker1 # [ 0.379750] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1104lb # [ 0.383668] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1105worker2 # [ 0.389268] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1106lb # [ 0.384588] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1107worker2 # [ 0.390235] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1108worker1 # [ 0.381309] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1109lb # [ 0.385506] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1110worker2 # [ 0.391308] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1111lb # [ 0.386546] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1112worker2 # [ 0.392362] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1113worker2 # [ 0.393601] PCI host bridge to bus 0000:001114lb # [ 0.388287] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115worker2 # [ 0.394235] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1116client # [ 0.389158] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1117worker2 # [ 0.395231] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1118client # [ 0.390017] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1119lb # [ 0.391620] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1120worker2 # [ 0.396230] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1121worker1 # [ 0.383877] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1122client # [ 0.390977] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1123lb # [ 0.392469] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1124worker2 # [ 0.397230] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1125worker1 # [ 0.384787] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1126lb # [ 0.393303] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1127worker2 # [ 0.398230] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1128worker1 # [ 0.385748] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1129client # [ 0.392309] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1130lb # [ 0.393582] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1131worker2 # [ 0.399230] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1132worker1 # [ 0.386742] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1133client # [ 0.393744] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1134worker2 # [ 0.400231] pci_bus 0000:00: root bus resource [bus 00-ff]1135lb # [ 0.395157] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1136worker2 # [ 0.401301] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1137worker1 # [ 0.388524] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138client # [ 0.395967] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1139worker2 # [ 0.402701] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1140lb # [ 0.397552] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1141client # [ 0.396868] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1142lb # [ 0.398249] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1143worker1 # [ 0.391825] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1144client # [ 0.397735] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1145lb # [ 0.398599] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1146worker1 # [ 0.392643] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1147client # [ 0.398834] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1148worker1 # [ 0.393478] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1149lb # [ 0.400231] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1150worker1 # [ 0.393781] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1151client # [ 0.400481] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1152worker1 # [ 0.395322] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1153client # [ 0.401977] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1154client # [ 0.402857] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1155worker2 # [ 0.405312] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1156lb # [ 0.403918] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1157client # [ 0.403758] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1158worker2 # [ 0.406244] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1159worker1 # [ 0.396817] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1160lb # [ 0.405246] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1161worker1 # [ 0.397621] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1162worker2 # [ 0.407256] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1163lb # [ 0.405598] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1164client # [ 0.405474] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1165worker1 # [ 0.398469] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1166worker2 # [ 0.408615] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1167lb # [ 0.407145] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1168worker1 # [ 0.399369] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169client # [ 0.407982] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1170worker2 # [ 0.409978] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1171client # [ 0.408899] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1172lb # [ 0.409459] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1173lb # [ 0.410258] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1174client # [ 0.409770] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1175worker2 # [ 0.412263] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1176lb # [ 0.410599] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1177worker1 # [ 0.401828] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1178client # [ 0.411472] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1179worker2 # [ 0.413209] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1180worker1 # [ 0.403383] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1181lb # [ 0.411582] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1182worker2 # [ 0.414008] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1183worker1 # [ 0.403796] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1184worker2 # [ 0.415080] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1185client # [ 0.412960] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1186lb # [ 0.413869] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187worker1 # [ 0.405359] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1188client # [ 0.413857] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1189worker2 # [ 0.416874] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1190client # [ 0.414717] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1191lb # [ 0.415589] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1192worker1 # [ 0.406818] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1193client # [ 0.415957] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194lb # [ 0.416593] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1195worker1 # [ 0.407683] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1196worker2 # [ 0.418262] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1197client # [ 0.417545] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1198worker1 # [ 0.408533] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1199worker2 # [ 0.419203] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1200lb # [ 0.418221] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1201worker1 # [ 0.408781] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1202worker2 # [ 0.420053] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1203client # [ 0.418827] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1204client # [ 0.419766] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1205worker2 # [ 0.421771] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1206worker1 # [ 0.410638] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1207lb # [ 0.419619] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1208lb # [ 0.420601] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1209client # [ 0.421493] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210worker1 # [ 0.412822] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1211lb # [ 0.422187] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1212worker1 # [ 0.413797] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1213worker2 # [ 0.424261] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1214client # [ 0.422965] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1215worker2 # [ 0.425146] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1216client # [ 0.424006] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1217worker1 # [ 0.415338] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1218lb # [ 0.423616] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1219worker2 # [ 0.426049] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1220lb # [ 0.424601] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1221client # [ 0.425567] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222worker1 # [ 0.416821] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1223worker2 # [ 0.427763] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1224lb # [ 0.426166] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1225client # [ 0.426826] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1226worker1 # [ 0.417797] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1227worker2 # [ 0.429239] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1228client # [ 0.427771] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1229worker1 # [ 0.419344] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1230worker2 # [ 0.430148] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1231lb # [ 0.427590] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1232client # [ 0.429417] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1233lb # [ 0.429305] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1234worker2 # [ 0.431047] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1235worker1 # [ 0.420822] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1236worker2 # [ 0.432236] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1237worker1 # [ 0.421791] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1238client # [ 0.431152] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1239lb # [ 0.430159] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1240client # [ 0.431976] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1241worker2 # [ 0.433825] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1242worker1 # [ 0.423313] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1243lb # [ 0.432445] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1244client # [ 0.433551] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1245worker2 # [ 0.435145] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1246worker1 # [ 0.424824] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1247lb # [ 0.433252] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248worker2 # [ 0.436019] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1249lb # [ 0.433598] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1250client # [ 0.434991] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1251worker1 # [ 0.425786] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1252worker2 # [ 0.437744] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1253client # [ 0.435958] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1254worker1 # [ 0.427308] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1255client # [ 0.436972] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1256lb # [ 0.435187] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1257worker2 # [ 0.439244] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1258worker1 # [ 0.428833] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1259client # [ 0.438550] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1260worker2 # [ 0.440254] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1261lb # [ 0.437353] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1262worker1 # [ 0.429700] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1263lb # [ 0.437585] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1264client # [ 0.439961] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1265worker1 # [ 0.430533] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1266worker2 # [ 0.441873] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1267lb # [ 0.438558] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1268client # [ 0.440889] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1269worker1 # [ 0.431335] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1270client # [ 0.441802] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1271worker2 # [ 0.443068] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1272lb # [ 0.440030] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1273worker2 # [ 0.444010] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1274lb # [ 0.441303] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1275client # [ 0.443485] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1276worker1 # [ 0.433629] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1277worker2 # [ 0.445841] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1278client # [ 0.444622] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1279worker1 # [ 0.434452] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1280lb # [ 0.441820] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1281worker1 # [ 0.434798] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1282client # [ 0.445156] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283worker2 # [ 0.447231] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1284lb # [ 0.443398] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1285worker2 # [ 0.448252] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1286client # [ 0.446513] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1287worker1 # [ 0.436361] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1288lb # [ 0.444479] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1289worker1 # [ 0.437490] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1290client # [ 0.447165] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1291worker2 # [ 0.449849] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1292lb # [ 0.446330] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1293worker1 # [ 0.437976] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1294client # [ 0.448537] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1295worker2 # [ 0.451246] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1296worker1 # [ 0.439409] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1297worker2 # [ 0.452179] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1298worker2 # [ 0.453072] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1299worker1 # [ 0.439971] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1300worker1 # [ 0.441447] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1301worker2 # [ 0.454803] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1302worker2 # [ 0.456238] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1303worker2 # [ 0.457181] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1304worker2 # [ 0.458075] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1305worker2 # [ 0.459832] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1306lb # [ 0.446817] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1307worker2 # [ 0.460925] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1308client # [ 0.449257] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1309lb # [ 0.449357] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1310client # [ 0.451493] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1311worker2 # [ 0.461454] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1312lb # [ 0.450655] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1313worker2 # [ 0.462802] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1314client # [ 0.452378] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1315worker1 # [ 0.441994] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1316lb # [ 0.451882] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1317worker2 # [ 0.463461] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1318client # [ 0.453324] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1319worker1 # [ 0.444405] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320worker2 # [ 0.464789] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1321lb # [ 0.452848] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1322client # [ 0.454234] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1323worker1 # [ 0.445055] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1324lb # [ 0.454635] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1325client # [ 0.455826] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1326worker1 # [ 0.446063] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1327lb # [ 0.455464] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1328client # [ 0.456802] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1329worker1 # [ 0.447155] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1330lb # [ 0.456700] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1331client # [ 0.458241] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1332worker1 # [ 0.448839] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1333worker1 # [ 0.449681] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1334client # [ 0.459525] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1335lb # [ 0.458428] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1336worker1 # [ 0.450822] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1337worker1 # [ 0.452467] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1338lb # [ 0.462659] ACPI: PCI: Interrupt link LNKA configured for IRQ 101339client # [ 0.463484] ACPI: PCI: Interrupt link LNKA configured for IRQ 101340lb # [ 0.463688] ACPI: PCI: Interrupt link LNKB configured for IRQ 101341client # [ 0.464061] ACPI: PCI: Interrupt link LNKB configured for IRQ 101342worker2 # [ 0.465470] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1343lb # [ 0.464680] ACPI: PCI: Interrupt link LNKC configured for IRQ 111344client # [ 0.465081] ACPI: PCI: Interrupt link LNKC configured for IRQ 111345worker1 # [ 0.455908] ACPI: PCI: Interrupt link LNKA configured for IRQ 101346worker2 # [ 0.467785] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1347client # [ 0.466059] ACPI: PCI: Interrupt link LNKD configured for IRQ 111348lb # [ 0.465675] ACPI: PCI: Interrupt link LNKD configured for IRQ 111349worker1 # [ 0.456884] ACPI: PCI: Interrupt link LNKB configured for IRQ 101350lb # [ 0.466679] ACPI: PCI: Interrupt link LNKE configured for IRQ 101351client # [ 0.467051] ACPI: PCI: Interrupt link LNKE configured for IRQ 101352worker2 # [ 0.468529] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1353worker1 # [ 0.457881] ACPI: PCI: Interrupt link LNKC configured for IRQ 111354lb # [ 0.467676] ACPI: PCI: Interrupt link LNKF configured for IRQ 101355client # [ 0.468059] ACPI: PCI: Interrupt link LNKF configured for IRQ 101356worker1 # [ 0.458874] ACPI: PCI: Interrupt link LNKD configured for IRQ 111357worker2 # [ 0.469562] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1358lb # [ 0.468679] ACPI: PCI: Interrupt link LNKG configured for IRQ 111359client # [ 0.469059] ACPI: PCI: Interrupt link LNKG configured for IRQ 111360worker1 # [ 0.459872] ACPI: PCI: Interrupt link LNKE configured for IRQ 101361lb # [ 0.469675] ACPI: PCI: Interrupt link LNKH configured for IRQ 111362client # [ 0.470053] ACPI: PCI: Interrupt link LNKH configured for IRQ 111363worker1 # [ 0.460872] ACPI: PCI: Interrupt link LNKF configured for IRQ 101364worker2 # [ 0.470501] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1365lb # [ 0.470614] ACPI: PCI: Interrupt link GSIA configured for IRQ 161366client # [ 0.470987] ACPI: PCI: Interrupt link GSIA configured for IRQ 161367worker1 # [ 0.461893] ACPI: PCI: Interrupt link LNKG configured for IRQ 111368lb # [ 0.471591] ACPI: PCI: Interrupt link GSIB configured for IRQ 171369worker2 # [ 0.472097] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1370worker1 # [ 0.462898] ACPI: PCI: Interrupt link LNKH configured for IRQ 111371client # [ 0.471964] ACPI: PCI: Interrupt link GSIB configured for IRQ 171372lb # [ 0.472590] ACPI: PCI: Interrupt link GSIC configured for IRQ 181373worker2 # [ 0.472910] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1374worker1 # [ 0.463810] ACPI: PCI: Interrupt link GSIA configured for IRQ 161375client # [ 0.472968] ACPI: PCI: Interrupt link GSIC configured for IRQ 181376lb # [ 0.473587] ACPI: PCI: Interrupt link GSID configured for IRQ 191377worker1 # [ 0.464730] ACPI: PCI: Interrupt link GSIB configured for IRQ 171378client # [ 0.473968] ACPI: PCI: Interrupt link GSID configured for IRQ 191379worker2 # [ 0.473572] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1380lb # [ 0.474587] ACPI: PCI: Interrupt link GSIE configured for IRQ 201381worker1 # [ 0.465472] ACPI: PCI: Interrupt link GSIC configured for IRQ 181382client # [ 0.474966] ACPI: PCI: Interrupt link GSIE configured for IRQ 201383worker1 # [ 0.465790] ACPI: PCI: Interrupt link GSID configured for IRQ 191384lb # [ 0.475587] ACPI: PCI: Interrupt link GSIF configured for IRQ 211385worker2 # [ 0.474858] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1386client # [ 0.475965] ACPI: PCI: Interrupt link GSIF configured for IRQ 211387worker1 # [ 0.466728] ACPI: PCI: Interrupt link GSIE configured for IRQ 201388lb # [ 0.476591] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389client # [ 0.476966] ACPI: PCI: Interrupt link GSIG configured for IRQ 221390worker1 # [ 0.467513] ACPI: PCI: Interrupt link GSIF configured for IRQ 211391lb # [ 0.477580] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392client # [ 0.477964] ACPI: PCI: Interrupt link GSIH configured for IRQ 231393worker1 # [ 0.467791] ACPI: PCI: Interrupt link GSIG configured for IRQ 221394worker1 # [ 0.468782] ACPI: PCI: Interrupt link GSIH configured for IRQ 231395lb # [ 0.479704] iommu: Default domain type: Translated1396client # [ 0.480033] iommu: Default domain type: Translated1397worker2 # [ 0.478888] ACPI: PCI: Interrupt link LNKA configured for IRQ 101398lb # [ 0.480544] iommu: DMA domain TLB invalidation policy: lazy mode1399client # [ 0.480937] iommu: DMA domain TLB invalidation policy: lazy mode1400worker1 # [ 0.470858] iommu: Default domain type: Translated1401worker2 # [ 0.479347] ACPI: PCI: Interrupt link LNKB configured for IRQ 101402lb # [ 0.481764] ACPI: bus type USB registered1403worker1 # [ 0.471661] iommu: DMA domain TLB invalidation policy: lazy mode1404client # [ 0.482067] ACPI: bus type USB registered1405lb # [ 0.482584] usbcore: registered new interface driver usbfs1406worker2 # [ 0.480331] ACPI: PCI: Interrupt link LNKC configured for IRQ 111407worker1 # [ 0.472805] ACPI: bus type USB registered1408client # [ 0.482862] usbcore: registered new interface driver usbfs1409lb # [ 0.483515] usbcore: registered new interface driver hub1410worker2 # [ 0.481336] ACPI: PCI: Interrupt link LNKD configured for IRQ 111411client # [ 0.483705] usbcore: registered new interface driver hub1412lb # [ 0.484218] usbcore: registered new device driver usb1413worker1 # [ 0.473801] usbcore: registered new interface driver usbfs1414worker2 # [ 0.482331] ACPI: PCI: Interrupt link LNKE configured for IRQ 101415client # [ 0.483961] usbcore: registered new device driver usb1416worker1 # [ 0.474696] usbcore: registered new interface driver hub1417lb # [ 0.485507] NetLabel: Initializing1418worker2 # [ 0.483329] ACPI: PCI: Interrupt link LNKF configured for IRQ 101419worker1 # [ 0.475406] usbcore: registered new device driver usb1420client # [ 0.485759] NetLabel: Initializing1421lb # [ 0.486103] NetLabel: domain hash size = 1281422worker2 # [ 0.484357] ACPI: PCI: Interrupt link LNKG configured for IRQ 111423client # [ 0.486446] NetLabel: domain hash size = 1281424worker1 # [ 0.476634] NetLabel: Initializing1425lb # [ 0.486576] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker1 # [ 0.477230] NetLabel: domain hash size = 1281427worker2 # [ 0.485335] ACPI: PCI: Interrupt link LNKH configured for IRQ 111428client # [ 0.486953] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1429lb # [ 0.487620] NetLabel: unlabeled traffic allowed by default1430worker1 # [ 0.477775] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1431worker2 # [ 0.486271] ACPI: PCI: Interrupt link GSIA configured for IRQ 161432client # [ 0.488008] NetLabel: unlabeled traffic allowed by default1433lb # [ 0.488583] PCI: Using ACPI for IRQ routing1434worker1 # [ 0.478711] NetLabel: unlabeled traffic allowed by default1435client # [ 0.488964] PCI: Using ACPI for IRQ routing1436worker2 # [ 0.487243] ACPI: PCI: Interrupt link GSIB configured for IRQ 171437worker1 # [ 0.479483] PCI: Using ACPI for IRQ routing1438worker2 # [ 0.488246] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439worker2 # [ 0.489251] ACPI: PCI: Interrupt link GSID configured for IRQ 191440worker2 # [ 0.490243] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441worker2 # [ 0.491243] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442worker2 # [ 0.492244] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443worker2 # [ 0.493246] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker2 # [ 0.495284] iommu: Default domain type: Translated1445worker2 # [ 0.496138] iommu: DMA domain TLB invalidation policy: lazy mode1446worker2 # [ 0.497330] ACPI: bus type USB registered1447worker2 # [ 0.498124] usbcore: registered new interface driver usbfs1448worker2 # [ 0.498979] usbcore: registered new interface driver hub1449worker2 # [ 0.499238] usbcore: registered new device driver usb1450worker2 # [ 0.501015] NetLabel: Initializing1451worker2 # [ 0.501237] NetLabel: domain hash size = 1281452worker2 # [ 0.502053] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker2 # [ 0.503041] NetLabel: unlabeled traffic allowed by default1454worker2 # [ 0.503989] PCI: Using ACPI for IRQ routing1455worker1 # [ 0.524617] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker1 # [ 0.524771] pci 0000:00:01.0: vgaarb: bridge control possible1457worker1 # [ 0.524771] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458lb # [ 0.538544] pci 0000:00:01.0: vgaarb: setting as boot VGA device1459worker1 # [ 0.524783] vgaarb: loaded1460lb # [ 0.538572] pci 0000:00:01.0: vgaarb: bridge control possible1461worker1 # [ 0.525564] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01462worker1 # [ 0.526362] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1463lb # [ 0.538572] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1464lb # [ 0.538580] vgaarb: loaded1465lb # [ 0.539363] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01466client # [ 0.542390] pci 0000:00:01.0: vgaarb: setting as boot VGA device1467lb # [ 0.540197] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1468client # [ 0.542947] pci 0000:00:01.0: vgaarb: bridge control possible1469worker1 # [ 0.529884] clocksource: Switched to clocksource kvm-clock1470client # [ 0.542947] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471client # [ 0.542955] vgaarb: loaded1472lb # [ 0.543656] clocksource: Switched to clocksource kvm-clock1473client # [ 0.543791] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01474worker1 # [ 0.533614] VFS: Disk quotas dquot_6.6.01475client # [ 0.544605] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1476worker1 # [ 0.534348] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1477worker1 # [ 0.535764] pnp: PnP ACPI init1478lb # [ 0.547428] VFS: Disk quotas dquot_6.6.01479worker1 # [ 0.536634] ACPI: IRQ 4 override to edge(!), high(!)1480lb # [ 0.548205] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1481worker1 # [ 0.537646] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1482lb # [ 0.549720] pnp: PnP ACPI init1483worker1 # [ 0.539203] pnp: PnP ACPI: found 5 devices1484lb # [ 0.550613] ACPI: IRQ 4 override to edge(!), high(!)1485client # [ 0.551048] clocksource: Switched to clocksource kvm-clock1486lb # [ 0.551636] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1487lb # [ 0.553316] pnp: PnP ACPI: found 5 devices1488client # [ 0.554998] VFS: Disk quotas dquot_6.6.01489worker2 # [ 0.555964] pci 0000:00:01.0: vgaarb: setting as boot VGA device1490client # [ 0.555774] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1491worker2 # [ 0.556226] pci 0000:00:01.0: vgaarb: bridge control possible1492client # [ 0.557189] pnp: PnP ACPI init1493worker1 # [ 0.546845] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1494worker2 # [ 0.556226] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1495client # [ 0.558036] ACPI: IRQ 4 override to edge(!), high(!)1496worker2 # [ 0.556234] vgaarb: loaded1497worker1 # [ 0.548328] clocksource: Switched to clocksource acpi_pm1498worker2 # [ 0.557017] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01499worker1 # [ 0.549314] NET: Registered PF_INET protocol family1500client # [ 0.559021] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1501worker2 # [ 0.557853] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1502client # [ 0.560603] pnp: PnP ACPI: found 5 devices1503lb # [ 0.560968] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1504worker1 # [ 0.550519] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1505lb # [ 0.562432] clocksource: Switched to clocksource acpi_pm1506lb # [ 0.563454] NET: Registered PF_INET protocol family1507lb # [ 0.564679] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1508worker2 # [ 0.564322] clocksource: Switched to clocksource kvm-clock1509worker2 # [ 0.566047] VFS: Disk quotas dquot_6.6.01510client # [ 0.568344] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1511client # [ 0.569930] clocksource: Switched to clocksource acpi_pm1512client # [ 0.570963] NET: Registered PF_INET protocol family1513worker2 # [ 0.568596] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1514client # [ 0.572224] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515worker2 # [ 0.570095] pnp: PnP ACPI init1516worker2 # [ 0.570942] ACPI: IRQ 4 override to edge(!), high(!)1517worker2 # [ 0.571979] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1518worker2 # [ 0.573693] pnp: PnP ACPI: found 5 devices1519worker1 # [ 0.567425] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1520worker1 # [ 0.569067] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521worker1 # [ 0.570405] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522lb # [ 0.581205] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1523worker1 # [ 0.571785] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1524lb # [ 0.582845] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1525worker1 # [ 0.573052] TCP: Hash tables configured (established 16384 bind 16384)1526lb # [ 0.584222] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1527worker1 # [ 0.574198] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1528worker2 # [ 0.581366] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1529lb # [ 0.585666] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1530worker1 # [ 0.575467] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1531worker2 # [ 0.582802] clocksource: Switched to clocksource acpi_pm1532lb # [ 0.586955] TCP: Hash tables configured (established 16384 bind 16384)1533worker2 # [ 0.583793] NET: Registered PF_INET protocol family1534worker1 # [ 0.576564] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1535lb # [ 0.588140] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1536worker1 # [ 0.577786] NET: Registered PF_UNIX/PF_LOCAL protocol family1537worker2 # [ 0.584978] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1538worker1 # [ 0.578720] NET: Registered PF_XDP protocol family1539lb # [ 0.589443] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1540worker1 # [ 0.579551] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1541client # [ 0.589134] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1542lb # [ 0.590579] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1543worker1 # [ 0.580594] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1544lb # [ 0.591856] NET: Registered PF_UNIX/PF_LOCAL protocol family1545client # [ 0.590759] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1546worker1 # [ 0.581635] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1547lb # [ 0.592849] NET: Registered PF_XDP protocol family1548client # [ 0.592134] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1549worker1 # [ 0.582748] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1550lb # [ 0.593733] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1551client # [ 0.593538] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1552worker1 # [ 0.583833] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1553lb # [ 0.594778] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554client # [ 0.594847] TCP: Hash tables configured (established 16384 bind 16384)1555worker1 # [ 0.584885] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1556lb # [ 0.595833] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1557lb # [ 0.596990] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1558client # [ 0.596014] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1559worker1 # [ 0.586646] ACPI: \_SB_.GSIA: Enabled at IRQ 161560lb # [ 0.598146] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561client # [ 0.597356] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1562lb # [ 0.599326] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563client # [ 0.598551] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1564worker1 # [ 0.589222] ACPI: \_SB_.GSIB: Enabled at IRQ 171565client # [ 0.599920] NET: Registered PF_UNIX/PF_LOCAL protocol family1566lb # [ 0.601133] ACPI: \_SB_.GSIA: Enabled at IRQ 161567client # [ 0.600954] NET: Registered PF_XDP protocol family1568worker1 # [ 0.591606] ACPI: \_SB_.GSIC: Enabled at IRQ 181569client # [ 0.601870] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1570lb # [ 0.603796] ACPI: \_SB_.GSIB: Enabled at IRQ 171571client # [ 0.602930] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1572worker1 # [ 0.593725] ACPI: \_SB_.GSID: Enabled at IRQ 191573client # [ 0.604000] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1574lb # [ 0.605759] ACPI: \_SB_.GSIC: Enabled at IRQ 181575worker2 # [ 0.601280] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1576worker1 # [ 0.595442] PCI: CLS 0 bytes, default 641577client # [ 0.605177] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1578worker2 # [ 0.602913] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1579client # [ 0.606336] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1580lb # [ 0.607725] ACPI: \_SB_.GSID: Enabled at IRQ 191581worker1 # [ 0.596462] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1582client # [ 0.607497] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1583worker2 # [ 0.604309] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1584worker1 # [ 0.598240] Trying to unpack rootfs image as initramfs...1585lb # [ 0.609371] PCI: CLS 0 bytes, default 641586worker2 # [ 0.605745] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1587client # [ 0.609248] ACPI: \_SB_.GSIA: Enabled at IRQ 161588worker2 # [ 0.607062] TCP: Hash tables configured (established 16384 bind 16384)1589lb # [ 0.610307] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1590lb # [ 0.612162] Trying to unpack rootfs image as initramfs...1591worker2 # [ 0.608161] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1592client # [ 0.611854] ACPI: \_SB_.GSIB: Enabled at IRQ 171593worker2 # [ 0.609529] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1594worker2 # [ 0.610689] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1595client # [ 0.614211] ACPI: \_SB_.GSIC: Enabled at IRQ 181596worker2 # [ 0.611965] NET: Registered PF_UNIX/PF_LOCAL protocol family1597worker2 # [ 0.613002] NET: Registered PF_XDP protocol family1598client # [ 0.616262] ACPI: \_SB_.GSID: Enabled at IRQ 191599worker2 # [ 0.613888] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1600client # [ 0.618056] PCI: CLS 0 bytes, default 641601worker2 # [ 0.614906] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1602worker2 # [ 0.615958] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1603client # [ 0.619097] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1604worker2 # [ 0.617101] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1605client # [ 0.620979] Trying to unpack rootfs image as initramfs...1606worker2 # [ 0.618235] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607worker2 # [ 0.619388] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker2 # [ 0.621132] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker2 # [ 0.623391] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker2 # [ 0.625315] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker2 # [ 0.627213] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker2 # [ 0.628914] PCI: CLS 0 bytes, default 641613worker2 # [ 0.629834] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614worker2 # [ 0.631817] Trying to unpack rootfs image as initramfs...1615worker1 # [ 0.640836] Initialise system trusted keyrings1616worker1 # [ 0.644679] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.658029] Initialise system trusted keyrings1618lb # [ 0.659025] workingset: timestamp_bits=40 max_order=19 bucket_order=01619client # [ 0.666044] Initialise system trusted keyrings1620client # [ 0.669690] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker1 # [ 0.667611] Key type asymmetric registered1622worker1 # [ 0.668386] Asymmetric key parser 'x509' registered1623worker2 # [ 0.676279] Initialise system trusted keyrings1624worker1 # [ 0.669323] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625lb # [ 0.682085] Key type asymmetric registered1626lb # [ 0.682835] Asymmetric key parser 'x509' registered1627worker2 # [ 0.679406] workingset: timestamp_bits=40 max_order=19 bucket_order=01628lb # [ 0.683754] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1629worker1 # [ 0.674624] io scheduler mq-deadline registered1630worker1 # [ 0.675451] io scheduler kyber registered1631lb # [ 0.689091] io scheduler mq-deadline registered1632lb # [ 0.689954] io scheduler kyber registered1633client # [ 0.689005] Key type asymmetric registered1634worker1 # [ 0.679172] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1635worker1 # [ 0.680464] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1636lb # [ 0.694103] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637client # [ 0.693654] Asymmetric key parser 'x509' registered1638worker1 # [ 0.684350] Linux agpgart interface v0.1031639client # [ 0.694523] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1640lb # [ 0.695478] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1641worker1 # [ 0.685188] ACPI: bus type drm_connector registered1642client # [ 0.697748] io scheduler mq-deadline registered1643client # [ 0.698552] io scheduler kyber registered1644lb # [ 0.700731] Linux agpgart interface v0.1031645worker1 # [ 0.689969] usbcore: registered new interface driver usbserial_generic1646lb # [ 0.701557] ACPI: bus type drm_connector registered1647worker1 # [ 0.691294] usbserial: USB Serial support registered for generic1648client # [ 0.702666] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1649worker2 # [ 0.701450] Key type asymmetric registered1650client # [ 0.704052] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1651worker1 # [ 0.694519] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1652worker2 # [ 0.702223] Asymmetric key parser 'x509' registered1653lb # [ 0.706425] usbcore: registered new interface driver usbserial_generic1654worker1 # [ 0.694519] Please enable it if your BIOS has the CPPC option.1655worker2 # [ 0.703112] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1656lb # [ 0.707560] usbserial: USB Serial support registered for generic1657worker1 # [ 0.696721] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658client # [ 0.709458] Linux agpgart interface v0.1031659lb # [ 0.709934] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1660worker1 # [ 0.699671] drop_monitor: Initializing network drop monitor service1661client # [ 0.710313] ACPI: bus type drm_connector registered1662lb # [ 0.709934] Please enable it if your BIOS has the CPPC option.1663worker1 # [ 0.700847] NET: Registered PF_INET6 protocol family1664worker2 # [ 0.708469] io scheduler mq-deadline registered1665lb # [ 0.712263] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1666worker2 # [ 0.709359] io scheduler kyber registered1667worker1 # [ 0.704028] Segment Routing with IPv61668client # [ 0.715126] usbcore: registered new interface driver usbserial_generic1669worker2 # [ 0.712968] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1670lb # [ 0.717064] drop_monitor: Initializing network drop monitor service1671client # [ 0.716310] usbserial: USB Serial support registered for generic1672worker1 # [ 0.707533] In-situ OAM (IOAM) with IPv61673lb # [ 0.718267] NET: Registered PF_INET6 protocol family1674worker2 # [ 0.714309] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1675worker1 # [ 0.708639] IPI shorthand broadcast: enabled1676lb # [ 0.721379] Segment Routing with IPv61677client # [ 0.719648] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1678client # [ 0.719648] Please enable it if your BIOS has the CPPC option.1679worker2 # [ 0.719496] Linux agpgart interface v0.1031680client # [ 0.722027] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1681worker2 # [ 0.720299] ACPI: bus type drm_connector registered1682lb # [ 0.724960] In-situ OAM (IOAM) with IPv61683lb # [ 0.726116] IPI shorthand broadcast: enabled1684client # [ 0.725821] drop_monitor: Initializing network drop monitor service1685worker2 # [ 0.723648] usbcore: registered new interface driver usbserial_generic1686client # [ 0.727058] NET: Registered PF_INET6 protocol family1687worker1 # [ 0.716919] sched_clock: Marking stable (583013026, 133477492)->(793314253, -76823735)1688worker2 # [ 0.724802] usbserial: USB Serial support registered for generic1689client # [ 0.730100] Segment Routing with IPv61690worker2 # [ 0.727371] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1691worker1 # [ 0.721786] registered taskstats version 11692worker2 # [ 0.727371] Please enable it if your BIOS has the CPPC option.1693worker1 # [ 0.722757] Loading compiled-in X.509 certificates1694worker2 # [ 0.729727] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1695client # [ 0.733683] In-situ OAM (IOAM) with IPv61696lb # [ 0.734469] sched_clock: Marking stable (600014426, 133901085)->(810309439, -76393928)1697client # [ 0.734853] IPI shorthand broadcast: enabled1698worker2 # [ 0.733499] drop_monitor: Initializing network drop monitor service1699worker2 # [ 0.734867] NET: Registered PF_INET6 protocol family1700lb # [ 0.739097] registered taskstats version 11701lb # [ 0.740101] Loading compiled-in X.509 certificates1702worker2 # [ 0.738787] Segment Routing with IPv61703client # [ 0.743421] sched_clock: Marking stable (604014711, 138614932)->(819285653, -76656010)1704worker2 # [ 0.741387] In-situ OAM (IOAM) with IPv61705worker2 # [ 0.743635] IPI shorthand broadcast: enabled1706client # [ 0.746850] registered taskstats version 11707client # [ 0.747895] Loading compiled-in X.509 certificates1708worker1 # [ 0.740508] Demotion targets for Node 0: null1709worker1 # [ 0.741439] Key type .fscrypt registered1710worker1 # [ 0.744496] Key type fscrypt-provisioning registered1711worker2 # [ 0.751941] sched_clock: Marking stable (609014768, 142328023)->(828845786, -77502995)1712worker1 # [ 0.745513] ima: No TPM chip found, activating TPM-bypass!1713worker1 # [ 0.746409] ima: Allocated hash algorithm: sha11714lb # [ 0.758931] Demotion targets for Node 0: null1715worker1 # [ 0.748541] ima: No architecture policies found1716worker2 # [ 0.755510] registered taskstats version 11717lb # [ 0.759852] Key type .fscrypt registered1718worker2 # [ 0.756526] Loading compiled-in X.509 certificates1719lb # [ 0.761924] Key type fscrypt-provisioning registered1720worker1 # [ 0.751504] PM: Magic number: 10:213:4291721lb # [ 0.762942] ima: No TPM chip found, activating TPM-bypass!1722worker1 # [ 0.753251] RAS: Correctable Errors collector initialized.1723lb # [ 0.765933] ima: Allocated hash algorithm: sha11724lb # [ 0.766778] ima: No architecture policies found1725client # [ 0.766650] Demotion targets for Node 0: null1726lb # [ 0.768932] PM: Magic number: 10:213:4291727client # [ 0.769685] Key type .fscrypt registered1728lb # [ 0.770576] RAS: Correctable Errors collector initialized.1729client # [ 0.770404] Key type fscrypt-provisioning registered1730client # [ 0.771429] ima: No TPM chip found, activating TPM-bypass!1731worker1 # [ 0.762227] clk: Disabling unused clocks1732client # [ 0.773654] ima: Allocated hash algorithm: sha11733client # [ 0.774459] ima: No architecture policies found1734worker1 # [ 0.764526] PM: genpd: Disabling unused power domains1735client # [ 0.776644] PM: Magic number: 10:213:4291736worker2 # [ 0.775004] Demotion targets for Node 0: null1737client # [ 0.778407] RAS: Correctable Errors collector initialized.1738lb # [ 0.779605] clk: Disabling unused clocks1739worker2 # [ 0.778389] Key type .fscrypt registered1740worker2 # [ 0.779091] Key type fscrypt-provisioning registered1741lb # [ 0.783931] PM: genpd: Disabling unused power domains1742worker2 # [ 0.780085] ima: No TPM chip found, activating TPM-bypass!1743worker2 # [ 0.782361] ima: Allocated hash algorithm: sha11744worker2 # [ 0.783171] ima: No architecture policies found1745client # [ 0.789418] clk: Disabling unused clocks1746worker2 # [ 0.787365] PM: Magic number: 10:213:4291747client # [ 0.791663] PM: genpd: Disabling unused power domains1748worker2 # [ 0.789003] RAS: Correctable Errors collector initialized.1749worker2 # [ 0.797052] clk: Disabling unused clocks1750worker2 # [ 0.801368] PM: genpd: Disabling unused power domains1751lb # [ 0.938778] Freeing initrd memory: 29064K1752worker1 # [ 0.929959] Freeing initrd memory: 29056K1753lb # [ 0.941952] Freeing unused decrypted memory: 2028K1754worker1 # [ 0.933319] Freeing unused decrypted memory: 2028K1755lb # [ 0.944585] Freeing unused kernel image (initmem) memory: 3652K1756lb # [ 0.945870] Write protecting the kernel read-only data: 32768k1757worker1 # [ 0.936218] Freeing unused kernel image (initmem) memory: 3652K1758lb # [ 0.947801] Freeing unused kernel image (text/rodata gap) memory: 1184K1759worker1 # [ 0.937355] Write protecting the kernel read-only data: 32768k1760lb # [ 0.949377] Freeing unused kernel image (rodata/data gap) memory: 720K1761worker1 # [ 0.939315] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker1 # [ 0.940913] Freeing unused kernel image (rodata/data gap) memory: 720K1763client # [ 0.959604] Freeing initrd memory: 29056K1764client # [ 0.962777] Freeing unused decrypted memory: 2028K1765client # [ 0.965345] Freeing unused kernel image (initmem) memory: 3652K1766client # [ 0.966493] Write protecting the kernel read-only data: 32768k1767worker2 # [ 0.963702] Freeing initrd memory: 29060K1768client # [ 0.968440] Freeing unused kernel image (text/rodata gap) memory: 1184K1769worker2 # [ 0.966958] Freeing unused decrypted memory: 2028K1770client # [ 0.969980] Freeing unused kernel image (rodata/data gap) memory: 720K1771worker2 # [ 0.969658] Freeing unused kernel image (initmem) memory: 3652K1772worker2 # [ 0.970767] Write protecting the kernel read-only data: 32768k1773worker2 # [ 0.972666] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker2 # [ 0.974200] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 1.000406] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 1.001530] Run /init as init process1777worker1 # [ 0.992268] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778worker1 # [ 0.993471] Run /init as init process1779lb # [ 1.012000] systemd[1]: Inserted module 'autofs4'1780worker1 # [ 1.005059] systemd[1]: Inserted module 'autofs4'1781client # [ 1.020932] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782client # [ 1.022091] Run /init as init process1783worker2 # [ 1.025716] x86/mm: Checked W+X mappings: passed, no W+X pages found.1784worker2 # [ 1.026876] Run /init as init process1785lb # [ 1.031803] fuse: init (API version 7.45)1786client # [ 1.032575] systemd[1]: Inserted module 'autofs4'1787worker1 # [ 1.025581] fuse: init (API version 7.45)1788lb # [ 1.039256] ACPI: \_SB_.GSIG: Enabled at IRQ 221789lb # [ 1.041776] ACPI: \_SB_.GSIH: Enabled at IRQ 231790worker2 # [ 1.038045] systemd[1]: Inserted module 'autofs4'1791worker1 # [ 1.033183] ACPI: \_SB_.GSIG: Enabled at IRQ 221792lb # [ 1.044856] ACPI: \_SB_.GSIE: Enabled at IRQ 201793worker1 # [ 1.035679] ACPI: \_SB_.GSIH: Enabled at IRQ 231794lb # [ 1.046822] ACPI: \_SB_.GSIF: Enabled at IRQ 211795worker1 # [ 1.038631] ACPI: \_SB_.GSIE: Enabled at IRQ 201796worker1 # [ 1.040646] ACPI: \_SB_.GSIF: Enabled at IRQ 211797lb # [ 1.052116] virtiofs virtio5: discovered new tag: nix-store1798client # [ 1.052404] fuse: init (API version 7.45)1799lb # [ 1.053703] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1800worker1 # [ 1.045851] virtiofs virtio5: discovered new tag: nix-store1801worker1 # [ 1.047466] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1802lb # [ 1.060110] virtiofs virtio6: discovered new tag: shared1803client # [ 1.059824] ACPI: \_SB_.GSIG: Enabled at IRQ 221804worker2 # [ 1.058034] fuse: init (API version 7.45)1805lb # [ 1.061632] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1806client # [ 1.062302] ACPI: \_SB_.GSIH: Enabled at IRQ 231807lb # [ 1.065150] virtiofs virtio7: discovered new tag: xchg1808client # [ 1.065376] ACPI: \_SB_.GSIE: Enabled at IRQ 201809worker1 # [ 1.055658] virtiofs virtio6: discovered new tag: shared1810lb # [ 1.066629] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1811client # [ 1.067448] ACPI: \_SB_.GSIF: Enabled at IRQ 211812worker1 # [ 1.057426] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1813worker2 # [ 1.065469] ACPI: \_SB_.GSIG: Enabled at IRQ 221814worker1 # [ 1.060842] virtiofs virtio7: discovered new tag: xchg1815worker2 # [ 1.068209] ACPI: \_SB_.GSIH: Enabled at IRQ 231816worker1 # [ 1.062574] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817client # [ 1.072649] virtiofs virtio5: discovered new tag: nix-store1818worker2 # [ 1.071238] ACPI: \_SB_.GSIE: Enabled at IRQ 201819client # [ 1.074420] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1820worker2 # [ 1.073306] ACPI: \_SB_.GSIF: Enabled at IRQ 211821client # [ 1.080811] virtiofs virtio6: discovered new tag: shared1822worker2 # [ 1.078577] virtiofs virtio5: discovered new tag: nix-store1823client # [ 1.082476] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1824worker2 # [ 1.080260] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1825client # [ 1.085736] virtiofs virtio7: discovered new tag: xchg1826lb # [ 1.086630] systemd[1]: Successfully made /usr/ read-only.1827client # [ 1.087268] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1828worker2 # [ 1.087672] virtiofs virtio6: discovered new tag: shared1829worker2 # [ 1.089532] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830worker1 # [ 1.085424] systemd[1]: Successfully made /usr/ read-only.1831worker2 # [ 1.094226] virtiofs virtio7: discovered new tag: xchg1832worker2 # [ 1.095839] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833client # [ 1.109418] systemd[1]: Successfully made /usr/ read-only.1834worker2 # [ 1.119412] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.423110] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker1 # [ 1.421032] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1837lb # [ 1.434976] systemd[1]: Detected virtualization kvm.1838worker1 # [ 1.425788] systemd[1]: Detected virtualization kvm.1839worker1 # [ 1.426616] systemd[1]: Detected architecture x86-64.1840worker1 # [ 1.427467] systemd[1]: Running in initrd.1841lb # [ 1.437073] systemd[1]: Detected architecture x86-64.1842worker1 # [ 1.428615] systemd[1]: Initializing machine ID from random generator.1843lb # [ 1.439168] systemd[1]: Running in initrd.1844worker1 # [ 1.429861] systemd[1]: Hostname set to <worker1>.1845lb # [ 1.441519] systemd[1]: Initializing machine ID from random generator.1846lb # [ 1.444362] systemd[1]: Hostname set to <lb>.1847client # [ 1.446001] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.458247] systemd[1]: Detected virtualization kvm.1849client # [ 1.460496] systemd[1]: Detected architecture x86-64.1850worker2 # [ 1.455139] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1851worker2 # [ 1.460360] systemd[1]: Detected virtualization kvm.1852client # [ 1.462741] systemd[1]: Running in initrd.1853worker2 # [ 1.461246] systemd[1]: Detected architecture x86-64.1854worker2 # [ 1.462093] systemd[1]: Running in initrd.1855worker2 # [ 1.463244] systemd[1]: Initializing machine ID from random generator.1856client # [ 1.465224] systemd[1]: Initializing machine ID from random generator.1857worker2 # [ 1.464491] systemd[1]: Hostname set to <worker2>.1858client # [ 1.468222] systemd[1]: Hostname set to <client>.1859worker1 # [ 1.637087] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.651105] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker2 # [ 1.661007] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862lb # [ 1.688465] systemd[1]: Queued start job for default target Initrd Default Target.1863client # [ 1.688568] systemd[1]: bpf-restrict-fs: LSM BPF program attached1864lb # [ 1.692219] systemd[1]: Created slice Slice /system/modprobe.1865lb # [ 1.693396] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866lb # [ 1.694924] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867lb # [ 1.696052] systemd[1]: Reached target Path Units.1868lb # [ 1.696961] systemd[1]: Reached target Slice Units.1869lb # [ 1.697857] systemd[1]: Reached target Swaps.1870lb # [ 1.698714] systemd[1]: Reached target Timer Units.1871worker1 # [ 1.688662] systemd[1]: Queued start job for default target Initrd Default Target.1872lb # [ 1.699697] systemd[1]: Listening on D-Bus System Message Bus Socket.1873lb # [ 1.700939] systemd[1]: Listening on Journal Socket (/dev/log).1874lb # [ 1.702140] systemd[1]: Listening on Journal Sockets.1875worker1 # [ 1.691843] systemd[1]: Created slice Slice /system/modprobe.1876lb # [ 1.703178] systemd[1]: Listening on udev Control Socket.1877worker1 # [ 1.693019] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1878lb # [ 1.704225] systemd[1]: Listening on udev Kernel Socket.1879worker2 # [ 1.700302] systemd[1]: Queued start job for default target Initrd Default Target.1880lb # [ 1.705266] systemd[1]: Reached target Socket Units.1881worker1 # [ 1.694377] systemd[1]: Expecting device /dev/disk/by-label/nixos...1882worker1 # [ 1.695450] systemd[1]: Reached target Path Units.1883worker1 # [ 1.696367] systemd[1]: Reached target Slice Units.1884lb # [ 1.706979] systemd[1]: Starting Create List of Static Device Nodes...1885worker1 # [ 1.697241] systemd[1]: Reached target Swaps.1886worker2 # [ 1.704736] systemd[1]: Created slice Slice /system/modprobe.1887worker1 # [ 1.698043] systemd[1]: Reached target Timer Units.1888worker1 # [ 1.698970] systemd[1]: Listening on D-Bus System Message Bus Socket.1889worker2 # [ 1.705953] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1890lb # [ 1.710712] systemd[1]: Starting Load Kernel Module configfs...1891worker1 # [ 1.700222] systemd[1]: Listening on Journal Socket (/dev/log).1892worker2 # [ 1.707363] systemd[1]: Expecting device /dev/disk/by-label/nixos...1893worker1 # [ 1.701377] systemd[1]: Listening on Journal Sockets.1894worker2 # [ 1.708441] systemd[1]: Reached target Path Units.1895worker2 # [ 1.709311] systemd[1]: Reached target Slice Units.1896worker1 # [ 1.702374] systemd[1]: Listening on udev Control Socket.1897worker2 # [ 1.710144] systemd[1]: Reached target Swaps.1898worker1 # [ 1.703383] systemd[1]: Listening on udev Kernel Socket.1899worker2 # [ 1.710949] systemd[1]: Reached target Timer Units.1900worker1 # [ 1.704341] systemd[1]: Reached target Socket Units.1901worker2 # [ 1.711915] systemd[1]: Listening on D-Bus System Message Bus Socket.1902worker2 # [ 1.713113] systemd[1]: Listening on Journal Socket (/dev/log).1903worker1 # [ 1.706184] systemd[1]: Starting Create List of Static Device Nodes...1904worker2 # [ 1.714208] systemd[1]: Listening on Journal Sockets.1905worker2 # [ 1.715170] systemd[1]: Listening on udev Control Socket.1906worker2 # [ 1.716175] systemd[1]: Listening on udev Kernel Socket.1907worker2 # [ 1.717170] systemd[1]: Reached target Socket Units.1908worker1 # [ 1.710559] systemd[1]: Starting Load Kernel Module configfs...1909worker2 # [ 1.719073] systemd[1]: Starting Create List of Static Device Nodes...1910worker2 # [ 1.723416] systemd[1]: Starting Load Kernel Module configfs...1911client # [ 1.729216] systemd[1]: Queued start job for default target Initrd Default Target.1912lb # [ 1.720382] systemd[1]: Starting Journal Service...1913client # [ 1.734043] systemd[1]: Created slice Slice /system/modprobe.1914client # [ 1.735300] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1915client # [ 1.736837] systemd[1]: Expecting device /dev/disk/by-label/nixos...1916client # [ 1.738044] systemd[1]: Reached target Path Units.1917worker1 # [ 1.717176] systemd[1]: Starting Journal Service...1918client # [ 1.739038] systemd[1]: Reached target Slice Units.1919client # [ 1.740065] systemd[1]: Reached target Swaps.1920client # [ 1.740947] systemd[1]: Reached target Timer Units.1921lb # [ 1.742851] systemd[1]: Starting Load Kernel Modules...1922client # [ 1.742069] systemd[1]: Listening on D-Bus System Message Bus Socket.1923client # [ 1.743407] systemd[1]: Listening on Journal Socket (/dev/log).1924client # [ 1.744704] systemd[1]: Listening on Journal Sockets.1925client # [ 1.745761] systemd[1]: Listening on udev Control Socket.1926client # [ 1.746821] systemd[1]: Listening on udev Kernel Socket.1927worker2 # [ 1.732871] systemd[1]: Starting Journal Service...1928client # [ 1.747869] systemd[1]: Reached target Socket Units.1929lb # [ 1.749027] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1930client # [ 1.749895] systemd[1]: Starting Create List of Static Device Nodes...1931worker1 # [ 1.741673] systemd[1]: Starting Load Kernel Modules...1932client # [ 1.755720] systemd[1]: Starting Load Kernel Module configfs...1933lb # [ 1.756990] systemd[1]: Starting Coldplug All udev Devices...1934worker1 # [ 1.746598] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1935worker2 # [ 1.759540] systemd[1]: Starting Load Kernel Modules...1936worker1 # [ 1.755684] systemd[1]: Starting Coldplug All udev Devices...1937worker2 # [ 1.763454] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1938lb # [ 1.768120] systemd-journald[66]: Collecting audit messages is disabled.1939lb # [ 1.769891] systemd[1]: Finished Create List of Static Device Nodes.1940worker2 # [ 1.772628] systemd[1]: Starting Coldplug All udev Devices...1941lb # [ 1.776433] systemd[1]: modprobe@configfs.service: Deactivated successfully.1942client # [ 1.764708] systemd[1]: Starting Journal Service...1943worker1 # [ 1.766707] systemd-journald[66]: Collecting audit messages is disabled.1944worker1 # [ 1.768342] systemd[1]: Finished Create List of Static Device Nodes.1945lb # [ 1.783946] systemd[1]: Finished Load Kernel Module configfs.1946worker1 # [ 1.775088] systemd[1]: modprobe@configfs.service: Deactivated successfully.1947worker2 # [ 1.783613] systemd-journald[66]: Collecting audit messages is disabled.1948lb # [ 1.790332] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1949worker2 # [ 1.787416] systemd[1]: Finished Create List of Static Device Nodes.1950worker1 # [ 1.781909] systemd[1]: Finished Load Kernel Module configfs.1951client # [ 1.792845] systemd[1]: Starting Load Kernel Modules...1952worker2 # [ 1.790155] systemd[1]: modprobe@configfs.service: Deactivated successfully.1953client # [ 1.798740] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1954worker1 # [ 1.788930] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1955lb # [ 1.802057] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1956worker2 # [ 1.799749] systemd[1]: Finished Load Kernel Module configfs.1957client # [ 1.806892] systemd[1]: Starting Coldplug All udev Devices...1958lb # [ 1.807319] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1959worker2 # [ 1.804834] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1960worker1 # [ 1.801571] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1961client # [ 1.817377] systemd-journald[66]: Collecting audit messages is disabled.1962lb # [ 1.819120] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1963worker1 # [ 1.808168] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1964worker2 # [ 1.816421] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965client # [ 1.822716] systemd[1]: Finished Create List of Static Device Nodes.1966client # [ 1.828311] systemd[1]: modprobe@configfs.service: Deactivated successfully.1967worker1 # [ 1.819563] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1968worker2 # [ 1.828541] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969client # [ 1.836065] systemd[1]: Finished Load Kernel Module configfs.1970worker2 # [ 1.836449] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971lb # [ 1.842457] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972client # [ 1.842116] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1973worker1 # [ 1.840085] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1974lb # [ 1.851109] systemd[1]: Starting Create Static Device Nodes in /dev...1975client # [ 1.855708] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1976worker1 # [ 1.848703] systemd[1]: Starting Create Static Device Nodes in /dev...1977lb # [ 1.862469] systemd[1]: Finished Load Kernel Modules.1978client # [ 1.861186] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1979worker2 # [ 1.859062] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1980lb # [ 1.870591] systemd[1]: Starting Apply Kernel Variables...1981worker1 # [ 1.861571] systemd[1]: Finished Load Kernel Modules.1982client # [ 1.872896] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1983worker2 # [ 1.870826] systemd[1]: Starting Create Static Device Nodes in /dev...1984worker1 # [ 1.871093] systemd[1]: Starting Apply Kernel Variables...1985worker2 # [ 1.884409] systemd[1]: Finished Load Kernel Modules.1986lb # [ 1.890987] systemd[1]: Finished Create Static Device Nodes in /dev.1987lb # [ 1.759575] systemd-modules-load[67]: Inserted module 'dm_mod'1988worker2 # [ 1.891580] systemd[1]: Starting Apply Kernel Variables...1989lb # [ 1.762253] systemd-modules-load[67]: Inserted module 'virtio_balloon'1990lb # [ 1.763327] systemd-modules-load[67]: Inserted module 'virtio_gpu'1991lb # [ 1.898307] systemd[1]: Started Journal Service.1992client # [ 1.898346] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1993worker1 # [ 1.891638] systemd[1]: Started Journal Service.1994worker1 # [ 1.756989] systemd-modules-load[67]: Inserted module 'dm_mod'1995worker1 # [ 1.760990] systemd-modules-load[67]: Inserted module 'virtio_balloon'1996worker1 # [ 1.763559] systemd-modules-load[67]: Inserted module 'virtio_gpu'1997worker1 # [ 1.764722] systemd[1]: Finished Create Static Device Nodes in /dev.1998lb # [ 1.777109] systemd[1]: Finished Apply Kernel Variables.1999lb # [ 1.778061] systemd[1]: Reached target Preparation for Local File Systems.2000client # [ 1.911015] systemd[1]: Starting Create Static Device Nodes in /dev...
loading 2,579 hidden lines…
4580worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4581lb: must succeed: curl -sf localhost:9901/clusters4582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4583lb: must succeed: curl -sf localhost:9901/clusters4584lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4585worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4586worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4587worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4588worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4589worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4590worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4591worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4592worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4593(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.65 seconds)4594subtest: balancer auth: client cert, bearer token, nothing, unknown CN4595worker2 # [ 32.132440] nix-grpc-daemon[782]: ts=2026-09-22T11:25:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529324596worker2 # [ 32.137323] systemd[1]: Started Nix Daemon.4597worker2 # [ 32.219977] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4598worker2 # [ 32.235255] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)4599worker2 # [ 32.319353] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 2/5)4600worker2 # [ 32.420595] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 3/5)4601worker2 # [ 32.546258] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 728 ms (attempt 4/5)4602worker2 # [ 33.278890] nix-grpc-daemon[782]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4603worker2 # [ 33.287811] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4604worker1 # [ 33.319538] nix-grpc-daemon[781]: ts=2026-09-22T11:25:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462324605worker1 # [ 33.329541] systemd[1]: Started Nix Daemon.4606worker1 # [ 33.441495] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4607worker1 # [ 33.468476] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)4608worker1 # [ 33.476874] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 2/5)4609worker1 # [ 33.576485] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 3/5)4610worker1 # [ 33.614146] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 4/5)4611worker1 # [ 33.783683] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4612worker1 # [ 33.787120] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4613worker1 # [ 33.800136] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=path_info_miss path=lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test error="error: path '/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test' is required, but there is no substituter that can build it"4614worker2 # [ 33.814609] nix-grpc-daemon[782]: ts=2026-09-22T11:25:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:529324615worker2 # [ 33.826155] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=path_info_miss path=lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test error="error: path '/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test' is required, but there is no substituter that can build it"4616client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4617client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.03 seconds)4618client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4619client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds)4620worker1 # [ 34.019945] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=info event=oidc_jwks provider=mock keys=14621worker1 # [ 34.024467] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:462324622worker1 # [ 34.036975] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:462324623worker1 # [ 34.045505] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=path_info_miss path=lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test error="error: path '/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test' is required, but there is no substituter that can build it"4624worker1 # [ 34.054515] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:462324625worker1 # [ 34.063607] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=path_info_miss path=lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test error="error: path '/nix/store/lhx66ikp98yri7sp1yzjik9bkkmk12nb-nixos-system-client-test' is required, but there is no substituter that can build it"4626client: must succeed: rm -r /var/lib/nix-grpc-store4627client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds)4628worker1 # [ 34.202804] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=info event=denied method=QueryMissing cn=- role=none4629worker2 # [ 34.328743] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=info event=denied method=QueryMissing cn=stranger role=none4630(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.39 seconds)4631subtest: DAG build is scheduled across workers and lands in the cache4632client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24633worker2 # [ 34.466208] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529324634client # don't know how to build these paths:4635client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4636worker1 # [ 34.478936] nix-grpc-daemon[781]: ts=2026-09-22T11:25:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462324637worker2 # [ 34.502339] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04638worker2 # [ 34.504158] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14639worker2 # [ 34.552357] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324640worker2 # [ 34.561722] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529324641client # copying 1 paths to 'lb:50051'...4642worker2 # [ 34.569509] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529324643client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4644worker2 # [ 34.584246] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4645worker2 # [ 34.615532] nix-grpc-daemon[782]: ts=2026-09-22T11:25:10Z level=info event=niks3_push_started pid=8834646lb # [ 34.687903] envoy[896]: conn peer=[2001:db8:1::1]:49268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=406 ms=17154647lb # [ 34.692163] envoy[896]: conn peer=[2001:db8:1::1]:45606 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=594648lb # [ 34.696096] envoy[896]: conn peer=[2001:db8:1::1]:45610 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=54649lb # [ 34.700469] envoy[896]: conn peer=[2001:db8:1::1]:45612 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=104650worker2 # [ 34.773236] nix-grpc-daemon[883]: time=2026-09-22T11:25:10.949Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4651worker2 # [ 34.868176] nix-daemon[868]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4652worker2 # [ 34.886397] nix-daemon[868]: reaped child process 899, status = succeeded4653worker2 # [ 34.971911] nix-daemon[868]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4654worker2 # [ 34.989630] nix-daemon[868]: reaped child process 904, status = succeeded4655lb # [ 35.003915] niks3-server[969]: 2026/09/22 11:25:10 INFO Received uploads request method=POST path=/api/pending_closures4656worker2 # [ 35.006289] nix-grpc-daemon[883]: time=2026-09-22T11:25:11.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4657worker2 # [ 35.008773] nix-grpc-daemon[883]: time=2026-09-22T11:25:11.184Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4658lb # [ 35.059074] niks3-server[969]: 2026/09/22 11:25:10 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4659lb # [ 35.065830] niks3-server[969]: 2026/09/22 11:25:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4660lb # [ 35.078414] niks3-server[969]: 2026/09/22 11:25:10 INFO Signed narinfos id=1 count=14661worker2 # [ 35.070142] nix-grpc-daemon[883]: time=2026-09-22T11:25:11.246Z level=INFO msg="Uploading 1 narinfos"4662lb # [ 35.084032] niks3-server[969]: 2026/09/22 11:25:10 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4663lb # [ 35.089558] niks3-server[969]: 2026/09/22 11:25:10 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4664lb # [ 35.100550] niks3-server[969]: 2026/09/22 11:25:10 INFO Completed upload id=14665worker2 # [ 35.088732] nix-grpc-daemon[883]: time=2026-09-22T11:25:11.265Z level=INFO msg="Upload complete. (311ms)"4666worker2 # [ 35.093892] nix-grpc-daemon[782]: ts=2026-09-22T11:25:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=5044667worker2 # [ 35.102377] nix-grpc-daemon[782]: ts=2026-09-22T11:25:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324668worker2 # [ 35.111474] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4669worker2 # [ 35.124890] nix-daemon[868]: reaped child process 881, status = succeeded4670worker2 # [ 35.127929] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4671client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4672client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4673client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4674client # farm-a> LOG-farm-a-t14675worker2 # [ 36.146186] nix-daemon[868]: reaped child process 908, status = succeeded4676worker2 # [ 36.229201] nix-daemon[868]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4677worker2 # [ 36.246383] nix-daemon[868]: reaped child process 927, status = succeeded4678lb # [ 36.264791] niks3-server[969]: 2026/09/22 11:25:11 INFO Received uploads request method=POST path=/api/pending_closures4679worker2 # [ 36.265321] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4680worker2 # [ 36.266802] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.443Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4681lb # [ 36.327436] niks3-server[969]: 2026/09/22 11:25:11 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4682lb # [ 36.330670] niks3-server[969]: 2026/09/22 11:25:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4683lb # [ 36.338582] niks3-server[969]: 2026/09/22 11:25:11 INFO Signed narinfos id=2 count=14684lb # [ 36.339822] niks3-server[969]: 2026/09/22 11:25:11 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4685worker2 # [ 36.327679] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.503Z level=INFO msg="Uploading 1 narinfos"4686lb # [ 36.346190] niks3-server[969]: 2026/09/22 11:25:11 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4687lb # [ 36.348206] niks3-server[969]: 2026/09/22 11:25:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4688lb # [ 36.354411] niks3-server[969]: 2026/09/22 11:25:11 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4689lb # [ 36.356778] niks3-server[969]: 2026/09/22 11:25:11 INFO Completed upload id=24690worker2 # [ 36.344856] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.521Z level=INFO msg="Upload complete. (209ms)"4691worker2 # [ 36.358650] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14692worker2 # [ 36.366275] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04693worker2 # [ 36.369325] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14694worker2 # [ 36.378328] nix-daemon[868]: reaped child process 906, status = succeeded4695worker2 # [ 36.408889] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324696worker2 # [ 36.421898] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529324697worker2 # [ 36.434511] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.611Z level=INFO msg="All 1 paths already cached"4698client # copying 1 paths to 'lb:50051'...4699worker2 # [ 36.442527] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529324700client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4701worker2 # [ 36.461166] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4702worker2 # [ 36.568092] nix-daemon[868]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4703worker2 # [ 36.583086] nix-daemon[868]: reaped child process 936, status = succeeded4704worker2 # [ 36.672656] nix-daemon[868]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4705worker2 # [ 36.686803] nix-daemon[868]: reaped child process 941, status = succeeded4706lb # [ 36.706400] niks3-server[969]: 2026/09/22 11:25:12 INFO Received uploads request method=POST path=/api/pending_closures4707worker2 # [ 36.702316] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.879Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4708worker2 # [ 36.703906] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.880Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4709lb # [ 36.734519] niks3-server[969]: 2026/09/22 11:25:12 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4710lb # [ 36.746581] niks3-server[969]: 2026/09/22 11:25:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4711lb # [ 36.750969] niks3-server[969]: 2026/09/22 11:25:12 INFO Signed narinfos id=3 count=14712worker2 # [ 36.741703] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.918Z level=INFO msg="Uploading 1 narinfos"4713lb # [ 36.760156] niks3-server[969]: 2026/09/22 11:25:12 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4714lb # [ 36.772402] niks3-server[969]: 2026/09/22 11:25:12 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4715lb # [ 36.782893] niks3-server[969]: 2026/09/22 11:25:12 INFO Completed upload id=34716worker2 # [ 36.773120] nix-grpc-daemon[883]: time=2026-09-22T11:25:12.949Z level=INFO msg="Upload complete. (298ms)"4717lb # [ 36.786109] niks3-server[969]: 2026/09/22 11:25:12 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4718worker2 # [ 36.777365] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=6244719worker2 # [ 36.783716] nix-grpc-daemon[782]: ts=2026-09-22T11:25:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324720worker2 # [ 36.797064] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4721worker2 # [ 36.800165] nix-daemon[868]: reaped child process 931, status = succeeded4722worker2 # [ 36.805623] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4723client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4724client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4725client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4726client # farm-b> LOG-farm-b-t14727worker2 # [ 37.844379] nix-daemon[868]: reaped child process 945, status = succeeded4728worker2 # [ 37.926663] nix-daemon[868]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4729worker2 # [ 37.942990] nix-daemon[868]: reaped child process 965, status = succeeded4730lb # [ 37.960948] niks3-server[969]: 2026/09/22 11:25:13 INFO Received uploads request method=POST path=/api/pending_closures4731worker2 # [ 37.958931] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.135Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4732worker2 # [ 37.961202] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.137Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4733lb # [ 37.995811] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4734lb # [ 38.010916] niks3-server[969]: 2026/09/22 11:25:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4735lb # [ 38.017572] niks3-server[969]: 2026/09/22 11:25:13 INFO Signed narinfos id=4 count=14736worker2 # [ 38.008481] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.185Z level=INFO msg="Uploading 1 narinfos"4737lb # [ 38.024460] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4738lb # [ 38.032902] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4739lb # [ 38.041155] niks3-server[969]: 2026/09/22 11:25:13 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4740lb # [ 38.045141] niks3-server[969]: 2026/09/22 11:25:13 INFO Completed upload id=44741worker2 # [ 38.034558] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.211Z level=INFO msg="Upload complete. (219ms)"4742lb # [ 38.048666] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4743worker2 # [ 38.042635] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14744worker2 # [ 38.053523] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04745worker2 # [ 38.058230] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14746worker2 # [ 38.068047] nix-daemon[868]: reaped child process 943, status = succeeded4747worker2 # [ 38.098255] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324748worker2 # [ 38.113236] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529324749worker2 # [ 38.127268] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.304Z level=INFO msg="All 2 paths already cached"4750client # copying 1 paths to 'lb:50051'...4751client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4752worker2 # [ 38.134605] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529324753worker2 # [ 38.147333] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4754worker2 # [ 38.247060] nix-daemon[868]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4755worker2 # [ 38.260477] nix-daemon[868]: reaped child process 974, status = succeeded4756worker2 # [ 38.346375] nix-daemon[868]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4757worker2 # [ 38.360169] nix-daemon[868]: reaped child process 979, status = succeeded4758lb # [ 38.380431] niks3-server[969]: 2026/09/22 11:25:13 INFO Received uploads request method=POST path=/api/pending_closures4759worker2 # [ 38.376627] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.553Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4760worker2 # [ 38.378126] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.553Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4761lb # [ 38.406733] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4762lb # [ 38.416662] niks3-server[969]: 2026/09/22 11:25:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4763lb # [ 38.422664] niks3-server[969]: 2026/09/22 11:25:13 INFO Signed narinfos id=5 count=14764worker2 # [ 38.413590] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.590Z level=INFO msg="Uploading 1 narinfos"4765lb # [ 38.435412] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4766lb # [ 38.442976] niks3-server[969]: 2026/09/22 11:25:13 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4767lb # [ 38.450050] niks3-server[969]: 2026/09/22 11:25:13 INFO Completed upload id=54768worker2 # [ 38.437587] nix-grpc-daemon[883]: time=2026-09-22T11:25:14.614Z level=INFO msg="Upload complete. (277ms)"4769lb # [ 38.452518] niks3-server[969]: 2026/09/22 11:25:13 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4770worker2 # [ 38.440877] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=7524771worker2 # [ 38.446289] nix-grpc-daemon[782]: ts=2026-09-22T11:25:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324772worker2 # [ 38.455686] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4773worker2 # [ 38.462129] nix-daemon[868]: reaped child process 969, status = succeeded4774worker2 # [ 38.465448] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4775client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4776client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4777client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4778client # farm-top> LOG-farm-top-t14779worker2 # [ 39.110165] nix-daemon[868]: reaped child process 983, status = succeeded4780worker2 # [ 39.189760] nix-daemon[868]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4781worker2 # [ 39.203709] nix-daemon[868]: reaped child process 1002, status = succeeded4782lb # [ 39.222404] niks3-server[969]: 2026/09/22 11:25:14 INFO Received uploads request method=POST path=/api/pending_closures4783worker2 # [ 39.219113] nix-grpc-daemon[883]: time=2026-09-22T11:25:15.395Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4784worker2 # [ 39.220629] nix-grpc-daemon[883]: time=2026-09-22T11:25:15.397Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4785lb # [ 39.259859] niks3-server[969]: 2026/09/22 11:25:14 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4786lb # [ 39.276994] niks3-server[969]: 2026/09/22 11:25:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4787lb # [ 39.279883] niks3-server[969]: 2026/09/22 11:25:14 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4788lb # [ 39.283857] niks3-server[969]: 2026/09/22 11:25:14 INFO Signed narinfos id=6 count=14789worker2 # [ 39.274323] nix-grpc-daemon[883]: time=2026-09-22T11:25:15.450Z level=INFO msg="Uploading 1 narinfos"4790lb # [ 39.289937] niks3-server[969]: 2026/09/22 11:25:14 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4791lb # [ 39.297885] niks3-server[969]: 2026/09/22 11:25:14 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4792lb # [ 39.301688] niks3-server[969]: 2026/09/22 11:25:14 INFO Completed upload id=64793worker2 # [ 39.290867] nix-grpc-daemon[883]: time=2026-09-22T11:25:15.467Z level=INFO msg="Upload complete. (218ms)"4794lb # [ 39.304465] niks3-server[969]: 2026/09/22 11:25:14 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4795worker2 # [ 39.299304] nix-grpc-daemon[782]: ts=2026-09-22T11:25:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14796worker2 # [ 39.313113] nix-daemon[868]: reaped child process 981, status = succeeded4797client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 4.98 seconds)4798client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4799client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4800client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4801client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4802client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4803client # copying 3 paths...4804client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4805client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4806client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4807client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4808(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.32 seconds)4809subtest: repeat is answered Cached by the scheduler without building4810worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4811worker1 # finding garbage collector roots...4812worker1 # deleting specified paths...4813worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4814worker1 # deleting unused links...4815worker1 # note: hard linking is currently saving 0.0 KiB4816worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4817worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4818worker2 # finding garbage collector roots...4819worker2 # deleting specified paths...4820worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4821worker2 # deleting unused links...4822worker2 # note: hard linking is currently saving 0.0 KiB4823worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.13 seconds)4824client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24825worker2 # [ 40.105463] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529324826client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4827client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4828worker1 # [ 40.120876] nix-grpc-daemon[781]: ts=2026-09-22T11:25:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462324829worker2 # [ 40.141402] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14830client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4831client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4832client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4833lb: must succeed: curl -sf localhost:9901/clusters4834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4835lb: must succeed: curl -sf localhost:9901/clusters4836lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4837worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4838worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4839(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.77 seconds)4840subtest: inputs already in the cache are not sent to the scheduler4841lb: must succeed: curl -sf localhost:9901/clusters4842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4843lb: must succeed: curl -sf localhost:9901/clusters4844lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4845worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4846worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4847client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24848worker1 # [ 40.654388] nix-grpc-daemon[781]: ts=2026-09-22T11:25:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462324849client # don't know how to build these paths:4850client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4851worker2 # [ 40.662031] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529324852worker2 # [ 40.667688] nix-grpc-daemon[883]: time=2026-09-22T11:25:16.844Z level=INFO msg="All 2 paths already cached"4853worker2 # [ 40.678269] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04854worker2 # [ 40.683284] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14855worker2 # [ 40.688960] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324856worker2 # [ 40.702695] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529324857worker2 # [ 40.713877] nix-grpc-daemon[883]: time=2026-09-22T11:25:16.890Z level=INFO msg="All 2 paths already cached"4858client # copying 1 paths to 'lb:50051'...4859worker2 # [ 40.720707] nix-grpc-daemon[782]: ts=2026-09-22T11:25:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529324860client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4861worker2 # [ 40.735655] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4862worker2 # [ 40.829547] nix-daemon[868]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4863worker2 # [ 40.847607] nix-daemon[868]: reaped child process 1035, status = succeeded4864worker2 # [ 40.937600] nix-daemon[868]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)4865worker2 # [ 40.951154] nix-daemon[868]: reaped child process 1040, status = succeeded4866lb # [ 40.972379] niks3-server[969]: 2026/09/22 11:25:16 INFO Received uploads request method=POST path=/api/pending_closures4867worker2 # [ 40.967948] nix-grpc-daemon[883]: time=2026-09-22T11:25:17.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4868worker2 # [ 40.969463] nix-grpc-daemon[883]: time=2026-09-22T11:25:17.146Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4869lb # [ 41.000967] niks3-server[969]: 2026/09/22 11:25:16 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4870lb # [ 41.017297] niks3-server[969]: 2026/09/22 11:25:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4871lb # [ 41.025151] niks3-server[969]: 2026/09/22 11:25:16 INFO Signed narinfos id=7 count=14872worker2 # [ 41.016531] nix-grpc-daemon[883]: time=2026-09-22T11:25:17.193Z level=INFO msg="Uploading 1 narinfos"4873lb # [ 41.032106] niks3-server[969]: 2026/09/22 11:25:16 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4874lb # [ 41.044863] niks3-server[969]: 2026/09/22 11:25:16 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4875lb # [ 41.052613] niks3-server[969]: 2026/09/22 11:25:16 INFO Completed upload id=74876worker2 # [ 41.040521] nix-grpc-daemon[883]: time=2026-09-22T11:25:17.217Z level=INFO msg="Upload complete. (295ms)"4877worker2 # [ 41.042532] nix-grpc-daemon[782]: ts=2026-09-22T11:25:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=7524878lb # [ 41.055688] niks3-server[969]: 2026/09/22 11:25:16 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4879worker2 # [ 41.045710] nix-grpc-daemon[782]: ts=2026-09-22T11:25:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529324880worker2 # [ 41.056099] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4881worker2 # [ 41.061875] nix-daemon[868]: reaped child process 1030, status = succeeded4882worker2 # [ 41.067494] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4883client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4884client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4885client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4886client # farm-top> LOG-farm-top-t1b4887worker2 # [ 42.085670] nix-daemon[868]: reaped child process 1044, status = succeeded4888worker2 # [ 42.159636] nix-daemon[868]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)4889worker2 # [ 42.174916] nix-daemon[868]: reaped child process 1063, status = succeeded4890lb # [ 42.192444] niks3-server[969]: 2026/09/22 11:25:17 INFO Received uploads request method=POST path=/api/pending_closures4891worker2 # [ 42.188044] nix-grpc-daemon[883]: time=2026-09-22T11:25:18.364Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4892worker2 # [ 42.189570] nix-grpc-daemon[883]: time=2026-09-22T11:25:18.366Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4893lb # [ 42.234623] niks3-server[969]: 2026/09/22 11:25:17 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4894lb # [ 42.241277] niks3-server[969]: 2026/09/22 11:25:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4895worker2 # [ 42.237792] nix-grpc-daemon[883]: time=2026-09-22T11:25:18.414Z level=INFO msg="Uploading 1 narinfos"4896lb # [ 42.251242] niks3-server[969]: 2026/09/22 11:25:17 INFO Signed narinfos id=8 count=14897lb # [ 42.259823] niks3-server[969]: 2026/09/22 11:25:17 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4898lb # [ 42.274930] niks3-server[969]: 2026/09/22 11:25:17 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4899lb # [ 42.277888] niks3-server[969]: 2026/09/22 11:25:17 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4900lb # [ 42.280504] niks3-server[969]: 2026/09/22 11:25:17 INFO Completed upload id=84901worker2 # [ 42.268319] nix-grpc-daemon[883]: time=2026-09-22T11:25:18.445Z level=INFO msg="Upload complete. (190ms)"4902worker2 # [ 42.271722] nix-grpc-daemon[782]: ts=2026-09-22T11:25:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14903worker2 # [ 42.279048] nix-daemon[868]: reaped child process 1042, status = succeeded4904client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.79 seconds)4905client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4906client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4907lb: must succeed: curl -sf localhost:9901/clusters4908lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4909lb: must succeed: curl -sf localhost:9901/clusters4910lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4911worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4912worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4913(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.07 seconds)4914subtest: two clients wanting the same drv build it once4915client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4916client # [ 42.550563] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4917client # Running as unit: dedup1.service; invocation ID: be51f7325ab14e17bdd9bd533f514cd34918client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.05 seconds)4919client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4920client # [ 42.681347] nix[887]: don't know how to build these paths:4921client # [ 42.682227] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4922worker1 # [ 42.675819] nix-grpc-daemon[781]: ts=2026-09-22T11:25:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462324923worker2 # [ 42.679829] nix-grpc-daemon[782]: ts=2026-09-22T11:25:18Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04924worker2 # [ 42.681915] nix-grpc-daemon[782]: ts=2026-09-22T11:25:18Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14925worker1 # [ 42.687626] nix-grpc-daemon[781]: ts=2026-09-22T11:25:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462324926worker1 # [ 42.699105] nix-grpc-daemon[781]: ts=2026-09-22T11:25:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462324927client # [ 42.713075] nix[887]: copying 1 paths to 'lb:50051'...4928client # [ 42.713867] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4929worker1 # [ 42.709307] nix-grpc-daemon[781]: ts=2026-09-22T11:25:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462324930worker1 # [ 42.720261] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4931worker1 # [ 42.740196] nix-grpc-daemon[781]: ts=2026-09-22T11:25:17Z level=info event=niks3_push_started pid=8754932worker1 # [ 42.867465] nix-grpc-daemon[875]: time=2026-09-22T11:25:18.077Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4933worker1 # [ 42.956087] nix-daemon[847]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4934worker1 # [ 42.970764] nix-daemon[847]: reaped child process 891, status = succeeded4935worker1 # [ 43.044753] nix-daemon[847]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4936worker1 # [ 43.060290] nix-daemon[847]: reaped child process 896, status = succeeded4937lb # [ 43.069842] niks3-server[969]: 2026/09/22 11:25:18 INFO Received uploads request method=POST path=/api/pending_closures4938worker1 # [ 43.070647] nix-grpc-daemon[875]: time=2026-09-22T11:25:18.280Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4939worker1 # [ 43.072700] nix-grpc-daemon[875]: time=2026-09-22T11:25:18.280Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4940lb # [ 43.104633] niks3-server[969]: 2026/09/22 11:25:18 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4941lb # [ 43.111488] niks3-server[969]: 2026/09/22 11:25:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4942lb # [ 43.118957] niks3-server[969]: 2026/09/22 11:25:18 INFO Signed narinfos id=9 count=14943worker1 # [ 43.111975] nix-grpc-daemon[875]: time=2026-09-22T11:25:18.322Z level=INFO msg="Uploading 1 narinfos"4944lb # [ 43.128294] niks3-server[969]: 2026/09/22 11:25:18 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4945lb # [ 43.138393] niks3-server[969]: 2026/09/22 11:25:18 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4946lb # [ 43.148873] niks3-server[969]: 2026/09/22 11:25:18 INFO Completed upload id=94947worker1 # [ 43.138704] nix-grpc-daemon[875]: time=2026-09-22T11:25:18.349Z level=INFO msg="Upload complete. (267ms)"4948lb # [ 43.150401] niks3-server[969]: 2026/09/22 11:25:18 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4949worker1 # [ 43.140989] nix-grpc-daemon[781]: ts=2026-09-22T11:25:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=4324950worker1 # [ 43.145721] nix-grpc-daemon[781]: ts=2026-09-22T11:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462324951worker1 # [ 43.156064] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4952worker1 # [ 43.160794] nix-daemon[847]: reaped child process 873, status = succeeded4953client # [ 43.169153] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4954worker1 # [ 43.164851] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4955client # [ 43.596880] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4956client # Running as unit: dedup2.service; invocation ID: 251b88ce768143bd9a05518b9cfe4d784957client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.04 seconds)4958client # [ 43.727339] nix[903]: don't know how to build these paths:4959worker2 # [ 43.718521] nix-grpc-daemon[782]: ts=2026-09-22T11:25:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529324960client # [ 43.728173] nix[903]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4961worker2 # [ 43.729983] nix-grpc-daemon[782]: ts=2026-09-22T11:25:19Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04962worker1 # [ 43.735836] nix-grpc-daemon[781]: ts=2026-09-22T11:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462324963client # [ 43.745505] nix[903]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4964lb # [ 44.688581] envoy[896]: conn peer=[2001:db8:1::1]:45616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9464 tx=4044 ms=48464965lb # [ 44.692567] envoy[896]: conn peer=[2001:db8:1::1]:45618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=479 ms=474966lb # [ 44.696140] envoy[896]: conn peer=[2001:db8:1::1]:45634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1788 ms=16344967client: must succeed: sleep 24968client: (finished: must succeed: sleep 2, in 2.03 seconds)4969client: waiting for success: ! systemctl is-active dedup1 dedup24970worker1 # [ 46.995595] nix-daemon[847]: reaped child process 900, status = succeeded4971worker1 # [ 47.073851] nix-daemon[847]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4972worker1 # [ 47.088160] nix-daemon[847]: reaped child process 933, status = succeeded4973lb # [ 47.102912] niks3-server[969]: 2026/09/22 11:25:22 INFO Received uploads request method=POST path=/api/pending_closures4974worker1 # [ 47.101055] nix-grpc-daemon[875]: time=2026-09-22T11:25:22.311Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4975worker1 # [ 47.104058] nix-grpc-daemon[875]: time=2026-09-22T11:25:22.312Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4976lb # [ 47.149300] niks3-server[969]: 2026/09/22 11:25:22 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4977lb # [ 47.155438] niks3-server[969]: 2026/09/22 11:25:22 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4978lb # [ 47.166956] niks3-server[969]: 2026/09/22 11:25:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4979lb # [ 47.171064] niks3-server[969]: 2026/09/22 11:25:22 INFO Signed narinfos id=10 count=14980worker1 # [ 47.162160] nix-grpc-daemon[875]: time=2026-09-22T11:25:22.371Z level=INFO msg="Uploading 1 narinfos"4981lb # [ 47.180727] niks3-server[969]: 2026/09/22 11:25:22 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4982lb # [ 47.185168] niks3-server[969]: 2026/09/22 11:25:22 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4983lb # [ 47.189793] niks3-server[969]: 2026/09/22 11:25:22 INFO Completed upload id=104984lb # [ 47.191129] niks3-server[969]: 2026/09/22 11:25:22 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4985worker1 # [ 47.179771] nix-grpc-daemon[875]: time=2026-09-22T11:25:22.390Z level=INFO msg="Upload complete. (202ms)"4986worker1 # [ 47.186536] nix-grpc-daemon[781]: ts=2026-09-22T11:25:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14987worker1 # [ 47.193493] nix-grpc-daemon[781]: ts=2026-09-22T11:25:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14988worker1 # [ 47.206159] nix-daemon[847]: reaped child process 898, status = succeeded4989client # [ 47.245488] systemd[1]: dedup1.service: Deactivated successfully.4990client # [ 47.248258] systemd[1]: dedup1.service: Consumed 110ms CPU time over 4.694s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.5K outgoing IP traffic.4991client # [ 47.253648] systemd[1]: dedup2.service: Deactivated successfully.4992client # [ 47.256628] systemd[1]: dedup2.service: Consumed 135ms CPU time over 3.659s wall clock time, 39.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4993client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4994client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4995client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4996worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4997worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4998worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4999worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5000worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5001worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5002worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5003worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5004(finished: subtest: two clients wanting the same drv build it once, in 5.70 seconds)5005subtest: --timeout and --max-silent-time reach the builder5006worker1 # [ 48.349963] nix-grpc-daemon[781]: ts=2026-09-22T11:25:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462325007worker2 # [ 48.357935] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=want client=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05008worker2 # [ 48.363827] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15009worker2 # [ 48.370904] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325010worker2 # [ 48.378665] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325011worker2 # [ 48.389425] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529325012worker2 # [ 48.400843] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5013worker2 # [ 48.506084] nix-daemon[868]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)5014worker2 # [ 48.521921] nix-daemon[868]: reaped child process 1102, status = succeeded5015worker2 # [ 48.611602] nix-daemon[868]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)5016worker2 # [ 48.625652] nix-daemon[868]: reaped child process 1107, status = succeeded5017lb # [ 48.645818] niks3-server[969]: 2026/09/22 11:25:24 INFO Received uploads request method=POST path=/api/pending_closures5018worker2 # [ 48.639744] nix-grpc-daemon[883]: time=2026-09-22T11:25:24.816Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5019worker2 # [ 48.641309] nix-grpc-daemon[883]: time=2026-09-22T11:25:24.818Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5020lb # [ 48.676175] niks3-server[969]: 2026/09/22 11:25:24 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5021lb # [ 48.683893] niks3-server[969]: 2026/09/22 11:25:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5022worker2 # [ 48.678695] nix-grpc-daemon[883]: time=2026-09-22T11:25:24.855Z level=INFO msg="Uploading 1 narinfos"5023lb # [ 48.690654] niks3-server[969]: 2026/09/22 11:25:24 INFO Signed narinfos id=11 count=15024lb # [ 48.701109] niks3-server[969]: 2026/09/22 11:25:24 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5025lb # [ 48.709763] niks3-server[969]: 2026/09/22 11:25:24 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5026worker2 # [ 48.705098] nix-grpc-daemon[883]: time=2026-09-22T11:25:24.882Z level=INFO msg="Upload complete. (293ms)"5027lb # [ 48.717445] niks3-server[969]: 2026/09/22 11:25:24 INFO Completed upload id=115028lb # [ 48.720854] niks3-server[969]: 2026/09/22 11:25:24 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5029worker2 # [ 48.708383] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=4245030worker2 # [ 48.713478] nix-grpc-daemon[782]: ts=2026-09-22T11:25:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325031worker2 # [ 48.722474] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5032worker2 # [ 48.731072] nix-daemon[868]: reaped child process 1097, status = succeeded5033worker2 # [ 48.736777] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5034lb # [ 54.689613] envoy[896]: conn peer=[2001:db8:1::1]:45644 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1448 ms=45265035lb # [ 54.693373] envoy[896]: conn peer=[2001:db8:1::1]:34332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=35045036worker2 # [ 54.949485] nix-grpc-daemon[782]: ts=2026-09-22T11:25:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=6 outputs=05037worker2 # [ 54.963237] nix-daemon[868]: reaped child process 1109, status = succeeded5038worker2 # [ 54.964540] nix-daemon[868]: reaped child process 1111, status = succeeded5039worker2 # [ 55.109229] nix-grpc-daemon[782]: ts=2026-09-22T11:25:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529325040worker2 # [ 55.118673] nix-grpc-daemon[782]: ts=2026-09-22T11:25:31Z level=debug event=want client=7 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05041worker1 # [ 55.123880] nix-grpc-daemon[781]: ts=2026-09-22T11:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325042worker2 # [ 55.122929] nix-grpc-daemon[782]: ts=2026-09-22T11:25:31Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15043worker1 # [ 55.134217] nix-grpc-daemon[781]: ts=2026-09-22T11:25:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325044worker1 # [ 55.142784] nix-grpc-daemon[781]: ts=2026-09-22T11:25:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325045worker1 # [ 55.153733] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5046worker1 # [ 55.254457] nix-daemon[847]: accepted connection from pid 946, user nix-grpc-daemon (trusted)5047worker1 # [ 55.268137] nix-daemon[847]: reaped child process 949, status = succeeded5048worker1 # [ 55.353655] nix-daemon[847]: accepted connection from pid 951, user nix-grpc-daemon (trusted)5049worker1 # [ 55.365810] nix-daemon[847]: reaped child process 954, status = succeeded5050lb # [ 55.381140] niks3-server[969]: 2026/09/22 11:25:30 INFO Received uploads request method=POST path=/api/pending_closures5051worker1 # [ 55.376614] nix-grpc-daemon[875]: time=2026-09-22T11:25:30.586Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5052worker1 # [ 55.378590] nix-grpc-daemon[875]: time=2026-09-22T11:25:30.588Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5053lb # [ 55.397059] niks3-server[969]: 2026/09/22 11:25:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5054lb # [ 55.399260] niks3-server[969]: 2026/09/22 11:25:30 INFO Signed narinfos id=12 count=15055worker1 # [ 55.390886] nix-grpc-daemon[875]: time=2026-09-22T11:25:30.601Z level=INFO msg="Uploading 1 narinfos"5056lb # [ 55.406219] niks3-server[969]: 2026/09/22 11:25:30 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5057lb # [ 55.418143] niks3-server[969]: 2026/09/22 11:25:30 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5058lb # [ 55.426728] niks3-server[969]: 2026/09/22 11:25:30 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5059lb # [ 55.432057] niks3-server[969]: 2026/09/22 11:25:30 INFO Completed upload id=125060worker1 # [ 55.421833] nix-grpc-daemon[875]: time=2026-09-22T11:25:30.632Z level=INFO msg="Upload complete. (256ms)"5061worker1 # [ 55.423858] nix-grpc-daemon[781]: ts=2026-09-22T11:25:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=4245062lb # [ 55.435307] niks3-server[969]: 2026/09/22 11:25:30 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5063worker1 # [ 55.427490] nix-grpc-daemon[781]: ts=2026-09-22T11:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325064worker1 # [ 55.434134] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5065worker1 # [ 55.436380] nix-daemon[847]: reaped child process 944, status = succeeded5066worker1 # [ 55.441372] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5067worker1 # [ 61.255191] nix-grpc-daemon[781]: ts=2026-09-22T11:25:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=7 outputs=05068worker1 # [ 61.279887] nix-daemon[847]: reaped child process 956, status = succeeded5069worker1 # [ 61.282258] nix-daemon[847]: reaped child process 958, status = succeeded5070(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.16 seconds)5071subtest: the first client leaving does not fail the second5072client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5073client # [ 61.423916] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5074client # Running as unit: share1.service; invocation ID: 127f24b33dd148dd9bdab7ae4d7bd6ca5075client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5076client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5077client # [ 61.565546] nix[976]: don't know how to build these paths:5078client # [ 61.567094] nix[976]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5079worker1 # [ 61.559895] nix-grpc-daemon[781]: ts=2026-09-22T11:25:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462325080worker2 # [ 61.569920] nix-grpc-daemon[782]: ts=2026-09-22T11:25:37Z level=debug event=want client=8 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05081worker2 # [ 61.575821] nix-grpc-daemon[782]: ts=2026-09-22T11:25:37Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15082worker2 # [ 61.580293] nix-grpc-daemon[782]: ts=2026-09-22T11:25:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325083worker2 # [ 61.592078] nix-grpc-daemon[782]: ts=2026-09-22T11:25:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325084client # [ 61.608612] nix[976]: copying 1 paths to 'lb:50051'...5085client # [ 61.609552] nix[976]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5086worker2 # [ 61.603297] nix-grpc-daemon[782]: ts=2026-09-22T11:25:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529325087worker2 # [ 61.611949] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5088worker2 # [ 61.717274] nix-daemon[868]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)5089worker2 # [ 61.731783] nix-daemon[868]: reaped child process 1141, status = succeeded5090worker2 # [ 61.817081] nix-daemon[868]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)5091worker2 # [ 61.831247] nix-daemon[868]: reaped child process 1146, status = succeeded5092lb # [ 61.850559] niks3-server[969]: 2026/09/22 11:25:37 INFO Received uploads request method=POST path=/api/pending_closures5093worker2 # [ 61.842863] nix-grpc-daemon[883]: time=2026-09-22T11:25:38.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5094worker2 # [ 61.844446] nix-grpc-daemon[883]: time=2026-09-22T11:25:38.021Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5095lb # [ 61.868902] niks3-server[969]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5096lb # [ 61.870569] niks3-server[969]: 2026/09/22 11:25:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5097lb # [ 61.873232] niks3-server[969]: 2026/09/22 11:25:37 INFO Signed narinfos id=13 count=15098worker2 # [ 61.860918] nix-grpc-daemon[883]: time=2026-09-22T11:25:38.037Z level=INFO msg="Uploading 1 narinfos"5099lb # [ 61.877043] niks3-server[969]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5100lb # [ 61.881171] niks3-server[969]: 2026/09/22 11:25:37 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5101lb # [ 61.886356] niks3-server[969]: 2026/09/22 11:25:37 INFO Completed upload id=135102worker2 # [ 61.874114] nix-grpc-daemon[883]: time=2026-09-22T11:25:38.051Z level=INFO msg="Upload complete. (248ms)"5103lb # [ 61.887593] niks3-server[969]: 2026/09/22 11:25:37 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5104worker2 # [ 61.876368] nix-grpc-daemon[782]: ts=2026-09-22T11:25:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=4325105worker2 # [ 61.879076] nix-grpc-daemon[782]: ts=2026-09-22T11:25:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325106worker2 # [ 61.887279] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5107worker2 # [ 61.891100] nix-daemon[868]: reaped child process 1136, status = succeeded5108client # [ 61.900571] nix[976]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5109worker2 # [ 61.894754] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5110client # [ 62.490866] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5111client # Running as unit: share2.service; invocation ID: dc3c66b8480f4de584ce778f7ab4f4ed5112client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5113worker1 # [ 62.625766] nix-grpc-daemon[781]: ts=2026-09-22T11:25:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462325114client # [ 62.636623] nix[992]: don't know how to build these paths:5115client # [ 62.637643] nix[992]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5116worker2 # [ 62.635739] nix-grpc-daemon[782]: ts=2026-09-22T11:25:38Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05117worker2 # [ 62.638538] nix-grpc-daemon[782]: ts=2026-09-22T11:25:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325118client # [ 62.645720] nix[992]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5119client: must succeed: sleep 2; systemctl kill -s INT share15120lb # [ 64.689667] envoy[896]: conn peer=[2001:db8:1::1]:34342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2983 tx=977 ms=66195121lb # [ 64.693533] envoy[896]: conn peer=[2001:db8:1::1]:58894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=933 ms=61585122client # [ 65.707679] systemd[1]: share1.service: Sent signal SIGINT to main process 976 (nix) on client request.5123client # [ 65.715814] nix[976]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5124client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)5125client: waiting for success: ! systemctl is-active share15126client # [ 65.724148] nix[976]: error: interrupted by the user5127client # [ 65.762473] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5128client # [ 65.764067] systemd[1]: share1.service: Failed with result 'exit-code'.5129client # [ 65.765954] systemd[1]: share1.service: Consumed 110ms CPU time over 4.341s wall clock time, 41.4M memory peak, 6K incoming IP traffic, 8K outgoing IP traffic.5130client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5131client: waiting for success: ! systemctl is-active share25132worker2 # [ 67.138539] nix-daemon[868]: reaped child process 1150, status = succeeded5133worker2 # [ 67.219939] nix-daemon[868]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5134worker2 # [ 67.233195] nix-daemon[868]: reaped child process 1189, status = succeeded5135lb # [ 67.252222] niks3-server[969]: 2026/09/22 11:25:42 INFO Received uploads request method=POST path=/api/pending_closures5136worker2 # [ 67.249292] nix-grpc-daemon[883]: time=2026-09-22T11:25:43.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5137worker2 # [ 67.253052] nix-grpc-daemon[883]: time=2026-09-22T11:25:43.427Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5138lb # [ 67.292954] niks3-server[969]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5139lb # [ 67.301613] niks3-server[969]: 2026/09/22 11:25:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5140lb # [ 67.307510] niks3-server[969]: 2026/09/22 11:25:42 INFO Signed narinfos id=14 count=15141worker2 # [ 67.298380] nix-grpc-daemon[883]: time=2026-09-22T11:25:43.475Z level=INFO msg="Uploading 1 narinfos"5142lb # [ 67.314816] niks3-server[969]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5143lb # [ 67.316808] niks3-server[969]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5144lb # [ 67.323060] niks3-server[969]: 2026/09/22 11:25:42 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5145lb # [ 67.327212] niks3-server[969]: 2026/09/22 11:25:42 INFO Completed upload id=145146lb # [ 67.329169] niks3-server[969]: 2026/09/22 11:25:42 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5147worker2 # [ 67.315062] nix-grpc-daemon[883]: time=2026-09-22T11:25:43.492Z level=INFO msg="Upload complete. (208ms)"5148worker2 # [ 67.322320] nix-grpc-daemon[782]: ts=2026-09-22T11:25:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15149worker2 # [ 67.326782] nix-grpc-daemon[782]: ts=2026-09-22T11:25:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15150worker2 # [ 67.338071] nix-daemon[868]: reaped child process 1148, status = succeeded5151client # [ 67.398910] systemd[1]: share2.service: Deactivated successfully.5152client # [ 67.402565] systemd[1]: share2.service: Consumed 115ms CPU time over 4.911s wall clock time, 35.5M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.5153client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5154client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5155client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5156(finished: subtest: the first client leaving does not fail the second, in 6.71 seconds)5157subtest: build hook: nix-daemon with builders = grpc://lb5158client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5159client # [ 68.173858] systemd[1]: Started Nix Daemon.5160client # [ 68.251257] nix-daemon[1044]: accepted connection from pid 1041, user root (trusted)5161worker2 # [ 68.346913] nix-grpc-daemon[782]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:529325162worker1 # [ 68.353489] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325163worker2 # [ 68.358724] nix-grpc-daemon[782]: ts=2026-09-22T11:25:44Z level=debug event=want client=10 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05164worker2 # [ 68.360886] nix-grpc-daemon[782]: ts=2026-09-22T11:25:44Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15165worker1 # [ 68.367735] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325166worker1 # [ 68.379892] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325167worker1 # [ 68.391116] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325168worker1 # [ 68.401652] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5169worker1 # [ 68.502648] nix-daemon[847]: accepted connection from pid 997, user nix-grpc-daemon (trusted)5170worker1 # [ 68.515416] nix-daemon[847]: reaped child process 1000, status = succeeded5171worker1 # [ 68.599922] nix-daemon[847]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)5172worker1 # [ 68.613185] nix-daemon[847]: reaped child process 1005, status = succeeded5173lb # [ 68.632589] niks3-server[969]: 2026/09/22 11:25:44 INFO Received uploads request method=POST path=/api/pending_closures5174worker1 # [ 68.629112] nix-grpc-daemon[875]: time=2026-09-22T11:25:43.839Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5175worker1 # [ 68.631069] nix-grpc-daemon[875]: time=2026-09-22T11:25:43.841Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5176lb # [ 68.660945] niks3-server[969]: 2026/09/22 11:25:44 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5177lb # [ 68.675292] niks3-server[969]: 2026/09/22 11:25:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5178lb # [ 68.681775] niks3-server[969]: 2026/09/22 11:25:44 INFO Signed narinfos id=15 count=15179worker1 # [ 68.675868] nix-grpc-daemon[875]: time=2026-09-22T11:25:43.886Z level=INFO msg="Uploading 1 narinfos"5180lb # [ 68.696577] niks3-server[969]: 2026/09/22 11:25:44 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5181lb # [ 68.702091] niks3-server[969]: 2026/09/22 11:25:44 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5182lb # [ 68.708202] niks3-server[969]: 2026/09/22 11:25:44 INFO Completed upload id=155183worker1 # [ 68.697990] nix-grpc-daemon[875]: time=2026-09-22T11:25:43.908Z level=INFO msg="Upload complete. (285ms)"5184lb # [ 68.711089] niks3-server[969]: 2026/09/22 11:25:44 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5185worker1 # [ 68.702122] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=5125186worker1 # [ 68.707271] nix-grpc-daemon[781]: ts=2026-09-22T11:25:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325187worker1 # [ 68.716152] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5188worker1 # [ 68.725124] nix-daemon[847]: reaped child process 995, status = succeeded5189worker1 # [ 68.729264] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5190worker1 # [ 69.577427] nix-daemon[847]: reaped child process 1009, status = succeeded5191worker1 # [ 69.637091] nix-daemon[847]: accepted connection from pid 1025, user nix-grpc-daemon (trusted)5192worker1 # [ 69.649052] nix-daemon[847]: reaped child process 1028, status = succeeded5193lb # [ 69.664978] niks3-server[969]: 2026/09/22 11:25:45 INFO Received uploads request method=POST path=/api/pending_closures5194worker1 # [ 69.658751] nix-grpc-daemon[875]: time=2026-09-22T11:25:44.868Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5195worker1 # [ 69.660350] nix-grpc-daemon[875]: time=2026-09-22T11:25:44.870Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5196lb # [ 69.682583] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5197lb # [ 69.686280] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5198lb # [ 69.689715] niks3-server[969]: 2026/09/22 11:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5199lb # [ 69.691429] niks3-server[969]: 2026/09/22 11:25:45 INFO Signed narinfos id=16 count=15200worker1 # [ 69.681378] nix-grpc-daemon[875]: time=2026-09-22T11:25:44.891Z level=INFO msg="Uploading 1 narinfos"5201lb # [ 69.697907] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5202lb # [ 69.699298] niks3-server[969]: 2026/09/22 11:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5203lb # [ 69.702481] niks3-server[969]: 2026/09/22 11:25:45 INFO Completed upload id=165204worker1 # [ 69.692250] nix-grpc-daemon[875]: time=2026-09-22T11:25:44.902Z level=INFO msg="Upload complete. (156ms)"5205lb # [ 69.704928] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5206worker1 # [ 69.694561] nix-grpc-daemon[781]: ts=2026-09-22T11:25:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=9 outputs=15207worker1 # [ 69.701051] nix-daemon[847]: reaped child process 1007, status = succeeded5208worker1 # [ 69.740415] nix-grpc-daemon[781]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325209worker1 # [ 69.752104] nix-grpc-daemon[781]: ts=2026-09-22T11:25:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:462325210worker1 # [ 69.756439] nix-grpc-daemon[781]: ts=2026-09-22T11:25:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_out=1285211worker1 # [ 69.956604] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:462325212worker2 # [ 69.959221] nix-grpc-daemon[782]: ts=2026-09-22T11:25:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325213worker2 # [ 69.976263] nix-grpc-daemon[782]: ts=2026-09-22T11:25:46Z level=debug event=want client=11 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05214worker2 # [ 69.981149] nix-grpc-daemon[782]: ts=2026-09-22T11:25:46Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15215worker1 # [ 69.984706] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325216worker1 # [ 69.994549] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325217worker1 # [ 70.004405] nix-grpc-daemon[875]: time=2026-09-22T11:25:45.214Z level=INFO msg="All 1 paths already cached"5218worker1 # [ 70.010202] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325219worker1 # [ 70.020163] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5220worker1 # [ 70.117258] nix-daemon[847]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)5221worker1 # [ 70.130541] nix-daemon[847]: reaped child process 1035, status = succeeded5222worker1 # [ 70.208911] nix-daemon[847]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5223worker1 # [ 70.222531] nix-daemon[847]: reaped child process 1040, status = succeeded5224lb # [ 70.237395] niks3-server[969]: 2026/09/22 11:25:45 INFO Received uploads request method=POST path=/api/pending_closures5225worker1 # [ 70.230948] nix-grpc-daemon[875]: time=2026-09-22T11:25:45.441Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5226worker1 # [ 70.232797] nix-grpc-daemon[875]: time=2026-09-22T11:25:45.442Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5227lb # [ 70.251809] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5228lb # [ 70.254487] niks3-server[969]: 2026/09/22 11:25:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5229lb # [ 70.256590] niks3-server[969]: 2026/09/22 11:25:45 INFO Signed narinfos id=17 count=15230worker1 # [ 70.246637] nix-grpc-daemon[875]: time=2026-09-22T11:25:45.456Z level=INFO msg="Uploading 1 narinfos"5231lb # [ 70.261481] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5232lb # [ 70.264576] niks3-server[969]: 2026/09/22 11:25:45 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5233lb # [ 70.269322] niks3-server[969]: 2026/09/22 11:25:45 INFO Completed upload id=175234worker1 # [ 70.258792] nix-grpc-daemon[875]: time=2026-09-22T11:25:45.469Z level=INFO msg="Upload complete. (226ms)"5235worker1 # [ 70.260781] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=6245236lb # [ 70.271670] niks3-server[969]: 2026/09/22 11:25:45 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5237worker1 # [ 70.263773] nix-grpc-daemon[781]: ts=2026-09-22T11:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325238worker1 # [ 70.270628] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5239worker1 # [ 70.272861] nix-daemon[847]: reaped child process 1030, status = succeeded5240worker1 # [ 70.277909] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5241worker1 # [ 71.493242] nix-daemon[847]: reaped child process 1044, status = succeeded5242worker1 # [ 71.565095] nix-daemon[847]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)5243worker1 # [ 71.577402] nix-daemon[847]: reaped child process 1063, status = succeeded5244lb # [ 71.594576] niks3-server[969]: 2026/09/22 11:25:47 INFO Received uploads request method=POST path=/api/pending_closures5245worker1 # [ 71.592731] nix-grpc-daemon[875]: time=2026-09-22T11:25:46.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5246worker1 # [ 71.595070] nix-grpc-daemon[875]: time=2026-09-22T11:25:46.804Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5247lb # [ 71.630083] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5248lb # [ 71.643613] niks3-server[969]: 2026/09/22 11:25:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5249lb # [ 71.647781] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5250lb # [ 71.651794] niks3-server[969]: 2026/09/22 11:25:47 INFO Signed narinfos id=18 count=15251worker1 # [ 71.641756] nix-grpc-daemon[875]: time=2026-09-22T11:25:46.851Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 71.655708] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5253lb # [ 71.660240] niks3-server[969]: 2026/09/22 11:25:47 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5254lb # [ 71.664202] niks3-server[969]: 2026/09/22 11:25:47 INFO Completed upload id=185255worker1 # [ 71.654132] nix-grpc-daemon[875]: time=2026-09-22T11:25:46.864Z level=INFO msg="Upload complete. (168ms)"5256lb # [ 71.666393] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5257worker1 # [ 71.658921] nix-grpc-daemon[781]: ts=2026-09-22T11:25:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=10 outputs=15258worker1 # [ 71.665278] nix-grpc-daemon[781]: ts=2026-09-22T11:25:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325259worker1 # [ 71.673849] nix-daemon[847]: reaped child process 1042, status = succeeded5260worker2 # [ 71.674928] nix-grpc-daemon[782]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:529325261worker2 # [ 71.684720] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5262worker2 # [ 71.696165] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5263worker2 # [ 71.707978] nix-grpc-daemon[782]: ts=2026-09-22T11:25:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_out=1765264worker2 # [ 71.913171] nix-grpc-daemon[782]: ts=2026-09-22T11:25:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:529325265worker1 # [ 71.917135] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325266worker1 # [ 71.925055] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.135Z level=INFO msg="All 2 paths already cached"5267worker2 # [ 71.936280] nix-grpc-daemon[782]: ts=2026-09-22T11:25:48Z level=debug event=want client=12 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05268worker1 # [ 71.939728] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325269worker2 # [ 71.940552] nix-grpc-daemon[782]: ts=2026-09-22T11:25:48Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15270worker1 # [ 71.949407] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325271worker1 # [ 71.958909] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.169Z level=INFO msg="All 2 paths already cached"5272worker1 # [ 71.964815] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325273worker1 # [ 71.972824] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5274worker1 # [ 72.070367] nix-daemon[847]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)5275worker1 # [ 72.084515] nix-daemon[847]: reaped child process 1070, status = succeeded5276worker1 # [ 72.161261] nix-daemon[847]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5277worker1 # [ 72.174884] nix-daemon[847]: reaped child process 1075, status = succeeded5278lb # [ 72.191792] niks3-server[969]: 2026/09/22 11:25:47 INFO Received uploads request method=POST path=/api/pending_closures5279worker1 # [ 72.189640] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.399Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5280worker1 # [ 72.191082] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.399Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5281lb # [ 72.221975] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5282lb # [ 72.229825] niks3-server[969]: 2026/09/22 11:25:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5283worker1 # [ 72.225880] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.436Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 72.236434] niks3-server[969]: 2026/09/22 11:25:47 INFO Signed narinfos id=19 count=15285lb # [ 72.242937] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5286lb # [ 72.256321] niks3-server[969]: 2026/09/22 11:25:47 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5287worker1 # [ 72.252667] nix-grpc-daemon[875]: time=2026-09-22T11:25:47.463Z level=INFO msg="Upload complete. (265ms)"5288lb # [ 72.263106] niks3-server[969]: 2026/09/22 11:25:47 INFO Completed upload id=195289worker1 # [ 72.255579] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=7525290lb # [ 72.267605] niks3-server[969]: 2026/09/22 11:25:47 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5291worker1 # [ 72.262407] nix-grpc-daemon[781]: ts=2026-09-22T11:25:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325292worker1 # [ 72.272740] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5293worker1 # [ 72.275931] nix-daemon[847]: reaped child process 1065, status = succeeded5294worker1 # [ 72.281158] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5295worker1 # [ 72.896286] nix-daemon[847]: reaped child process 1079, status = succeeded5296worker1 # [ 72.974407] nix-daemon[847]: accepted connection from pid 1095, user nix-grpc-daemon (trusted)5297worker1 # [ 72.985670] nix-daemon[847]: reaped child process 1098, status = succeeded5298lb # [ 73.005456] niks3-server[969]: 2026/09/22 11:25:48 INFO Received uploads request method=POST path=/api/pending_closures5299worker1 # [ 73.003732] nix-grpc-daemon[875]: time=2026-09-22T11:25:48.213Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5300worker1 # [ 73.005264] nix-grpc-daemon[875]: time=2026-09-22T11:25:48.215Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5301lb # [ 73.047862] niks3-server[969]: 2026/09/22 11:25:48 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5302lb # [ 73.055330] niks3-server[969]: 2026/09/22 11:25:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5303lb # [ 73.063149] niks3-server[969]: 2026/09/22 11:25:48 INFO Signed narinfos id=20 count=15304worker1 # [ 73.056321] nix-grpc-daemon[875]: time=2026-09-22T11:25:48.266Z level=INFO msg="Uploading 1 narinfos"5305lb # [ 73.069415] niks3-server[969]: 2026/09/22 11:25:48 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5306lb # [ 73.074187] niks3-server[969]: 2026/09/22 11:25:48 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5307lb # [ 73.085143] niks3-server[969]: 2026/09/22 11:25:48 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5308lb # [ 73.087774] niks3-server[969]: 2026/09/22 11:25:48 INFO Completed upload id=205309worker1 # [ 73.077703] nix-grpc-daemon[875]: time=2026-09-22T11:25:48.288Z level=INFO msg="Upload complete. (190ms)"5310lb # [ 73.090290] niks3-server[969]: 2026/09/22 11:25:48 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5311worker1 # [ 73.082809] nix-grpc-daemon[781]: ts=2026-09-22T11:25:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=11 outputs=15312worker1 # [ 73.090302] nix-grpc-daemon[781]: ts=2026-09-22T11:25:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325313worker1 # [ 73.100514] nix-daemon[847]: reaped child process 1077, status = succeeded5314worker2 # [ 73.100759] nix-grpc-daemon[782]: ts=2026-09-22T11:25:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:529325315worker2 # [ 73.109841] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5316worker2 # [ 73.121611] nix-grpc-daemon[782]: ts=2026-09-22T11:25:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_out=2325317client # [ 73.199222] nix-daemon[1044]: reaped child process 1046, status = succeeded5318client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.12 seconds)5319client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5320client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5321client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5322client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5323client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5324client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5325(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.17 seconds)5326subtest: an input only one worker has still reaches the builder5327client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5328client # this derivation will be built:5329client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5330client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5331client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.22 seconds)5332client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5333client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5334worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5335worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5336worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5337worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5338client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25339client # [ 73.847732] nix-daemon[1044]: accepted connection from pid 1126, user root (trusted)5340client # this derivation will be built:5341client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5342worker1 # [ 73.956493] nix-grpc-daemon[781]: ts=2026-09-22T11:25:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:462325343client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5344worker2 # [ 73.961700] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325345client # copying 1 paths...5346client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5347worker1 # [ 73.972637] nix-grpc-daemon[781]: ts=2026-09-22T11:25:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325348worker1 # [ 73.983414] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5349worker1 # [ 74.088495] nix-daemon[847]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5350worker1 # [ 74.102413] nix-daemon[847]: reaped child process 1110, status = succeeded5351lb # [ 74.119658] niks3-server[969]: 2026/09/22 11:25:49 INFO Received uploads request method=POST path=/api/pending_closures5352worker1 # [ 74.117200] nix-grpc-daemon[875]: time=2026-09-22T11:25:49.327Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5353worker1 # [ 74.119093] nix-grpc-daemon[875]: time=2026-09-22T11:25:49.329Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5354lb # [ 74.143364] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5355lb # [ 74.156672] niks3-server[969]: 2026/09/22 11:25:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5356lb # [ 74.161650] niks3-server[969]: 2026/09/22 11:25:49 INFO Signed narinfos id=21 count=15357worker1 # [ 74.154441] nix-grpc-daemon[875]: time=2026-09-22T11:25:49.364Z level=INFO msg="Uploading 1 narinfos"5358lb # [ 74.172289] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5359lb # [ 74.183932] niks3-server[969]: 2026/09/22 11:25:49 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5360worker1 # [ 74.179967] nix-grpc-daemon[875]: time=2026-09-22T11:25:49.390Z level=INFO msg="Upload complete. (186ms)"5361lb # [ 74.190485] niks3-server[969]: 2026/09/22 11:25:49 INFO Completed upload id=215362lb # [ 74.193148] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5363worker1 # [ 74.183986] nix-grpc-daemon[781]: ts=2026-09-22T11:25:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_in=1205364worker2 # [ 74.193447] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=want client=13 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05365worker1 # [ 74.196701] nix-daemon[847]: reaped child process 1105, status = succeeded5366worker2 # [ 74.196324] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15367worker2 # [ 74.198183] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325368worker2 # [ 74.203835] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325369client # copying 2 paths to 'lb:50051'...5370worker2 # [ 74.212309] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529325371client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5372client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5373worker2 # [ 74.221748] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5374worker2 # [ 74.315321] nix-daemon[868]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5375worker2 # [ 74.332473] nix-daemon[868]: reaped child process 1210, status = succeeded5376worker2 # [ 74.417608] nix-daemon[868]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5377worker2 # [ 74.435457] nix-daemon[868]: reaped child process 1215, status = succeeded5378lb # [ 74.452257] niks3-server[969]: 2026/09/22 11:25:49 INFO Received uploads request method=POST path=/api/pending_closures5379lb # [ 74.458969] niks3-server[969]: 2026/09/22 11:25:49 INFO Received uploads request method=POST path=/api/pending_closures5380worker2 # [ 74.451032] nix-grpc-daemon[883]: time=2026-09-22T11:25:50.627Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5381worker2 # [ 74.453066] nix-grpc-daemon[883]: time=2026-09-22T11:25:50.629Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5382worker2 # [ 74.454650] nix-grpc-daemon[883]: time=2026-09-22T11:25:50.629Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5383lb # [ 74.484146] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5384lb # [ 74.487162] niks3-server[969]: 2026/09/22 11:25:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5385lb # [ 74.489177] niks3-server[969]: 2026/09/22 11:25:49 INFO Signed narinfos id=22 count=15386lb # [ 74.490174] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5387lb # [ 74.494072] niks3-server[969]: 2026/09/22 11:25:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5388lb # [ 74.496205] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5389lb # [ 74.499101] niks3-server[969]: 2026/09/22 11:25:49 INFO Signed narinfos id=23 count=25390worker2 # [ 74.488826] nix-grpc-daemon[883]: time=2026-09-22T11:25:50.665Z level=INFO msg="Uploading 3 narinfos"5391lb # [ 74.501094] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5392lb # [ 74.512943] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5393lb # [ 74.517071] niks3-server[969]: 2026/09/22 11:25:49 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5394lb # [ 74.520982] niks3-server[969]: 2026/09/22 11:25:49 INFO Completed upload id=225395lb # [ 74.523222] niks3-server[969]: 2026/09/22 11:25:49 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5396lb # [ 74.525324] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5397lb # [ 74.527676] niks3-server[969]: 2026/09/22 11:25:49 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5398lb # [ 74.530322] niks3-server[969]: 2026/09/22 11:25:49 INFO Completed upload id=235399worker2 # [ 74.518178] nix-grpc-daemon[883]: time=2026-09-22T11:25:50.695Z level=INFO msg="Upload complete. (289ms)"5400worker2 # [ 74.520522] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=2 nar_bytes_in=9765401worker2 # [ 74.524177] nix-grpc-daemon[782]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325402worker2 # [ 74.529490] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5403worker2 # [ 74.534380] nix-daemon[868]: reaped child process 1205, status = succeeded5404worker2 # [ 74.541258] nix-grpc-daemon[782]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5405client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5406worker2 # [ 74.548045] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5407client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5408lb # [ 74.689196] envoy[896]: conn peer=[2001:db8:1::1]:58902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=797 ms=41695409lb # [ 74.690762] envoy[896]: conn peer=[2001:db8:1::1]:58918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1085 ms=47495410lb # [ 74.692172] envoy[896]: conn peer=[2001:db8:1::1]:56344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=545411lb # [ 74.693521] envoy[896]: conn peer=[2001:db8:1::1]:56328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3379 tx=1718 ms=14595412lb # [ 74.694960] envoy[896]: conn peer=[2001:db8:1::1]:56356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3820 tx=1895 ms=18025413lb # [ 74.696337] envoy[896]: conn peer=[2001:db8:1::1]:56364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=845414lb # [ 74.697662] envoy[896]: conn peer=[2001:db8:1::1]:56376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4296 tx=2145 ms=12175415lb # [ 74.699046] envoy[896]: conn peer=[2001:db8:1::1]:50228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=335416client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5417worker2 # [ 75.164780] nix-daemon[868]: reaped child process 1220, status = succeeded5418worker2 # [ 75.240654] nix-daemon[868]: accepted connection from pid 1236, user nix-grpc-daemon (trusted)5419worker2 # [ 75.255513] nix-daemon[868]: reaped child process 1239, status = succeeded5420lb # [ 75.273523] niks3-server[969]: 2026/09/22 11:25:50 INFO Received uploads request method=POST path=/api/pending_closures5421worker2 # [ 75.269172] nix-grpc-daemon[883]: time=2026-09-22T11:25:51.446Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5422lb # [ 75.306138] niks3-server[969]: 2026/09/22 11:25:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5423lb # [ 75.313504] niks3-server[969]: 2026/09/22 11:25:50 INFO Signed narinfos id=24 count=15424worker2 # [ 75.305280] nix-grpc-daemon[883]: time=2026-09-22T11:25:51.482Z level=INFO msg="Uploading 1 narinfos"5425lb # [ 75.319363] niks3-server[969]: 2026/09/22 11:25:50 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5426lb # [ 75.325324] niks3-server[969]: 2026/09/22 11:25:50 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5427lb # [ 75.337115] niks3-server[969]: 2026/09/22 11:25:50 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5428worker2 # [ 75.331339] nix-grpc-daemon[883]: time=2026-09-22T11:25:51.508Z level=INFO msg="Upload complete. (177ms)"5429lb # [ 75.343667] niks3-server[969]: 2026/09/22 11:25:50 INFO Completed upload id=245430lb # [ 75.348839] niks3-server[969]: 2026/09/22 11:25:50 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5431worker2 # [ 75.335469] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=12 outputs=15432worker2 # [ 75.348127] nix-daemon[868]: reaped child process 1217, status = succeeded5433client # copying 1 paths...5434worker2 # [ 75.380995] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:529325435client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5436worker2 # [ 75.403133] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:529325437worker2 # [ 75.407370] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_out=1205438client # [ 75.522389] nix-daemon[1044]: reaped child process 1129, status = succeeded5439client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.80 seconds)5440client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25441client # [ 75.624241] nix-daemon[1044]: accepted connection from pid 1148, user root (trusted)5442client # this derivation will be built:5443client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5444worker1 # [ 75.731898] nix-grpc-daemon[781]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:462325445client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5446worker2 # [ 75.732934] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325447worker2 # [ 75.742880] nix-grpc-daemon[883]: time=2026-09-22T11:25:51.919Z level=INFO msg="All 1 paths already cached"5448worker2 # [ 75.756416] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=debug event=want client=14 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05449worker1 # [ 75.764982] nix-grpc-daemon[781]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325450worker2 # [ 75.761213] nix-grpc-daemon[782]: ts=2026-09-22T11:25:51Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15451worker1 # [ 75.772671] nix-grpc-daemon[781]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325452client # copying 1 paths to 'lb:50051'...5453client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5454worker1 # [ 75.786343] nix-grpc-daemon[781]: ts=2026-09-22T11:25:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:462325455worker1 # [ 75.790372] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5456worker1 # [ 75.823067] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)5457worker1 # [ 75.876266] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)5458worker1 # [ 75.927804] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 3/5)5459worker1 # [ 76.011609] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 494 ms (attempt 4/5)5460worker1 # [ 76.508760] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5461worker1 # [ 76.512137] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5462worker1 # [ 76.517696] nix-grpc-daemon[781]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5463worker1 # [ 76.619810] nix-daemon[847]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5464worker1 # [ 76.633794] nix-daemon[847]: reaped child process 1122, status = succeeded5465worker1 # [ 76.718948] nix-daemon[847]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)5466worker1 # [ 76.734467] nix-daemon[847]: reaped child process 1127, status = succeeded5467lb # [ 76.752140] niks3-server[969]: 2026/09/22 11:25:52 INFO Received uploads request method=POST path=/api/pending_closures5468worker1 # [ 76.749986] nix-grpc-daemon[875]: time=2026-09-22T11:25:51.959Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5469worker1 # [ 76.751390] nix-grpc-daemon[875]: time=2026-09-22T11:25:51.959Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5470lb # [ 76.781652] niks3-server[969]: 2026/09/22 11:25:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5471lb # [ 76.788963] niks3-server[969]: 2026/09/22 11:25:52 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5472lb # [ 76.795123] niks3-server[969]: 2026/09/22 11:25:52 INFO Signed narinfos id=25 count=15473worker1 # [ 76.788377] nix-grpc-daemon[875]: time=2026-09-22T11:25:51.998Z level=INFO msg="Uploading 1 narinfos"5474lb # [ 76.802346] niks3-server[969]: 2026/09/22 11:25:52 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5475lb # [ 76.814419] niks3-server[969]: 2026/09/22 11:25:52 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5476worker1 # [ 76.812893] nix-grpc-daemon[875]: time=2026-09-22T11:25:52.023Z level=INFO msg="Upload complete. (277ms)"5477lb # [ 76.823312] niks3-server[969]: 2026/09/22 11:25:52 INFO Completed upload id=255478worker1 # [ 76.815552] nix-grpc-daemon[781]: ts=2026-09-22T11:25:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=1 paths=1 nar_bytes_in=5605479lb # [ 76.829419] niks3-server[969]: 2026/09/22 11:25:52 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5480worker1 # [ 76.821342] nix-grpc-daemon[781]: ts=2026-09-22T11:25:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:462325481worker1 # [ 76.831521] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5482worker1 # [ 76.840929] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5483client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5484worker1 # [ 76.842945] nix-daemon[847]: reaped child process 1112, status = succeeded5485client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5486client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5487worker1 # [ 78.262174] nix-daemon[847]: reaped child process 1131, status = succeeded5488worker1 # [ 78.329482] nix-daemon[847]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5489worker1 # [ 78.342839] nix-daemon[847]: reaped child process 1150, status = succeeded5490lb # [ 78.357427] niks3-server[969]: 2026/09/22 11:25:53 INFO Received uploads request method=POST path=/api/pending_closures5491worker1 # [ 78.354853] nix-grpc-daemon[875]: time=2026-09-22T11:25:53.565Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5492lb # [ 78.388685] niks3-server[969]: 2026/09/22 11:25:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5493worker1 # [ 78.385808] nix-grpc-daemon[875]: time=2026-09-22T11:25:53.596Z level=INFO msg="Uploading 1 narinfos"5494lb # [ 78.396630] niks3-server[969]: 2026/09/22 11:25:53 INFO Signed narinfos id=26 count=15495lb # [ 78.404572] niks3-server[969]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5496lb # [ 78.411989] niks3-server[969]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5497lb # [ 78.423890] niks3-server[969]: 2026/09/22 11:25:53 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5498lb # [ 78.427845] niks3-server[969]: 2026/09/22 11:25:53 INFO Completed upload id=265499worker1 # [ 78.418815] nix-grpc-daemon[875]: time=2026-09-22T11:25:53.629Z level=INFO msg="Upload complete. (169ms)"5500lb # [ 78.429764] niks3-server[969]: 2026/09/22 11:25:53 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5501worker1 # [ 78.421405] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=13 outputs=15502worker1 # [ 78.428391] nix-daemon[847]: reaped child process 1129, status = succeeded5503client # copying 1 paths...5504worker1 # [ 78.466074] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325505client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5506worker1 # [ 78.477828] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:462325507worker1 # [ 78.479883] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_out=1205508client # [ 78.577215] nix-daemon[1044]: reaped child process 1151, status = succeeded5509client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.05 seconds)5510client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25511client # [ 78.664205] nix-daemon[1044]: accepted connection from pid 1170, user root (trusted)5512client # this derivation will be built:5513client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5514worker1 # [ 78.770560] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:462325515client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5516worker1 # [ 78.776409] nix-grpc-daemon[781]: ts=2026-09-22T11:25:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:462325517worker1 # [ 78.781792] nix-grpc-daemon[875]: time=2026-09-22T11:25:53.990Z level=INFO msg="All 1 paths already cached"5518worker2 # [ 78.792306] nix-grpc-daemon[782]: ts=2026-09-22T11:25:54Z level=debug event=want client=15 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05519worker2 # [ 78.798996] nix-grpc-daemon[782]: ts=2026-09-22T11:25:54Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15520worker2 # [ 78.804585] nix-grpc-daemon[782]: ts=2026-09-22T11:25:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325521worker2 # [ 78.811500] nix-grpc-daemon[782]: ts=2026-09-22T11:25:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529325522worker2 # [ 78.825152] nix-grpc-daemon[883]: time=2026-09-22T11:25:55.001Z level=INFO msg="All 1 paths already cached"5523client # copying 1 paths to 'lb:50051'...5524client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5525worker2 # [ 78.831678] nix-grpc-daemon[782]: ts=2026-09-22T11:25:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529325526worker2 # [ 78.842265] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5527worker2 # [ 78.945756] nix-daemon[868]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)5528worker2 # [ 78.961902] nix-daemon[868]: reaped child process 1250, status = succeeded5529worker2 # [ 79.047806] nix-daemon[868]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5530worker2 # [ 79.061825] nix-daemon[868]: reaped child process 1255, status = succeeded5531lb # [ 79.081541] niks3-server[969]: 2026/09/22 11:25:54 INFO Received uploads request method=POST path=/api/pending_closures5532worker2 # [ 79.077872] nix-grpc-daemon[883]: time=2026-09-22T11:25:55.254Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5533worker2 # [ 79.079444] nix-grpc-daemon[883]: time=2026-09-22T11:25:55.256Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5534lb # [ 79.108855] niks3-server[969]: 2026/09/22 11:25:54 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5535lb # [ 79.122622] niks3-server[969]: 2026/09/22 11:25:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5536lb # [ 79.126879] niks3-server[969]: 2026/09/22 11:25:54 INFO Signed narinfos id=27 count=15537worker2 # [ 79.117211] nix-grpc-daemon[883]: time=2026-09-22T11:25:55.294Z level=INFO msg="Uploading 1 narinfos"5538lb # [ 79.132103] niks3-server[969]: 2026/09/22 11:25:54 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5539lb # [ 79.142192] niks3-server[969]: 2026/09/22 11:25:54 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5540lb # [ 79.146067] niks3-server[969]: 2026/09/22 11:25:54 INFO Completed upload id=275541lb # [ 79.147413] niks3-server[969]: 2026/09/22 11:25:54 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5542worker2 # [ 79.135135] nix-grpc-daemon[883]: time=2026-09-22T11:25:55.312Z level=INFO msg="Upload complete. (281ms)"5543worker2 # [ 79.138665] nix-grpc-daemon[782]: ts=2026-09-22T11:25:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 paths=1 nar_bytes_in=5605544worker2 # [ 79.143406] nix-grpc-daemon[782]: ts=2026-09-22T11:25:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529325545worker2 # [ 79.151565] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5546worker2 # [ 79.158094] nix-daemon[868]: reaped child process 1245, status = succeeded5547worker2 # [ 79.161470] nix-daemon[868]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5548client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5549client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5550client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5551worker2 # [ 79.993573] nix-daemon[868]: reaped child process 1259, status = succeeded5552worker2 # [ 80.070659] nix-daemon[868]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5553worker2 # [ 80.081977] nix-daemon[868]: reaped child process 1278, status = succeeded5554lb # [ 80.103132] niks3-server[969]: 2026/09/22 11:25:55 INFO Received uploads request method=POST path=/api/pending_closures5555worker2 # [ 80.095772] nix-grpc-daemon[883]: time=2026-09-22T11:25:56.272Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5556lb # [ 80.117807] niks3-server[969]: 2026/09/22 11:25:55 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5557lb # [ 80.121884] niks3-server[969]: 2026/09/22 11:25:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5558lb # [ 80.123962] niks3-server[969]: 2026/09/22 11:25:55 INFO Signed narinfos id=28 count=15559worker2 # [ 80.111729] nix-grpc-daemon[883]: time=2026-09-22T11:25:56.288Z level=INFO msg="Uploading 1 narinfos"5560lb # [ 80.128988] niks3-server[969]: 2026/09/22 11:25:55 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5561lb # [ 80.132625] niks3-server[969]: 2026/09/22 11:25:55 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5562lb # [ 80.135262] niks3-server[969]: 2026/09/22 11:25:55 INFO Completed upload id=285563worker2 # [ 80.123124] nix-grpc-daemon[883]: time=2026-09-22T11:25:56.300Z level=INFO msg="Upload complete. (154ms)"5564lb # [ 80.139452] niks3-server[969]: 2026/09/22 11:25:55 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5565worker2 # [ 80.125553] nix-grpc-daemon[782]: ts=2026-09-22T11:25:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52932 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=14 outputs=15566client # copying 1 paths...5567worker2 # [ 80.133035] nix-daemon[868]: reaped child process 1257, status = succeeded5568worker1 # [ 80.134551] nix-grpc-daemon[781]: ts=2026-09-22T11:25:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325569worker1 # [ 80.143238] nix-grpc-daemon[781]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5570client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5571worker1 # [ 80.162952] nix-grpc-daemon[781]: ts=2026-09-22T11:25:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:462325572worker1 # [ 80.167942] nix-grpc-daemon[781]: ts=2026-09-22T11:25:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46232 duration_s=0 paths=1 nar_bytes_out=1205573client # [ 80.280699] nix-daemon[1044]: reaped child process 1173, status = succeeded5574client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.71 seconds)5575(finished: subtest: an input only one worker has still reaches the builder, in 7.02 seconds)5576subtest: upload whose reference the worker lacks is completed from the cache5577client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5578client # this derivation will be built:5579client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5580client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5581client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.23 seconds)5582client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5583client # this derivation will be built:5584client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5585client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5586client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.22 seconds)5587client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5588client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5589worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5590worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5591worker2 # [ 80.969956] systemd[1]: Stopping Nix worker-protocol over gRPC...5592worker2 # [ 80.990712] nix-grpc-daemon[782]: ts=2026-09-22T11:25:57Z level=info event=signal_exit5593lb # [ 81.193449] niks3-server[969]: 2026/09/22 11:25:56 INFO lead: released remote=192.168.1.4:571985594worker1 # [ 82.182292] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_restarting addr=lb:500515595worker2 # [ 82.182565] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=scheduler_restarting5596worker2 # [ 82.183866] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529245597worker2 # [ 82.185728] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=worker_gone addr=192.168.1.3:500515598worker2 # [ 82.187866] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=scheduler_restarting addr=lb:500515599worker2 # [ 82.189220] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529245600worker2 # [ 82.190773] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=worker_gone addr=192.168.1.4:500515601worker2 # [ 82.206967] nix-daemon[868]: reaped child process 870, status = succeeded5602worker1 # [ 82.382531] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_connected addr=lb:500515603worker1 # [ 82.384461] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5604worker2 # [ 82.389320] nix-grpc-daemon[782]: ts=2026-09-22T11:25:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5605worker1 # [ 82.585234] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_connected addr=lb:500515606worker1 # [ 82.587245] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5607worker1 # [ 82.787944] nix-grpc-daemon[781]: ts=2026-09-22T11:25:57Z level=info event=scheduler_connected addr=lb:500515608worker1 # [ 82.790158] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5609worker1 # [ 82.991062] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_connected addr=lb:500515610worker1 # [ 82.995089] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5611worker2 # [ 83.097532] nix-grpc-daemon[782]: E0000 00:00:1790076359.274190 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5612worker2 # [ 83.102799] nix-grpc-daemon[782]: E0000 00:00:1790076359.274428 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5613worker1 # [ 83.196287] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_connected addr=lb:500515614worker1 # [ 83.198130] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5615worker1 # [ 83.398813] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_connected addr=lb:500515616worker1 # [ 83.400680] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5617worker1 # [ 83.601329] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_connected addr=lb:500515618worker1 # [ 83.603257] nix-grpc-daemon[781]: ts=2026-09-22T11:25:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5619worker1 # [ 83.804474] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_connected addr=lb:500515620worker1 # [ 83.809252] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5621worker1 # [ 84.009379] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_connected addr=lb:500515622worker1 # [ 84.011297] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5623worker1 # [ 84.212894] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_connected addr=lb:500515624worker1 # [ 84.214423] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5625worker2 # [ 84.262926] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5626worker2 # [ 84.264633] systemd[1]: Stopped Nix worker-protocol over gRPC.5627worker2 # [ 84.266072] systemd[1]: nix-grpc-daemon.service: Consumed 2.583s CPU time over 1min 9.420s wall clock time, 70.2M memory peak, 99.4K incoming IP traffic, 268.8K outgoing IP traffic.5628worker2 # [ 84.271355] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5629worker2 # [ 84.273142] systemd[1]: Closed Nix worker-protocol over gRPC.5630worker2 # [ 84.274340] systemd[1]: nix-grpc-daemon.socket: Consumed 321us CPU time over 1min 16.675s wall clock time, 256K memory peak, 97.5K incoming IP traffic, 75.8K outgoing IP traffic.5631lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5632lb: must succeed: curl -sf localhost:9901/clusters5633lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5634lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5635client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25636worker1 # [ 84.414125] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_connected addr=lb:500515637worker1 # [ 84.419916] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:462325638worker1 # [ 84.424558] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5639worker1 # [ 84.428272] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:462325640client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5641client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.14 seconds)5642worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5643worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.01 seconds)5644worker2 # [ 84.481831] systemd[1]: Listening on Nix worker-protocol over gRPC.5645worker1 # [ 84.509577] systemd[1]: Stopping Nix worker-protocol over gRPC...5646worker1 # [ 84.619817] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_connected addr=lb:500515647worker1 # [ 84.621787] nix-grpc-daemon[781]: ts=2026-09-22T11:25:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5648lb # [ 84.689804] envoy[896]: conn peer=[2001:db8:1::1]:50236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4196 tx=1782 ms=15045649lb # [ 84.691606] envoy[896]: conn peer=[2001:db8:1::1]:50248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=595650lb # [ 84.693372] envoy[896]: conn peer=[2001:db8:1::1]:50252 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=1899 ms=27985651lb # [ 84.694864] envoy[896]: conn peer=[2001:db8:1::1]:50256 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515652lb # [ 84.696247] envoy[896]: conn peer=[2001:db8:1::1]:50272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545653lb # [ 84.697892] envoy[896]: conn peer=[2001:db8:1::1]:50258 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3729 tx=1990 ms=14525654lb # [ 84.699348] envoy[896]: conn peer=[2001:db8:1::4]:42436 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=67892 tx=3508 ms=673575655lb # [ 84.700983] envoy[896]: conn peer=[2001:db8:1::1]:48834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=225656worker1 # [ 84.822559] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_connected addr=lb:500515657worker1 # [ 84.824607] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5658worker1 # [ 84.829767] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=signal_exit5659worker1 # [ 85.025163] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_connected addr=lb:500515660worker1 # [ 85.027199] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5661lb # [ 85.189117] niks3-server[969]: 2026/09/22 11:26:00 INFO lead: acquired remote=192.168.1.3:361525662worker1 # [ 85.181440] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_take_over5663lb # [ 85.194104] niks3-server[969]: 2026/09/22 11:26:00 INFO lead: released remote=192.168.1.3:361525664worker1 # [ 85.228398] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_connected addr=lb:500515665worker1 # [ 85.230046] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5666worker2 # [ 85.260090] systemd[1]: Starting scheduler-token.service...5667worker2 # [ 85.293350] systemd[1]: scheduler-token.service: Deactivated successfully.5668worker2 # [ 85.295089] systemd[1]: Finished scheduler-token.service.5669worker2 # [ 85.296781] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 34ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5670worker2 # [ 85.302169] systemd[1]: Starting Nix worker-protocol over gRPC...5671worker2 # [ 85.393157] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5672worker2 # [ 85.395483] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5673worker2 # [ 85.399202] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15674worker2 # [ 85.404057] nix-grpc-daemon[1302]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5675worker2 # [ 85.406176] nix-grpc-daemon[1302]: E0000 00:00:1790076361.580688 1302 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5676worker2 # [ 85.409779] nix-grpc-daemon[1302]: E0000 00:00:1790076361.586460 1302 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5677worker2 # [ 85.416686] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5678worker2 # [ 85.420993] systemd[1]: Started Nix worker-protocol over gRPC.5679lb # [ 85.437571] niks3-server[969]: 2026/09/22 11:26:00 INFO lead: acquired remote=192.168.1.4:558745680worker2 # [ 85.427252] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=scheduler_take_over5681worker1 # [ 85.430746] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_connected addr=lb:500515682worker1 # [ 85.432818] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5683worker2 # [ 85.430166] nix-grpc-daemon[1302]: E0000 00:00:1790076361.606605 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5684worker2 # [ 85.446077] nix-grpc-daemon[1302]: E0000 00:00:1790076361.622937 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5685worker2 # [ 85.452079] nix-grpc-daemon[1302]: E0000 00:00:1790076361.625681 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5686worker2 # [ 85.463234] nix-grpc-daemon[1302]: E0000 00:00:1790076361.640156 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5687worker2 # [ 85.472549] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5688worker1 # [ 85.633416] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_connected addr=lb:500515689worker1 # [ 85.635534] nix-grpc-daemon[781]: ts=2026-09-22T11:26:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5690worker2 # [ 85.674083] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=scheduler_connected addr=lb:500515691worker2 # [ 85.676773] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5692worker1 # [ 85.836500] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=scheduler_connected addr=lb:500515693worker1 # [ 85.839112] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5694worker2 # [ 85.877893] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:02Z level=info event=scheduler_connected addr=lb:500515695worker2 # [ 85.879946] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5696worker1 # [ 86.039832] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=scheduler_connected addr=lb:500515697worker1 # [ 86.045672] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:399105698worker1 # [ 86.047314] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05699worker2 # [ 86.081097] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:02Z level=info event=scheduler_connected addr=lb:500515700worker1 # [ 86.084722] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:399105701worker1 # [ 86.086470] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05702worker1 # [ 86.181937] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=scheduler_restarting5703worker2 # [ 86.181262] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:02Z level=info event=scheduler_restarting addr=lb:500515704worker1 # [ 86.185549] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:399105705worker1 # [ 86.187489] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_gone addr=192.168.1.4:500515706worker1 # [ 86.189832] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=scheduler_restarting addr=lb:500515707worker1 # [ 86.191198] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:399105708worker1 # [ 86.192839] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=worker_gone addr=192.168.1.3:500515709worker1 # [ 86.205818] nix-daemon[847]: reaped child process 849, status = succeeded5710worker2 # [ 86.382645] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:02Z level=info event=scheduler_connected addr=lb:500515711worker1 # [ 86.386806] nix-grpc-daemon[781]: E0000 00:00:1790076361.596507 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5712worker1 # [ 86.398613] nix-grpc-daemon[781]: ts=2026-09-22T11:26:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5713worker1 # [ 86.928039] nix-grpc-daemon[781]: E0000 00:00:1790076362.138210 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5714worker1 # [ 86.930290] nix-grpc-daemon[781]: E0000 00:00:1790076362.138344 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5715worker1 # [ 88.230688] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5716worker1 # [ 88.233296] systemd[1]: Stopped Nix worker-protocol over gRPC.5717worker1 # [ 88.234544] systemd[1]: nix-grpc-daemon.service: Consumed 2.040s CPU time over 1min 13.558s wall clock time, 59.8M memory peak, 77K incoming IP traffic, 146K outgoing IP traffic.5718worker2 # [ 88.238592] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=info event=oidc_jwks provider=mock keys=15719worker1 # [ 88.240968] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5720worker2 # [ 88.240117] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:391485721worker1 # [ 88.243327] systemd[1]: Closed Nix worker-protocol over gRPC.5722worker2 # [ 88.242032] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05723worker1 # [ 88.244973] systemd[1]: nix-grpc-daemon.socket: Consumed 327us CPU time over 1min 20.687s wall clock time, 256K memory peak, 85.9K incoming IP traffic, 71.4K outgoing IP traffic.5724lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5725lb: must succeed: curl -sf localhost:9901/clusters5726lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5727lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5728client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25729worker2 # [ 88.401087] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391605730worker2 # [ 88.407288] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5731worker2 # [ 88.415410] nix-grpc-daemon[1302]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)5732worker2 # [ 88.486336] nix-grpc-daemon[1302]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 2/5)5733worker2 # [ 88.555557] nix-grpc-daemon[1302]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 113 ms (attempt 3/5)5734worker2 # [ 88.669951] nix-grpc-daemon[1302]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 4/5)5735worker2 # [ 88.760603] nix-grpc-daemon[1302]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5736worker2 # [ 88.761945] nix-grpc-daemon[1302]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5737client # copying 2 paths...5738client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5739client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5740worker2 # [ 88.774107] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:391605741worker2 # [ 88.781090] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5742worker2 # [ 88.794226] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:04Z level=info event=niks3_push_started pid=13625743worker2 # [ 88.897168] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.073Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5744worker2 # [ 89.002283] nix-daemon[868]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5745worker2 # [ 89.017875] nix-daemon[868]: reaped child process 1378, status = succeeded5746lb # [ 89.036699] niks3-server[969]: 2026/09/22 11:26:04 INFO Received uploads request method=POST path=/api/pending_closures5747lb # [ 89.041215] niks3-server[969]: 2026/09/22 11:26:04 INFO Received uploads request method=POST path=/api/pending_closures5748worker2 # [ 89.032596] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.209Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5749worker2 # [ 89.034138] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.211Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5750worker2 # [ 89.036090] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.213Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5751lb # [ 89.071823] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5752lb # [ 89.087890] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5753lb # [ 89.090613] niks3-server[969]: 2026/09/22 11:26:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5754lb # [ 89.092560] niks3-server[969]: 2026/09/22 11:26:04 INFO Signed narinfos id=29 count=15755lb # [ 89.094782] niks3-server[969]: 2026/09/22 11:26:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5756lb # [ 89.097051] niks3-server[969]: 2026/09/22 11:26:04 INFO Signed narinfos id=30 count=25757worker2 # [ 89.085846] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.261Z level=INFO msg="Uploading 3 narinfos"5758lb # [ 89.098432] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5759lb # [ 89.103264] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5760lb # [ 89.116147] niks3-server[969]: 2026/09/22 11:26:04 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5761lb # [ 89.124814] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5762lb # [ 89.129499] niks3-server[969]: 2026/09/22 11:26:04 INFO Completed upload id=295763lb # [ 89.131825] niks3-server[969]: 2026/09/22 11:26:04 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5764lb # [ 89.134255] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5765worker2 # [ 89.125231] nix-grpc-daemon[1362]: time=2026-09-22T11:26:05.302Z level=INFO msg="Upload complete. (223ms)"5766lb # [ 89.138358] niks3-server[969]: 2026/09/22 11:26:04 INFO Completed upload id=305767worker2 # [ 89.129098] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=0 paths=2 nar_bytes_in=3125768lb # [ 89.143710] niks3-server[969]: 2026/09/22 11:26:04 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5769worker2 # [ 89.137104] nix-daemon[868]: reaped child process 1360, status = succeeded5770client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 0.87 seconds)5771worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5772worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5773worker1 # [ 89.184901] systemd[1]: Listening on Nix worker-protocol over gRPC.5774worker1 # [ 89.188195] systemd[1]: Starting Nix worker-protocol over gRPC...5775worker1 # [ 89.275847] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5776worker1 # [ 89.277940] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5777worker1 # [ 89.281737] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15778worker1 # [ 89.286065] nix-grpc-daemon[1174]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5779worker1 # [ 89.287413] nix-grpc-daemon[1174]: E0000 00:00:1790076364.496080 1174 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5780worker1 # [ 89.293561] nix-grpc-daemon[1174]: E0000 00:00:1790076364.501548 1174 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5781worker1 # [ 89.297116] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5782worker1 # [ 89.300867] systemd[1]: Started Nix worker-protocol over gRPC.5783worker1 # [ 89.308152] nix-grpc-daemon[1174]: E0000 00:00:1790076364.518052 1214 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5784worker1 # [ 89.323076] nix-grpc-daemon[1174]: E0000 00:00:1790076364.532078 1214 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5785worker1 # [ 89.325438] nix-grpc-daemon[1174]: E0000 00:00:1790076364.532367 1214 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5786lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5787lb: must succeed: curl -sf localhost:9901/clusters5788worker1 # [ 89.337318] nix-grpc-daemon[1174]: E0000 00:00:1790076364.547692 1214 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5789worker1 # [ 89.346515] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=scheduler_connected addr=lb:500515790worker2 # [ 89.346799] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:391485791worker2 # [ 89.350210] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05792lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5793lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5794(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.08 seconds)5795subtest: a niks3 restart does not cost the leader its role5796lb: must succeed: curl -sf localhost:9901/clusters5797lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5798lb: must succeed: curl -sf localhost:9901/clusters5799lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5800worker1 # [ 89.426086] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:04Z level=info event=scheduler_lock_lost status=2005801lb # [ 89.436915] niks3-server[969]: 2026/09/22 11:26:04 INFO Shutdown signal received, draining in-flight requests timeout=10s5802lb # [ 89.439475] systemd[1]: Stopping niks3 server...5803worker2 # [ 89.432103] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:05Z level=info event=scheduler_lock_lost status=2005804lb # [ 89.444935] niks3-server[969]: 2026/09/22 11:26:04 INFO lead: released remote=192.168.1.4:558745805lb # [ 89.459618] systemd[1]: niks3.service: Deactivated successfully.5806lb # [ 89.461140] systemd[1]: Stopped niks3 server.5807lb # [ 89.461944] systemd[1]: niks3.service: Consumed 556ms CPU time over 1min 9.523s wall clock time, 27.4M memory peak, 78.2K incoming IP traffic, 113.1K outgoing IP traffic.5808lb # [ 89.470480] systemd[1]: Starting niks3 server...5809lb # [ 89.535392] niks3-server[1111]: 2026/09/22 11:26:04 goose: no migrations to run. current version: 202609200000005810lb # [ 89.541354] niks3-server[1111]: 2026/09/22 11:26:04 OK 1_commit_pending_closure.sql (4.32ms)5811lb # [ 89.544825] niks3-server[1111]: 2026/09/22 11:26:04 OK 2_object_stats_trigger.sql (3.09ms)5812lb # [ 89.545902] niks3-server[1111]: 2026/09/22 11:26:04 goose: up to current file version: 25813lb # [ 89.547472] niks3-server[1111]: 2026/09/22 11:26:04 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5814lb # [ 89.558891] niks3-server[1111]: 2026/09/22 11:26:04 INFO Read proxy enabled — serving cache objects from S35815lb # [ 89.560633] niks3-server[1111]: 2026/09/22 11:26:04 INFO Using socket-activated listener address=0.0.0.0:57515816lb # [ 89.562351] niks3-server[1111]: 2026/09/22 11:26:04 INFO systemd watchdog enabled interval=15s5817lb # [ 89.563727] systemd[1]: Started niks3 server.5818lb # [ 89.565585] niks3-server[1111]: 2026/09/22 11:26:05 INFO Starting HTTP server address=0.0.0.0:57515819lb: waiting for unit niks3.service5820lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5821lb: must succeed: curl -sf localhost:9901/clusters5822lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5823worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5824worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5825lb: must succeed: curl -sf localhost:9901/clusters5826lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5827lb: must succeed: curl -sf localhost:9901/clusters5828lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5829worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5830worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5831client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25832worker1 # [ 89.837137] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399645833worker1 # [ 89.851431] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5834client # don't know how to build these paths:5835client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5836worker2 # [ 89.862835] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391605837worker2 # [ 89.878997] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05838worker2 # [ 89.881111] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15839worker2 # [ 89.923398] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605840worker2 # [ 89.935136] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391605841client # copying 1 paths to 'lb:50051'...5842client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5843worker2 # [ 89.945536] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:391605844worker2 # [ 89.957166] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5845worker2 # [ 90.058552] nix-daemon[868]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5846worker2 # [ 90.078604] nix-daemon[868]: reaped child process 1398, status = succeeded5847worker2 # [ 90.150587] nix-daemon[868]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5848worker2 # [ 90.166809] nix-daemon[868]: reaped child process 1403, status = succeeded5849lb # [ 90.182434] niks3-server[1111]: 2026/09/22 11:26:05 INFO Received uploads request method=POST path=/api/pending_closures5850worker2 # [ 90.181778] nix-grpc-daemon[1362]: time=2026-09-22T11:26:06.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5851worker2 # [ 90.183501] nix-grpc-daemon[1362]: time=2026-09-22T11:26:06.358Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5852lb # [ 90.215319] niks3-server[1111]: 2026/09/22 11:26:05 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5853lb # [ 90.224699] niks3-server[1111]: 2026/09/22 11:26:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5854lb # [ 90.238182] niks3-server[1111]: 2026/09/22 11:26:05 INFO Signed narinfos id=31 count=15855worker2 # [ 90.228136] nix-grpc-daemon[1362]: time=2026-09-22T11:26:06.404Z level=INFO msg="Uploading 1 narinfos"5856lb # [ 90.246315] niks3-server[1111]: 2026/09/22 11:26:05 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5857lb # [ 90.253935] niks3-server[1111]: 2026/09/22 11:26:05 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5858lb # [ 90.261331] niks3-server[1111]: 2026/09/22 11:26:05 INFO Completed upload id=315859worker2 # [ 90.249205] nix-grpc-daemon[1362]: time=2026-09-22T11:26:06.426Z level=INFO msg="Upload complete. (279ms)"5860worker2 # [ 90.251234] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=0 paths=1 nar_bytes_in=5365861worker2 # [ 90.254208] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605862worker2 # [ 90.262761] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5863worker2 # [ 90.264138] nix-daemon[868]: reaped child process 1393, status = succeeded5864client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5865client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5866worker2 # [ 90.269939] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5867lb # [ 90.448283] niks3-server[1111]: 2026/09/22 11:26:05 INFO lead: acquired remote=192.168.1.4:559785868client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5869client # farm-a> LOG-farm-a-after-niks3-restart5870worker2 # [ 91.093953] nix-daemon[868]: reaped child process 1407, status = succeeded5871worker2 # [ 91.178391] nix-daemon[868]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)5872worker2 # [ 91.193377] nix-daemon[868]: reaped child process 1428, status = succeeded5873lb # [ 91.214541] niks3-server[1111]: 2026/09/22 11:26:06 INFO Received uploads request method=POST path=/api/pending_closures5874worker2 # [ 91.231065] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5875worker2 # [ 91.233054] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.409Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5876lb # [ 91.289895] niks3-server[1111]: 2026/09/22 11:26:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5877lb # [ 91.293489] niks3-server[1111]: 2026/09/22 11:26:06 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5878lb # [ 91.302240] niks3-server[1111]: 2026/09/22 11:26:06 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5879lb # [ 91.313065] niks3-server[1111]: 2026/09/22 11:26:06 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5880lb # [ 91.316114] niks3-server[1111]: 2026/09/22 11:26:06 INFO Signed narinfos id=32 count=15881worker2 # [ 91.304232] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.480Z level=INFO msg="Uploading 1 narinfos"5882lb # [ 91.322901] niks3-server[1111]: 2026/09/22 11:26:06 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5883lb # [ 91.329097] niks3-server[1111]: 2026/09/22 11:26:06 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5884lb # [ 91.335929] niks3-server[1111]: 2026/09/22 11:26:06 INFO Completed upload id=325885worker2 # [ 91.323960] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.500Z level=INFO msg="Upload complete. (245ms)"5886worker2 # [ 91.330783] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15887worker2 # [ 91.346177] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05888worker2 # [ 91.350286] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15889worker2 # [ 91.360476] nix-daemon[868]: reaped child process 1405, status = succeeded5890worker2 # [ 91.389390] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605891worker2 # [ 91.400418] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391605892worker2 # [ 91.410838] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.587Z level=INFO msg="All 1 paths already cached"5893client # copying 1 paths to 'lb:50051'...5894client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5895worker2 # [ 91.417127] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:391605896worker2 # [ 91.427072] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5897worker2 # [ 91.521139] nix-daemon[868]: accepted connection from pid 1434, user nix-grpc-daemon (trusted)5898worker2 # [ 91.534894] nix-daemon[868]: reaped child process 1437, status = succeeded5899worker2 # [ 91.616273] nix-daemon[868]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)5900worker2 # [ 91.630788] nix-daemon[868]: reaped child process 1442, status = succeeded5901lb # [ 91.648512] niks3-server[1111]: 2026/09/22 11:26:07 INFO Received uploads request method=POST path=/api/pending_closures5902worker2 # [ 91.647958] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.824Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5903worker2 # [ 91.649540] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.826Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5904lb # [ 91.694126] niks3-server[1111]: 2026/09/22 11:26:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5905lb # [ 91.698130] niks3-server[1111]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5906worker2 # [ 91.694559] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.871Z level=INFO msg="Uploading 1 narinfos"5907lb # [ 91.707254] niks3-server[1111]: 2026/09/22 11:26:07 INFO Signed narinfos id=33 count=15908lb # [ 91.713396] niks3-server[1111]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5909lb # [ 91.723353] niks3-server[1111]: 2026/09/22 11:26:07 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5910lb # [ 91.732594] niks3-server[1111]: 2026/09/22 11:26:07 INFO Completed upload id=335911worker2 # [ 91.720532] nix-grpc-daemon[1362]: time=2026-09-22T11:26:07.897Z level=INFO msg="Upload complete. (283ms)"5912worker2 # [ 91.722631] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=0 paths=1 nar_bytes_in=6565913lb # [ 91.735854] niks3-server[1111]: 2026/09/22 11:26:07 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5914worker2 # [ 91.726972] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605915worker2 # [ 91.734109] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5916worker2 # [ 91.736540] nix-daemon[868]: reaped child process 1432, status = succeeded5917client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5918client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5919worker2 # [ 91.743617] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5920client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5921client # farm-b> LOG-farm-b-after-niks3-restart5922worker2 # [ 92.758244] nix-daemon[868]: reaped child process 1446, status = succeeded5923worker2 # [ 92.832843] nix-daemon[868]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5924worker2 # [ 92.846525] nix-daemon[868]: reaped child process 1465, status = succeeded5925lb # [ 92.866826] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received uploads request method=POST path=/api/pending_closures5926worker2 # [ 92.864812] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5927worker2 # [ 92.866415] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.043Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5928lb # [ 92.910226] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5929lb # [ 92.922784] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5930lb # [ 92.929488] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5931lb # [ 92.934601] niks3-server[1111]: 2026/09/22 11:26:08 INFO Signed narinfos id=34 count=15932worker2 # [ 92.925862] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.102Z level=INFO msg="Uploading 1 narinfos"5933lb # [ 92.945690] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5934lb # [ 92.948691] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5935lb # [ 92.955575] niks3-server[1111]: 2026/09/22 11:26:08 INFO Completed upload id=345936worker2 # [ 92.943385] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.120Z level=INFO msg="Upload complete. (193ms)"5937lb # [ 92.957504] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5938worker2 # [ 92.953655] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15939worker2 # [ 92.962239] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05940worker2 # [ 92.967271] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15941worker2 # [ 92.976607] nix-daemon[868]: reaped child process 1444, status = succeeded5942worker2 # [ 93.006912] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605943worker2 # [ 93.019745] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:391605944worker2 # [ 93.033586] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.210Z level=INFO msg="All 2 paths already cached"5945client # copying 1 paths to 'lb:50051'...5946client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5947worker2 # [ 93.042252] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:391605948worker2 # [ 93.050585] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5949worker2 # [ 93.148977] nix-daemon[868]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5950worker2 # [ 93.166786] nix-daemon[868]: reaped child process 1474, status = succeeded5951worker2 # [ 93.237116] nix-daemon[868]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)5952worker2 # [ 93.251292] nix-daemon[868]: reaped child process 1479, status = succeeded5953lb # [ 93.268055] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received uploads request method=POST path=/api/pending_closures5954worker2 # [ 93.265200] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5955worker2 # [ 93.267163] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.443Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5956lb # [ 93.297158] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5957lb # [ 93.311355] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5958lb # [ 93.316709] niks3-server[1111]: 2026/09/22 11:26:08 INFO Signed narinfos id=35 count=15959worker2 # [ 93.307532] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.484Z level=INFO msg="Uploading 1 narinfos"5960lb # [ 93.322815] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5961lb # [ 93.336553] niks3-server[1111]: 2026/09/22 11:26:08 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5962lb # [ 93.344529] niks3-server[1111]: 2026/09/22 11:26:08 INFO Completed upload id=355963worker2 # [ 93.332395] nix-grpc-daemon[1362]: time=2026-09-22T11:26:09.509Z level=INFO msg="Upload complete. (268ms)"5964lb # [ 93.346116] niks3-server[1111]: 2026/09/22 11:26:08 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5965worker2 # [ 93.336503] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=0 paths=1 nar_bytes_in=7845966worker2 # [ 93.341244] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:391605967worker2 # [ 93.350737] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5968worker2 # [ 93.360566] nix-daemon[868]: reaped child process 1469, status = succeeded5969worker2 # [ 93.364682] nix-daemon[868]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5970client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5971client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5972client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5973client # farm-top> LOG-farm-top-after-niks3-restart5974worker2 # [ 94.005612] nix-daemon[868]: reaped child process 1483, status = succeeded5975worker2 # [ 94.085092] nix-daemon[868]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5976worker2 # [ 94.097772] nix-daemon[868]: reaped child process 1502, status = succeeded5977lb # [ 94.116981] niks3-server[1111]: 2026/09/22 11:26:09 INFO Received uploads request method=POST path=/api/pending_closures5978worker2 # [ 94.114057] nix-grpc-daemon[1362]: time=2026-09-22T11:26:10.290Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5979worker2 # [ 94.116052] nix-grpc-daemon[1362]: time=2026-09-22T11:26:10.292Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5980lb # [ 94.160678] niks3-server[1111]: 2026/09/22 11:26:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5981lb # [ 94.164773] niks3-server[1111]: 2026/09/22 11:26:09 INFO Signed narinfos id=36 count=15982worker2 # [ 94.154986] nix-grpc-daemon[1362]: time=2026-09-22T11:26:10.331Z level=INFO msg="Uploading 1 narinfos"5983lb # [ 94.174128] niks3-server[1111]: 2026/09/22 11:26:09 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5984lb # [ 94.181099] niks3-server[1111]: 2026/09/22 11:26:09 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5985lb # [ 94.194147] niks3-server[1111]: 2026/09/22 11:26:09 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5986lb # [ 94.196513] niks3-server[1111]: 2026/09/22 11:26:09 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5987lb # [ 94.202133] niks3-server[1111]: 2026/09/22 11:26:09 INFO Completed upload id=365988worker2 # [ 94.190187] nix-grpc-daemon[1362]: time=2026-09-22T11:26:10.367Z level=INFO msg="Upload complete. (218ms)"5989lb # [ 94.205058] niks3-server[1111]: 2026/09/22 11:26:09 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5990worker2 # [ 94.197766] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39160 duration_s=0 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15991worker2 # [ 94.211612] nix-daemon[868]: reaped child process 1481, status = succeeded5992client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.51 seconds)5993client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5994client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)5995(finished: subtest: a niks3 restart does not cost the leader its role, in 4.98 seconds)5996subtest: scheduler down: the other node takes the lock and keeps it5997lb: must succeed: curl -sf localhost:9901/clusters5998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5999lb: must succeed: curl -sf localhost:9901/clusters6000lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6001lb: must succeed: curl -sf localhost:9901/clusters6002lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6003lb: must succeed: curl -sf localhost:9901/clusters6004lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6005worker2 # [ 94.510753] systemd[1]: Stopping Nix worker-protocol over gRPC...6006lb # [ 94.691476] envoy[896]: conn peer=[2001:db8:1::3]:60534 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=21438 tx=4026 ms=715516007lb # [ 94.696235] envoy[896]: conn peer=[2001:db8:1::1]:48848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1003 tx=282 ms=7366008lb # [ 94.701792] envoy[896]: conn peer=[2001:db8:1::1]:48850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4066 ms=43756009worker2 # [ 95.418139] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:11Z level=info event=signal_exit6010lb # [ 95.451584] niks3-server[1111]: 2026/09/22 11:26:10 INFO lead: released remote=192.168.1.4:559786011worker1 # [ 96.442378] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:11Z level=info event=scheduler_restarting addr=lb:500516012worker2 # [ 96.444886] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=scheduler_restarting6013worker2 # [ 96.448765] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:391486014worker2 # [ 96.452862] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=worker_gone addr=192.168.1.3:500516015worker2 # [ 96.456464] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=scheduler_restarting addr=lb:500516016worker2 # [ 96.459801] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:391486017worker2 # [ 96.463314] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=worker_gone addr=192.168.1.4:500516018worker2 # [ 96.466583] nix-grpc-daemon[1302]: E0000 00:00:1790076372.629597 1347 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6019worker2 # [ 96.471587] nix-grpc-daemon[1302]: E0000 00:00:1790076372.633278 1347 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6020worker2 # [ 96.489361] nix-daemon[868]: reaped child process 1354, status = succeeded6021worker1 # [ 96.642233] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:11Z level=info event=scheduler_connected addr=lb:500516022worker1 # [ 96.644341] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6023worker2 # [ 96.655729] nix-grpc-daemon[1302]: ts=2026-09-22T11:26:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6024worker1 # [ 96.845413] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_connected addr=lb:500516025worker1 # [ 96.849286] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6026worker1 # [ 97.049738] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_connected addr=lb:500516027worker1 # [ 97.052222] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6028worker1 # [ 97.253140] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_connected addr=lb:500516029worker1 # [ 97.257212] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6030worker1 # [ 97.458767] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_connected addr=lb:500516031worker1 # [ 97.462581] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6032worker1 # [ 97.663598] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_connected addr=lb:500516033worker2 # [ 97.666764] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6034worker1 # [ 97.667422] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6035worker2 # [ 97.668452] systemd[1]: Stopped Nix worker-protocol over gRPC.6036worker2 # [ 97.669811] systemd[1]: nix-grpc-daemon.service: Consumed 1.093s CPU time over 12.365s wall clock time, 39.2M memory peak, 46.5K incoming IP traffic, 82.9K outgoing IP traffic.6037worker2 # [ 97.674710] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6038worker2 # [ 97.676409] systemd[1]: Closed Nix worker-protocol over gRPC.6039worker2 # [ 97.677798] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 13.200s wall clock time, 256K memory peak, 36.8K incoming IP traffic, 28.7K outgoing IP traffic.6040worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6041worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)6042worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6043worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6044worker1 # [ 97.868291] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_connected addr=lb:500516045worker1 # [ 97.872177] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6046worker1 # [ 98.072964] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_connected addr=lb:500516047worker1 # [ 98.077131] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6048worker1 # [ 98.277525] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_connected addr=lb:500516049worker1 # [ 98.279363] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6050worker1 # [ 98.481297] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_connected addr=lb:500516051worker1 # [ 98.485203] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6052worker1 # [ 98.686298] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_connected addr=lb:500516053worker1 # [ 98.690634] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6054worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6055worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6056worker1 # [ 98.891747] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_connected addr=lb:500516057worker1 # [ 98.896479] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6058worker1 # [ 99.097054] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_connected addr=lb:500516059worker1 # [ 99.099372] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6060worker1 # [ 99.299836] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_connected addr=lb:500516061worker1 # [ 99.301676] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6062worker1 # [ 99.504473] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_connected addr=lb:500516063worker1 # [ 99.507612] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6064worker1 # [ 99.705209] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_connected addr=lb:500516065worker1 # [ 99.706981] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6066worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6067worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6068worker1 # [ 99.908094] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_connected addr=lb:500516069worker1 # [ 99.912287] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6070worker1 # [ 100.113292] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_connected addr=lb:500516071worker1 # [ 100.117573] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6072worker1 # [ 100.318546] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_connected addr=lb:500516073worker1 # [ 100.322582] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6074worker1 # [ 100.431990] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_take_over6075lb # [ 100.442401] niks3-server[1111]: 2026/09/22 11:26:15 INFO lead: acquired remote=192.168.1.3:515366076worker1 # [ 100.524759] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_connected addr=lb:500516077worker1 # [ 100.528373] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6078worker1 # [ 100.729480] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_connected addr=lb:500516079worker1 # [ 100.733562] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6080worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6081worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6082worker1 # [ 100.934404] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=scheduler_connected addr=lb:500516083worker1 # [ 100.936585] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6084worker1 # [ 101.137845] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=scheduler_connected addr=lb:500516085worker1 # [ 101.141961] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6086worker1 # [ 101.342393] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=scheduler_connected addr=lb:500516087worker1 # [ 101.349370] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:426766088worker1 # [ 101.353858] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06089worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6090worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6091client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26092worker1 # [ 102.032464] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399646093client # don't know how to build these paths:6094client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6095worker1 # [ 102.040856] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399646096worker1 # [ 102.051551] nix-grpc-daemon[1174]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)6097worker1 # [ 102.099081] nix-grpc-daemon[1174]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 2/5)6098worker1 # [ 102.159121] nix-grpc-daemon[1174]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 375 ms (attempt 3/5)6099worker1 # [ 102.535728] nix-grpc-daemon[1174]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 531 ms (attempt 4/5)6100worker1 # [ 103.069898] nix-grpc-daemon[1174]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6101worker1 # [ 103.073818] nix-grpc-daemon[1174]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6102worker1 # [ 103.099154] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06103worker1 # [ 103.103086] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16104worker1 # [ 103.150884] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646105worker1 # [ 103.160201] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399646106client # copying 1 paths to 'lb:50051'...6107client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6108worker1 # [ 103.171698] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399646109worker1 # [ 103.181075] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6110worker1 # [ 103.205086] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=info event=niks3_push_started pid=12596111worker1 # [ 103.316949] nix-grpc-daemon[1259]: time=2026-09-22T11:26:18.526Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6112worker1 # [ 103.409728] nix-daemon[847]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6113worker1 # [ 103.423444] nix-daemon[847]: reaped child process 1275, status = succeeded6114worker1 # [ 103.512466] nix-daemon[847]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6115worker1 # [ 103.527302] nix-daemon[847]: reaped child process 1280, status = succeeded6116lb # [ 103.543091] niks3-server[1111]: 2026/09/22 11:26:18 INFO Received uploads request method=POST path=/api/pending_closures6117worker1 # [ 103.542137] nix-grpc-daemon[1259]: time=2026-09-22T11:26:18.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6118worker1 # [ 103.544021] nix-grpc-daemon[1259]: time=2026-09-22T11:26:18.754Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6119lb # [ 103.583775] niks3-server[1111]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6120lb # [ 103.600446] niks3-server[1111]: 2026/09/22 11:26:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6121worker1 # [ 103.594615] nix-grpc-daemon[1259]: time=2026-09-22T11:26:18.804Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 103.604921] niks3-server[1111]: 2026/09/22 11:26:19 INFO Signed narinfos id=37 count=16123lb # [ 103.613352] niks3-server[1111]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6124lb # [ 103.622822] niks3-server[1111]: 2026/09/22 11:26:19 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6125lb # [ 103.625941] niks3-server[1111]: 2026/09/22 11:26:19 INFO Completed upload id=376126worker1 # [ 103.615917] nix-grpc-daemon[1259]: time=2026-09-22T11:26:18.826Z level=INFO msg="Upload complete. (295ms)"6127lb # [ 103.629893] niks3-server[1111]: 2026/09/22 11:26:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6128worker1 # [ 103.620526] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=0 paths=1 nar_bytes_in=5206129worker1 # [ 103.625965] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646130worker1 # [ 103.634550] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6131worker1 # [ 103.641234] nix-daemon[847]: reaped child process 1257, status = succeeded6132worker1 # [ 103.646663] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6133client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6134client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6135client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6136client # farm-a> LOG-farm-a-on-standby6137worker1 # [ 104.463925] nix-daemon[847]: reaped child process 1284, status = succeeded6138worker1 # [ 104.538663] nix-daemon[847]: accepted connection from pid 1300, user nix-grpc-daemon (trusted)6139worker1 # [ 104.552307] nix-daemon[847]: reaped child process 1303, status = succeeded6140lb # [ 104.568186] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received uploads request method=POST path=/api/pending_closures6141worker1 # [ 104.566499] nix-grpc-daemon[1259]: time=2026-09-22T11:26:19.776Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6142worker1 # [ 104.569083] nix-grpc-daemon[1259]: time=2026-09-22T11:26:19.778Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6143lb # [ 104.599417] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6144lb # [ 104.615595] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6145lb # [ 104.622627] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6146lb # [ 104.627703] niks3-server[1111]: 2026/09/22 11:26:20 INFO Signed narinfos id=38 count=16147worker1 # [ 104.619833] nix-grpc-daemon[1259]: time=2026-09-22T11:26:19.830Z level=INFO msg="Uploading 1 narinfos"6148lb # [ 104.633107] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6149lb # [ 104.642169] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6150lb # [ 104.645478] niks3-server[1111]: 2026/09/22 11:26:20 INFO Completed upload id=386151lb # [ 104.647429] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6152worker1 # [ 104.636791] nix-grpc-daemon[1259]: time=2026-09-22T11:26:19.847Z level=INFO msg="Upload complete. (183ms)"6153worker1 # [ 104.650954] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16154worker1 # [ 104.656518] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06155worker1 # [ 104.659652] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16156worker1 # [ 104.667866] nix-daemon[847]: reaped child process 1282, status = succeeded6157lb # [ 104.692150] envoy[896]: conn peer=[2001:db8:1::4]:48490 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4934 tx=641 ms=111776158worker1 # [ 104.695152] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646159worker1 # [ 104.706332] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399646160worker1 # [ 104.717258] nix-grpc-daemon[1259]: time=2026-09-22T11:26:19.927Z level=INFO msg="All 1 paths already cached"6161client # copying 1 paths to 'lb:50051'...6162client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6163worker1 # [ 104.724690] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399646164worker1 # [ 104.731289] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6165worker1 # [ 104.819174] nix-daemon[847]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)6166worker1 # [ 104.833524] nix-daemon[847]: reaped child process 1312, status = succeeded6167worker1 # [ 104.911190] nix-daemon[847]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6168worker1 # [ 104.926144] nix-daemon[847]: reaped child process 1317, status = succeeded6169lb # [ 104.941963] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received uploads request method=POST path=/api/pending_closures6170worker1 # [ 104.942866] nix-grpc-daemon[1259]: time=2026-09-22T11:26:20.152Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6171worker1 # [ 104.944219] nix-grpc-daemon[1259]: time=2026-09-22T11:26:20.153Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6172lb # [ 104.969630] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6173lb # [ 104.981814] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6174lb # [ 104.987558] niks3-server[1111]: 2026/09/22 11:26:20 INFO Signed narinfos id=39 count=16175worker1 # [ 104.980846] nix-grpc-daemon[1259]: time=2026-09-22T11:26:20.190Z level=INFO msg="Uploading 1 narinfos"6176lb # [ 104.993978] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6177lb # [ 105.006143] niks3-server[1111]: 2026/09/22 11:26:20 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6178lb # [ 105.013110] niks3-server[1111]: 2026/09/22 11:26:20 INFO Completed upload id=396179worker1 # [ 105.002349] nix-grpc-daemon[1259]: time=2026-09-22T11:26:20.212Z level=INFO msg="Upload complete. (262ms)"6180worker1 # [ 105.005974] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=0 paths=1 nar_bytes_in=6406181lb # [ 105.018974] niks3-server[1111]: 2026/09/22 11:26:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6182worker1 # [ 105.011302] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646183worker1 # [ 105.019490] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6184worker1 # [ 105.025760] nix-daemon[847]: reaped child process 1307, status = succeeded6185worker1 # [ 105.030321] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6186client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6187client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6188client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6189client # farm-b> LOG-farm-b-on-standby6190worker1 # [ 106.668278] nix-daemon[847]: reaped child process 1321, status = succeeded6191worker1 # [ 106.744698] nix-daemon[847]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)6192worker1 # [ 106.761318] nix-daemon[847]: reaped child process 1340, status = succeeded6193lb # [ 106.771540] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received uploads request method=POST path=/api/pending_closures6194worker1 # [ 106.769804] nix-grpc-daemon[1259]: time=2026-09-22T11:26:21.979Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6195worker1 # [ 106.771307] nix-grpc-daemon[1259]: time=2026-09-22T11:26:21.981Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6196lb # [ 106.810207] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6197lb # [ 106.820432] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6198lb # [ 106.827334] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6199lb # [ 106.835374] niks3-server[1111]: 2026/09/22 11:26:22 INFO Signed narinfos id=40 count=16200worker1 # [ 106.825838] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.035Z level=INFO msg="Uploading 1 narinfos"6201lb # [ 106.843076] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6202lb # [ 106.851205] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6203lb # [ 106.853430] niks3-server[1111]: 2026/09/22 11:26:22 INFO Completed upload id=406204worker1 # [ 106.844561] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.054Z level=INFO msg="Upload complete. (206ms)"6205lb # [ 106.856327] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6206worker1 # [ 106.847236] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16207worker1 # [ 106.854636] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06208worker1 # [ 106.856619] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16209worker1 # [ 106.862946] nix-daemon[847]: reaped child process 1319, status = succeeded6210worker1 # [ 106.898768] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646211worker1 # [ 106.910655] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399646212worker1 # [ 106.922865] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.133Z level=INFO msg="All 2 paths already cached"6213client # copying 1 paths to 'lb:50051'...6214client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6215worker1 # [ 106.931743] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399646216worker1 # [ 106.939933] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6217worker1 # [ 107.036082] nix-daemon[847]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)6218worker1 # [ 107.049874] nix-daemon[847]: reaped child process 1349, status = succeeded6219worker1 # [ 107.129738] nix-daemon[847]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)6220worker1 # [ 107.143209] nix-daemon[847]: reaped child process 1354, status = succeeded6221lb # [ 107.159179] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received uploads request method=POST path=/api/pending_closures6222worker1 # [ 107.154329] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.364Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6223worker1 # [ 107.155892] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.366Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6224lb # [ 107.176856] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6225lb # [ 107.180466] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6226lb # [ 107.182679] niks3-server[1111]: 2026/09/22 11:26:22 INFO Signed narinfos id=41 count=16227worker1 # [ 107.172365] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.382Z level=INFO msg="Uploading 1 narinfos"6228lb # [ 107.185722] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6229lb # [ 107.192121] niks3-server[1111]: 2026/09/22 11:26:22 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6230lb # [ 107.196156] niks3-server[1111]: 2026/09/22 11:26:22 INFO Completed upload id=416231worker1 # [ 107.185958] nix-grpc-daemon[1259]: time=2026-09-22T11:26:22.396Z level=INFO msg="Upload complete. (234ms)"6232lb # [ 107.198643] niks3-server[1111]: 2026/09/22 11:26:22 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6233worker1 # [ 107.188076] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=0 paths=1 nar_bytes_in=7686234worker1 # [ 107.191078] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646235worker1 # [ 107.196529] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6236worker1 # [ 107.200621] nix-daemon[847]: reaped child process 1344, status = succeeded6237worker1 # [ 107.205577] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6238client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6239client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6240client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6241client # farm-top> LOG-farm-top-on-standby6242worker1 # [ 108.424737] nix-daemon[847]: reaped child process 1358, status = succeeded6243worker1 # [ 108.494482] nix-daemon[847]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6244lb # [ 108.519901] niks3-server[1111]: 2026/09/22 11:26:23 INFO Received uploads request method=POST path=/api/pending_closures6245worker1 # [ 108.510481] nix-daemon[847]: reaped child process 1377, status = succeeded6246worker1 # [ 108.514314] nix-grpc-daemon[1259]: time=2026-09-22T11:26:23.724Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6247worker1 # [ 108.515758] nix-grpc-daemon[1259]: time=2026-09-22T11:26:23.726Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6248lb # [ 108.540249] niks3-server[1111]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6249lb # [ 108.545028] niks3-server[1111]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6250lb # [ 108.549244] niks3-server[1111]: 2026/09/22 11:26:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6251lb # [ 108.552068] niks3-server[1111]: 2026/09/22 11:26:23 INFO Signed narinfos id=42 count=16252worker1 # [ 108.541996] nix-grpc-daemon[1259]: time=2026-09-22T11:26:23.752Z level=INFO msg="Uploading 1 narinfos"6253lb # [ 108.555177] niks3-server[1111]: 2026/09/22 11:26:23 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6254lb # [ 108.560387] niks3-server[1111]: 2026/09/22 11:26:23 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6255lb # [ 108.564210] niks3-server[1111]: 2026/09/22 11:26:23 INFO Completed upload id=426256worker1 # [ 108.555281] nix-grpc-daemon[1259]: time=2026-09-22T11:26:23.765Z level=INFO msg="Upload complete. (142ms)"6257lb # [ 108.566465] niks3-server[1111]: 2026/09/22 11:26:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6258worker1 # [ 108.558866] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16259worker1 # [ 108.565885] nix-daemon[847]: reaped child process 1356, status = succeeded6260client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 6.66 seconds)6261client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6262client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.09 seconds)6263worker2 # [ 108.711814] systemd[1]: Listening on Nix worker-protocol over gRPC.6264worker2 # [ 108.717113] systemd[1]: Starting scheduler-token.service...6265worker2 # [ 108.760634] systemd[1]: scheduler-token.service: Deactivated successfully.6266worker2 # [ 108.762319] systemd[1]: Finished scheduler-token.service.6267worker2 # [ 108.763699] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 45ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6268worker2 # [ 108.768247] systemd[1]: Starting Nix worker-protocol over gRPC...6269worker2 # [ 108.848068] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6270worker2 # [ 108.850065] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6271worker2 # [ 108.853294] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16272worker2 # [ 108.857600] nix-grpc-daemon[1523]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6273worker2 # [ 108.858978] nix-grpc-daemon[1523]: E0000 00:00:1790076385.034441 1523 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6274worker2 # [ 108.865258] nix-grpc-daemon[1523]: E0000 00:00:1790076385.040007 1523 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6275worker2 # [ 108.868976] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6276worker2 # [ 108.872799] systemd[1]: Started Nix worker-protocol over gRPC.6277worker2 # [ 108.876128] nix-grpc-daemon[1523]: E0000 00:00:1790076385.051737 1564 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6278worker2 # [ 108.889069] nix-grpc-daemon[1523]: E0000 00:00:1790076385.065474 1564 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6279lb: must succeed: curl -sf localhost:9901/clusters6280worker2 # [ 108.892286] nix-grpc-daemon[1523]: E0000 00:00:1790076385.065601 1564 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6281worker2 # [ 108.902992] nix-grpc-daemon[1523]: E0000 00:00:1790076385.080007 1564 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6282worker2 # [ 108.914767] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:25Z level=info event=scheduler_connected addr=lb:500516283worker1 # [ 108.924174] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=info event=oidc_jwks provider=mock keys=16284lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6285worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6286worker1 # [ 108.928710] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:426766287worker1 # [ 108.933138] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06288worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6289lb: must succeed: curl -sf localhost:9901/clusters6290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6291lb: must succeed: curl -sf localhost:9901/clusters6292lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6293worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6294worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6295(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.72 seconds)6296subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6297client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6298client # [ 109.112957] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6299client # Running as unit: rs.service; invocation ID: b5645f3cf71e454daf9200618da3b4756300client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6301client # [ 109.257446] nix[1296]: don't know how to build these paths:6302worker1 # [ 109.251099] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399646303client # [ 109.258532] nix[1296]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6304worker1 # [ 109.259222] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06305worker1 # [ 109.261840] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16306worker1 # [ 109.302762] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646307worker1 # [ 109.313631] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399646308client # [ 109.327104] nix[1296]: copying 1 paths to 'lb:50051'...6309client # [ 109.327987] nix[1296]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6310worker1 # [ 109.322947] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399646311worker1 # [ 109.331109] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6312worker1 # [ 109.417412] nix-daemon[847]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6313worker1 # [ 109.433403] nix-daemon[847]: reaped child process 1393, status = succeeded6314worker1 # [ 109.509594] nix-daemon[847]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6315worker1 # [ 109.522687] nix-daemon[847]: reaped child process 1398, status = succeeded6316lb # [ 109.538422] niks3-server[1111]: 2026/09/22 11:26:24 INFO Received uploads request method=POST path=/api/pending_closures6317worker1 # [ 109.533039] nix-grpc-daemon[1259]: time=2026-09-22T11:26:24.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6318worker1 # [ 109.534443] nix-grpc-daemon[1259]: time=2026-09-22T11:26:24.742Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6319lb # [ 109.550589] niks3-server[1111]: 2026/09/22 11:26:24 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6320lb # [ 109.559981] niks3-server[1111]: 2026/09/22 11:26:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6321lb # [ 109.564368] niks3-server[1111]: 2026/09/22 11:26:24 INFO Signed narinfos id=43 count=16322worker1 # [ 109.558058] nix-grpc-daemon[1259]: time=2026-09-22T11:26:24.768Z level=INFO msg="Uploading 1 narinfos"6323lb # [ 109.571357] niks3-server[1111]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6324lb # [ 109.577061] niks3-server[1111]: 2026/09/22 11:26:25 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6325lb # [ 109.580921] niks3-server[1111]: 2026/09/22 11:26:25 INFO Completed upload id=436326worker1 # [ 109.570660] nix-grpc-daemon[1259]: time=2026-09-22T11:26:24.781Z level=INFO msg="Upload complete. (232ms)"6327worker1 # [ 109.572859] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=0 paths=1 nar_bytes_in=4166328lb # [ 109.583725] niks3-server[1111]: 2026/09/22 11:26:25 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6329worker1 # [ 109.576319] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399646330worker1 # [ 109.583142] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6331worker1 # [ 109.585443] nix-daemon[847]: reaped child process 1388, status = succeeded6332worker1 # [ 109.590423] nix-daemon[847]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)6333client # [ 109.594776] nix[1296]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6334lb: must succeed: curl -sf localhost:9901/clusters6335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6336lb: must succeed: curl -sf localhost:9901/clusters6337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6338lb: must succeed: curl -sf localhost:9901/clusters6339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6340lb: must succeed: curl -sf localhost:9901/clusters6341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6342worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6343worker2 # [ 111.687062] systemd-journald[284]: Received client request to rotate journal, rotating.6344worker2 # [ 111.697510] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9b94986bac5743088c4a144d9c816274.6345worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.03 seconds)6346worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6347worker1 # [ 111.593323] systemd[1]: Stopping Nix worker-protocol over gRPC...6348worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6349client: must succeed: sleep 5.06350worker1 # [ 112.300624] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:27Z level=info event=draining6351lb # [ 114.693909] envoy[896]: conn peer=[2001:db8:1::1]:37334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9620 tx=4097 ms=65386352client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6353worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6354worker1 # [ 116.812286] nix-daemon[847]: reaped child process 1402, status = succeeded6355worker1 # [ 116.886787] nix-daemon[847]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6356worker1 # [ 116.900255] nix-daemon[847]: reaped child process 1446, status = succeeded6357lb # [ 116.917966] niks3-server[1111]: 2026/09/22 11:26:32 INFO Received uploads request method=POST path=/api/pending_closures6358worker1 # [ 116.916907] nix-grpc-daemon[1259]: time=2026-09-22T11:26:32.127Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6359worker1 # [ 116.919812] nix-grpc-daemon[1259]: time=2026-09-22T11:26:32.128Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6360lb # [ 116.952167] niks3-server[1111]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6361lb # [ 116.963629] niks3-server[1111]: 2026/09/22 11:26:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6362worker1 # [ 116.963985] nix-grpc-daemon[1259]: time=2026-09-22T11:26:32.174Z level=INFO msg="Uploading 1 narinfos"6363lb # [ 116.974517] niks3-server[1111]: 2026/09/22 11:26:32 INFO Signed narinfos id=44 count=16364lb # [ 116.981766] niks3-server[1111]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6365lb # [ 116.997082] niks3-server[1111]: 2026/09/22 11:26:32 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6366lb # [ 117.000049] niks3-server[1111]: 2026/09/22 11:26:32 INFO Completed upload id=446367worker1 # [ 116.990000] nix-grpc-daemon[1259]: time=2026-09-22T11:26:32.200Z level=INFO msg="Upload complete. (187ms)"6368lb # [ 117.003025] niks3-server[1111]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6369lb # [ 117.009253] niks3-server[1111]: 2026/09/22 11:26:32 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6370worker1 # [ 116.997818] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39964 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16371worker1 # [ 117.011410] nix-daemon[847]: reaped child process 1400, status = succeeded6372client # [ 117.037323] systemd[1]: rs.service: Deactivated successfully.6373client # [ 117.040278] systemd[1]: rs.service: Consumed 128ms CPU time over 7.923s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.6374worker1 # [ 117.302194] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:32Z level=info event=signal_exit6375lb # [ 117.446036] niks3-server[1111]: 2026/09/22 11:26:32 INFO lead: released remote=192.168.1.3:515366376worker2 # [ 118.434110] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:34Z level=info event=scheduler_restarting addr=lb:500516377worker1 # [ 118.440886] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=scheduler_restarting6378worker1 # [ 118.444589] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=scheduler_restarting addr=lb:500516379worker1 # [ 118.447798] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:426766380worker1 # [ 118.451423] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=worker_gone addr=192.168.1.3:500516381worker1 # [ 118.454935] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:426766382worker1 # [ 118.458567] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=worker_gone addr=192.168.1.4:500516383worker1 # [ 118.474460] nix-daemon[847]: reaped child process 1226, status = succeeded6384worker2 # [ 118.634527] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:34Z level=info event=scheduler_connected addr=lb:500516385worker2 # [ 118.636897] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6386worker1 # [ 118.649344] nix-grpc-daemon[1174]: ts=2026-09-22T11:26:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6387worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.06 seconds)6388client: waiting for success: ! systemctl is-active rs6389worker2 # [ 118.837531] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516390worker2 # [ 118.839362] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6391client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6392client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6393lb # [ 118.887189] niks3-server[1111]: 2026/09/22 11:26:34 INFO lead: acquired remote=192.168.1.4:428246394worker2 # [ 118.877776] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_take_over6395client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6396client: must fail: journalctl -u rs | grep -q 'reconnecting'6397client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6398lb: must succeed: curl -sf localhost:9901/clusters6399lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6400worker1 # [ 119.019898] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6401worker1 # [ 119.022866] systemd[1]: Stopped Nix worker-protocol over gRPC.6402worker1 # [ 119.026522] systemd[1]: nix-grpc-daemon.service: Consumed 1.300s CPU time over 29.830s wall clock time, 42.7M memory peak, 51.2K incoming IP traffic, 104.3K outgoing IP traffic.6403worker1 # [ 119.043408] systemd[1]: Starting Nix worker-protocol over gRPC...6404worker2 # [ 119.040636] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516405worker2 # [ 119.044738] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6406worker1 # [ 119.138845] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6407worker1 # [ 119.140895] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6408worker1 # [ 119.143991] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16409worker1 # [ 119.148394] nix-grpc-daemon[1453]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6410worker1 # [ 119.149735] nix-grpc-daemon[1453]: E0000 00:00:1790076394.358570 1453 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6411worker1 # [ 119.155937] nix-grpc-daemon[1453]: E0000 00:00:1790076394.363951 1453 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6412worker1 # [ 119.159451] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6413worker1 # [ 119.162277] systemd[1]: Started Nix worker-protocol over gRPC.6414worker1 # [ 119.165090] nix-grpc-daemon[1453]: E0000 00:00:1790076394.374059 1494 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6415worker1 # [ 119.175677] nix-grpc-daemon[1453]: E0000 00:00:1790076394.386008 1494 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6416worker1 # [ 119.179126] nix-grpc-daemon[1453]: E0000 00:00:1790076394.388203 1494 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6417worker1 # [ 119.189825] nix-grpc-daemon[1453]: E0000 00:00:1790076394.400166 1494 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6418worker1 # [ 119.197240] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_connected addr=lb:500516419worker1 # [ 119.198795] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6420worker2 # [ 119.245702] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516421worker2 # [ 119.249534] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6422worker1 # [ 119.400278] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_connected addr=lb:500516423worker1 # [ 119.404151] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6424worker2 # [ 119.451595] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516425worker2 # [ 119.455224] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6426worker1 # [ 119.604935] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_connected addr=lb:500516427worker1 # [ 119.609087] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6428worker2 # [ 119.655877] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516429worker2 # [ 119.659984] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6430worker1 # [ 119.809398] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:35Z level=info event=scheduler_connected addr=lb:500516431worker2 # [ 119.817270] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:530846432worker2 # [ 119.821684] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06433worker2 # [ 119.860576] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=info event=scheduler_connected addr=lb:500516434worker2 # [ 119.870136] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=info event=oidc_jwks provider=mock keys=16435worker2 # [ 119.873947] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:530846436worker2 # [ 119.878167] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06437lb: must succeed: curl -sf localhost:9901/clusters6438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6439worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6440worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6441(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 10.98 seconds)6442subtest: scheduler restart mid-build: build finishes, no double build6443client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6444client # [ 120.094696] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6445client # Running as unit: mid.service; invocation ID: 53669231a416430daeb4ef3f5069f4436446client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6447worker2 # [ 120.230180] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:530966448worker2 # [ 120.242342] nix-daemon[868]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6449client # [ 120.259901] nix[1332]: don't know how to build these paths:6450client # [ 120.260912] nix[1332]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6451worker2 # [ 120.262081] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06452worker2 # [ 120.266076] nix-grpc-daemon[1523]: ts=2026-09-22T11:26:36Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16453worker1 # [ 120.319435] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443186454worker1 # [ 120.325529] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6455worker1 # [ 120.333614] nix-grpc-daemon[1453]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 1/5)6456worker1 # [ 120.406219] nix-grpc-daemon[1453]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 2/5)6457worker1 # [ 120.555962] nix-grpc-daemon[1453]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 3/5)6458worker1 # [ 120.831411] nix-grpc-daemon[1453]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 730 ms (attempt 4/5)6459worker1 # [ 121.564870] nix-grpc-daemon[1453]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6460worker1 # [ 121.568844] nix-grpc-daemon[1453]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6461worker1 # [ 121.585586] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443186462client # [ 121.601595] nix[1332]: copying 1 paths to 'lb:50051'...6463client # [ 121.605878] nix[1332]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6464worker1 # [ 121.600868] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443186465worker1 # [ 121.612827] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6466worker1 # [ 121.633334] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:36Z level=info event=niks3_push_started pid=15216467worker1 # [ 121.740191] nix-grpc-daemon[1521]: time=2026-09-22T11:26:36.950Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6468worker1 # [ 121.828795] nix-daemon[847]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)6469worker1 # [ 121.841250] nix-daemon[847]: reaped child process 1537, status = succeeded6470worker1 # [ 121.926197] nix-daemon[847]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6471worker1 # [ 121.941123] nix-daemon[847]: reaped child process 1542, status = succeeded6472lb # [ 121.957426] niks3-server[1111]: 2026/09/22 11:26:37 INFO Received uploads request method=POST path=/api/pending_closures6473worker1 # [ 121.956285] nix-grpc-daemon[1521]: time=2026-09-22T11:26:37.166Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6474worker1 # [ 121.958049] nix-grpc-daemon[1521]: time=2026-09-22T11:26:37.168Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6475lb # [ 121.993385] niks3-server[1111]: 2026/09/22 11:26:37 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6476lb # [ 122.005583] niks3-server[1111]: 2026/09/22 11:26:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6477worker1 # [ 122.001282] nix-grpc-daemon[1521]: time=2026-09-22T11:26:37.211Z level=INFO msg="Uploading 1 narinfos"6478lb # [ 122.011770] niks3-server[1111]: 2026/09/22 11:26:37 INFO Signed narinfos id=45 count=16479lb # [ 122.024735] niks3-server[1111]: 2026/09/22 11:26:37 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6480lb # [ 122.033416] niks3-server[1111]: 2026/09/22 11:26:37 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6481lb # [ 122.039949] niks3-server[1111]: 2026/09/22 11:26:37 INFO Completed upload id=456482worker1 # [ 122.029802] nix-grpc-daemon[1521]: time=2026-09-22T11:26:37.240Z level=INFO msg="Upload complete. (286ms)"6483lb # [ 122.044579] niks3-server[1111]: 2026/09/22 11:26:37 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6484worker1 # [ 122.032692] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44318 duration_s=0 paths=1 nar_bytes_in=4246485worker1 # [ 122.038941] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443186486worker1 # [ 122.050066] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6487worker1 # [ 122.055438] nix-daemon[847]: reaped child process 1519, status = succeeded6488worker1 # [ 122.059550] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6489client # [ 122.064256] nix[1332]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6490lb: must succeed: curl -sf localhost:9901/clusters6491lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6492lb: must succeed: curl -sf localhost:9901/clusters6493lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6494worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6495worker2 # [ 123.656749] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1523 (nix-grpc-daemon) on client request.6496lb # [ 123.689291] niks3-server[1111]: 2026/09/22 11:26:39 INFO lead: released remote=192.168.1.4:428246497client # [ 123.684266] nix[1332]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6498worker1 # [ 123.679517] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:38Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6499worker2 # [ 123.669950] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6500worker2 # [ 123.693166] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6501worker2 # [ 123.694633] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6502worker2 # [ 123.695860] systemd[1]: nix-grpc-daemon.service: Consumed 200ms CPU time over 14.922s wall clock time, 13.7M memory peak, 11.4K incoming IP traffic, 22K outgoing IP traffic.6503worker2 # [ 123.709948] systemd[1]: Starting scheduler-token.service...6504worker2 # [ 123.716349] nix-daemon[868]: reaped child process 1632, status = succeeded6505worker2 # [ 123.743353] systemd[1]: scheduler-token.service: Deactivated successfully.6506worker2 # [ 123.744963] systemd[1]: Finished scheduler-token.service.6507worker2 # [ 123.746876] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 36ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6508worker2 # [ 123.749088] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6509worker2 # [ 123.752467] systemd[1]: Starting scheduler-token.service...6510worker2 # [ 123.781749] systemd[1]: scheduler-token.service: Deactivated successfully.6511worker2 # [ 123.783447] systemd[1]: Finished scheduler-token.service.6512worker2 # [ 123.784790] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 30ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6513worker2 # [ 123.789248] systemd[1]: Starting Nix worker-protocol over gRPC...6514worker2 # [ 123.869765] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6515worker1 # [ 123.879837] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:39Z level=info event=scheduler_connected addr=lb:500516516worker2 # [ 123.877111] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6517worker2 # [ 123.881218] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16518worker2 # [ 123.883139] nix-grpc-daemon[1666]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6519worker2 # [ 123.889629] nix-grpc-daemon[1666]: E0000 00:00:1790076400.052644 1666 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6520worker2 # [ 123.894128] nix-grpc-daemon[1666]: E0000 00:00:1790076400.056890 1666 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6521lb # [ 123.911546] niks3-server[1111]: 2026/09/22 11:26:39 INFO lead: acquired remote=192.168.1.4:418306522worker2 # [ 123.898510] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6523worker2 # [ 123.901904] systemd[1]: Started Nix worker-protocol over gRPC.6524worker2 # [ 123.905954] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=scheduler_take_over6525worker2 # [ 123.908093] nix-grpc-daemon[1666]: E0000 00:00:1790076400.080135 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6526worker2 # [ 123.910234] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:490126527worker2 # [ 123.911616] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16528worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.29 seconds)6529client: must succeed: sleep 5.06530worker2 # [ 123.923845] nix-grpc-daemon[1666]: E0000 00:00:1790076400.100853 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6531worker2 # [ 123.927142] nix-grpc-daemon[1666]: E0000 00:00:1790076400.102992 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6532worker2 # [ 123.938055] nix-grpc-daemon[1666]: E0000 00:00:1790076400.115030 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6533worker2 # [ 123.945154] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=scheduler_connected addr=lb:500516534worker2 # [ 123.949406] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=oidc_jwks provider=mock keys=16535worker2 # [ 123.950719] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:490126536worker2 # [ 123.952333] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06537lb # [ 124.694892] envoy[896]: conn peer=[2001:db8:1::1]:59222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1384 ms=77606538lb # [ 124.698672] envoy[896]: conn peer=[2001:db8:1::3]:48650 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6173 tx=1376 ms=292956539lb # [ 124.702425] envoy[896]: conn peer=[2001:db8:1::4]:40784 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6953 tx=523 ms=147696540client: (finished: must succeed: sleep 5.0, in 5.02 seconds)6541client: waiting for success: ! systemctl is-active mid6542worker1 # [ 129.303621] nix-daemon[847]: reaped child process 1546, status = succeeded6543worker1 # [ 129.387723] nix-daemon[847]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6544worker1 # [ 129.400124] nix-daemon[847]: reaped child process 1586, status = succeeded6545lb # [ 129.415984] niks3-server[1111]: 2026/09/22 11:26:44 INFO Received uploads request method=POST path=/api/pending_closures6546worker1 # [ 129.409974] nix-grpc-daemon[1521]: time=2026-09-22T11:26:44.620Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6547worker1 # [ 129.413059] nix-grpc-daemon[1521]: time=2026-09-22T11:26:44.621Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6548lb # [ 129.431781] niks3-server[1111]: 2026/09/22 11:26:44 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6549lb # [ 129.439025] niks3-server[1111]: 2026/09/22 11:26:44 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6550lb # [ 129.445064] niks3-server[1111]: 2026/09/22 11:26:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6551lb # [ 129.447159] niks3-server[1111]: 2026/09/22 11:26:44 INFO Signed narinfos id=46 count=16552worker1 # [ 129.438862] nix-grpc-daemon[1521]: time=2026-09-22T11:26:44.648Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 129.450712] niks3-server[1111]: 2026/09/22 11:26:44 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6554lb # [ 129.456976] niks3-server[1111]: 2026/09/22 11:26:44 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6555lb # [ 129.463467] niks3-server[1111]: 2026/09/22 11:26:44 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6556lb # [ 129.472227] niks3-server[1111]: 2026/09/22 11:26:44 INFO Completed upload id=466557worker1 # [ 129.463312] nix-grpc-daemon[1521]: time=2026-09-22T11:26:44.673Z level=INFO msg="Upload complete. (190ms)"6558worker1 # [ 129.465925] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44318 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16559worker1 # [ 129.472947] nix-daemon[847]: reaped child process 1544, status = succeeded6560client # [ 129.502325] systemd[1]: mid.service: Deactivated successfully.6561client # [ 129.506307] systemd[1]: mid.service: Consumed 133ms CPU time over 9.410s wall clock time, 39.4M memory peak, 7.7K incoming IP traffic, 8.9K outgoing IP traffic.6562client: (finished: waiting for success: ! systemctl is-active mid, in 1.06 seconds)6563client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6564client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6565(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.16 seconds)6566subtest: interrupting the client stops the build on the worker6567client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6568client # [ 130.233905] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6569client # Running as unit: intr.service; invocation ID: a65d0ee84acb4d70a4bbe2f7c067cedc6570client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6571worker1 # [ 130.344797] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:443186572client # [ 130.353854] nix[1368]: don't know how to build these paths:6573client # [ 130.354798] nix[1368]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6574worker2 # [ 130.356908] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:46Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06575worker2 # [ 130.362619] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:46Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16576worker2 # [ 130.408668] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206577worker2 # [ 130.420924] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6578worker2 # [ 130.437565] nix-grpc-daemon[1666]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)6579worker2 # [ 130.544263] nix-grpc-daemon[1666]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 2/5)6580worker2 # [ 130.719723] nix-grpc-daemon[1666]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 3/5)6581worker2 # [ 130.776787] nix-grpc-daemon[1666]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 562 ms (attempt 4/5)6582worker2 # [ 131.341962] nix-grpc-daemon[1666]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6583worker2 # [ 131.344994] nix-grpc-daemon[1666]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6584worker2 # [ 131.362910] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206585client # [ 131.379594] nix[1368]: copying 1 paths to 'lb:50051'...6586client # [ 131.382871] nix[1368]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6587worker2 # [ 131.376569] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490206588worker2 # [ 131.384995] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6589worker2 # [ 131.412423] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:47Z level=info event=niks3_push_started pid=17436590worker2 # [ 131.524250] nix-grpc-daemon[1743]: time=2026-09-22T11:26:47.700Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6591worker2 # [ 131.619675] nix-daemon[868]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6592worker2 # [ 131.634188] nix-daemon[868]: reaped child process 1760, status = succeeded6593worker2 # [ 131.724263] nix-daemon[868]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6594worker2 # [ 131.738492] nix-daemon[868]: reaped child process 1765, status = succeeded6595lb # [ 131.759479] niks3-server[1111]: 2026/09/22 11:26:47 INFO Received uploads request method=POST path=/api/pending_closures6596worker2 # [ 131.758558] nix-grpc-daemon[1743]: time=2026-09-22T11:26:47.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6597worker2 # [ 131.761710] nix-grpc-daemon[1743]: time=2026-09-22T11:26:47.937Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6598lb # [ 131.799380] niks3-server[1111]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6599lb # [ 131.805493] niks3-server[1111]: 2026/09/22 11:26:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6600worker2 # [ 131.803750] nix-grpc-daemon[1743]: time=2026-09-22T11:26:47.980Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 131.815183] niks3-server[1111]: 2026/09/22 11:26:47 INFO Signed narinfos id=47 count=16602lb # [ 131.822342] niks3-server[1111]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6603lb # [ 131.834535] niks3-server[1111]: 2026/09/22 11:26:47 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6604lb # [ 131.840221] niks3-server[1111]: 2026/09/22 11:26:47 INFO Completed upload id=476605worker2 # [ 131.829492] nix-grpc-daemon[1743]: time=2026-09-22T11:26:48.006Z level=INFO msg="Upload complete. (301ms)"6606lb # [ 131.842778] niks3-server[1111]: 2026/09/22 11:26:47 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6607worker2 # [ 131.833738] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=4246608worker2 # [ 131.839290] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206609worker2 # [ 131.848859] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6610worker2 # [ 131.856177] nix-daemon[868]: reaped child process 1738, status = succeeded6611worker2 # [ 131.860265] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6612client # [ 131.866655] nix[1368]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6613client: must succeed: systemctl kill -s INT intr6614client # [ 133.646591] systemd[1]: intr.service: Sent signal SIGINT to main process 1368 (nix) on client request.6615worker2 # [ 133.647047] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:49Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6616client # [ 133.653661] nix[1368]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6617client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6618client # [ 133.659610] nix[1368]: [mutex.cc : 2506] RAW: thread should hold at least a read lock on Mutex 0x7b3c7fdfcc686619client # [ 133.670758] systemd-coredump[1382]: Process 1368 (nix) of user 0 terminated abnormally with signal 6/ABRT, processing...6620client # [ 133.685198] systemd[1]: Created slice Slice /system/systemd-coredump.6621client # [ 133.688470] systemd[1]: Started Process Core Dump (PID 1382/UID 0).6622worker2 # [ 133.681680] nix-daemon[868]: reaped child process 1767, status = succeeded6623worker2 # [ 133.684246] nix-daemon[1769]: unexpected Nix daemon error: error: interrupted by the user6624worker2 # [ 133.692658] nix-daemon[868]: reaped child process 1769, status = failed with exit code 16625(finished: subtest: interrupting the client stops the build on the worker, in 3.50 seconds)6626subtest: a deploy mid-build drains: the build finishes, the new generation takes over6627client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6628client # [ 133.751953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6629client # Running as unit: sw.service; invocation ID: 4517aa6e61b441c28c0bd13c2b903e0a6630client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.05 seconds)6631worker2 # [ 134.023362] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490206632client # [ 134.038351] nix[1389]: don't know how to build these paths:6633client # [ 134.039265] nix[1389]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6634worker2 # [ 134.046147] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:50Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06635worker2 # [ 134.050404] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:50Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16636worker1 # [ 134.092726] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443186637worker1 # [ 134.101625] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:443186638client # [ 134.114082] nix[1389]: copying 1 paths to 'lb:50051'...6639worker1 # [ 134.108765] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:443186640client # [ 134.122095] nix[1389]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6641worker1 # [ 134.129719] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6642worker1 # [ 134.214202] nix-daemon[847]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6643worker1 # [ 134.229340] nix-daemon[847]: reaped child process 1617, status = succeeded6644worker1 # [ 134.308484] nix-daemon[847]: accepted connection from pid 1619, user nix-grpc-daemon (trusted)6645worker1 # [ 134.321901] nix-daemon[847]: reaped child process 1622, status = succeeded6646lb # [ 134.339738] niks3-server[1111]: 2026/09/22 11:26:49 INFO Received uploads request method=POST path=/api/pending_closures6647worker1 # [ 134.338388] nix-grpc-daemon[1521]: time=2026-09-22T11:26:49.548Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6648worker1 # [ 134.339832] nix-grpc-daemon[1521]: time=2026-09-22T11:26:49.548Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6649lb # [ 134.370439] niks3-server[1111]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6650lb # [ 134.380449] niks3-server[1111]: 2026/09/22 11:26:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6651lb # [ 134.386958] niks3-server[1111]: 2026/09/22 11:26:49 INFO Signed narinfos id=48 count=16652worker1 # [ 134.380311] nix-grpc-daemon[1521]: time=2026-09-22T11:26:49.590Z level=INFO msg="Uploading 1 narinfos"6653lb # [ 134.394664] niks3-server[1111]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6654lb # [ 134.408945] niks3-server[1111]: 2026/09/22 11:26:49 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6655lb # [ 134.413130] niks3-server[1111]: 2026/09/22 11:26:49 INFO Completed upload id=486656worker1 # [ 134.403075] nix-grpc-daemon[1521]: time=2026-09-22T11:26:49.613Z level=INFO msg="Upload complete. (268ms)"6657lb # [ 134.414966] niks3-server[1111]: 2026/09/22 11:26:49 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6658worker1 # [ 134.406946] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44318 duration_s=0 paths=1 nar_bytes_in=4166659worker1 # [ 134.412269] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:443186660worker1 # [ 134.423184] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6661worker1 # [ 134.426729] nix-daemon[847]: reaped child process 1612, status = succeeded6662client # [ 134.436044] nix[1389]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6663worker1 # [ 134.431698] nix-daemon[847]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6664lb # [ 134.695067] envoy[896]: conn peer=[2001:db8:1::1]:60838 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3074 tx=1519 ms=92606665worker1: must succeed: systemctl reload nix-grpc-daemon.service6666worker1 # [ 135.030486] systemd[1]: Reloading Nix worker-protocol over gRPC...6667worker1 # [ 135.058760] systemd[1]: Reloaded Nix worker-protocol over gRPC.6668worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6669worker1: must succeed: pgrep -f 'read -t [9]0 x'6670worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6671client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26672worker1 # [ 135.161152] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:50Z level=info event=draining6673worker2 # [ 135.397831] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490206674client # don't know how to build these paths:6675client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6676worker2 # [ 135.404232] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206677worker2 # [ 135.420743] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06678worker2 # [ 135.423002] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16679worker2 # [ 135.469618] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206680worker2 # [ 135.477674] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206681client # copying 1 paths to 'lb:50051'...6682worker2 # [ 135.485815] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490206683client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6684worker2 # [ 135.502861] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6685worker2 # [ 135.606378] nix-daemon[868]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6686worker2 # [ 135.620517] nix-daemon[868]: reaped child process 1819, status = succeeded6687worker2 # [ 135.703535] nix-daemon[868]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6688worker2 # [ 135.716588] nix-daemon[868]: reaped child process 1824, status = succeeded6689lb # [ 135.737073] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received uploads request method=POST path=/api/pending_closures6690worker2 # [ 135.734695] nix-grpc-daemon[1743]: time=2026-09-22T11:26:51.911Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6691worker2 # [ 135.736254] nix-grpc-daemon[1743]: time=2026-09-22T11:26:51.911Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6692lb # [ 135.767606] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6693lb # [ 135.774172] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6694worker2 # [ 135.769438] nix-grpc-daemon[1743]: time=2026-09-22T11:26:51.946Z level=INFO msg="Uploading 1 narinfos"6695lb # [ 135.781404] niks3-server[1111]: 2026/09/22 11:26:51 INFO Signed narinfos id=49 count=16696lb # [ 135.789559] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6697lb # [ 135.794408] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6698lb # [ 135.806247] niks3-server[1111]: 2026/09/22 11:26:51 INFO Completed upload id=496699worker2 # [ 135.793873] nix-grpc-daemon[1743]: time=2026-09-22T11:26:51.970Z level=INFO msg="Upload complete. (279ms)"6700client # [ 135.802499] systemd-coredump[1383]: Cannot store coredump of 1368 (nix): No space left on device6701lb # [ 135.807954] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6702worker2 # [ 135.795984] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=5286703worker2 # [ 135.802129] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206704worker2 # [ 135.809743] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6705worker2 # [ 135.816190] nix-daemon[868]: reaped child process 1814, status = succeeded6706client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6707client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6708worker2 # [ 135.820530] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6709client # [ 135.886568] systemd-coredump[1383]: Process 1368 (nix) of user 0 terminated abnormally without generating a coredump.6710client # [ 135.900692] systemd[1]: systemd-coredump@0-1-1382_1383-0.service: Deactivated successfully.6711client # [ 135.907797] systemd[1]: systemd-coredump@0-1-1382_1383-0.service: Consumed 1.572s CPU time over 2.206s wall clock time, 967.8M memory peak, 929.8M written to disk.6712client # [ 135.909858] systemd[1]: intr.service: Main process exited, code=dumped, status=6/ABRT6713client # [ 135.911346] systemd[1]: intr.service: Failed with result 'core-dump'.6714client # [ 135.912876] systemd[1]: intr.service: Consumed 264ms CPU time over 5.673s wall clock time, 41.6M memory peak, 4K written to disk, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6715client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6716client # farm-a> LOG-farm-a-during-drain6717worker2 # [ 136.253788] nix-daemon[868]: reaped child process 1828, status = succeeded6718worker2 # [ 136.337871] nix-daemon[868]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6719worker2 # [ 136.349796] nix-daemon[868]: reaped child process 1847, status = succeeded6720lb # [ 136.368895] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received uploads request method=POST path=/api/pending_closures6721worker2 # [ 136.365501] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.542Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6722worker2 # [ 136.367071] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.544Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6723lb # [ 136.404436] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6724lb # [ 136.419450] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6725lb # [ 136.425527] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6726worker2 # [ 136.418077] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.595Z level=INFO msg="Uploading 1 narinfos"6727lb # [ 136.430266] niks3-server[1111]: 2026/09/22 11:26:51 INFO Signed narinfos id=50 count=16728lb # [ 136.435489] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6729lb # [ 136.441472] niks3-server[1111]: 2026/09/22 11:26:51 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6730lb # [ 136.445302] niks3-server[1111]: 2026/09/22 11:26:51 INFO Completed upload id=506731worker2 # [ 136.433334] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.610Z level=INFO msg="Upload complete. (206ms)"6732lb # [ 136.447648] niks3-server[1111]: 2026/09/22 11:26:51 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6733worker2 # [ 136.438743] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16734worker2 # [ 136.447866] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06735worker2 # [ 136.453222] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16736worker2 # [ 136.460467] nix-daemon[868]: reaped child process 1826, status = succeeded6737worker2 # [ 136.491811] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206738worker2 # [ 136.502886] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206739worker2 # [ 136.513597] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.690Z level=INFO msg="All 1 paths already cached"6740client # copying 1 paths to 'lb:50051'...6741client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6742worker2 # [ 136.520335] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490206743worker2 # [ 136.531274] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6744worker2 # [ 136.623545] nix-daemon[868]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6745worker2 # [ 136.636728] nix-daemon[868]: reaped child process 1856, status = succeeded6746worker2 # [ 136.720419] nix-daemon[868]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)6747worker2 # [ 136.733201] nix-daemon[868]: reaped child process 1861, status = succeeded6748lb # [ 136.752680] niks3-server[1111]: 2026/09/22 11:26:52 INFO Received uploads request method=POST path=/api/pending_closures6749worker2 # [ 136.745241] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.922Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6750worker2 # [ 136.747285] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.923Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6751lb # [ 136.769404] niks3-server[1111]: 2026/09/22 11:26:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6752lb # [ 136.772612] niks3-server[1111]: 2026/09/22 11:26:52 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6753lb # [ 136.774615] niks3-server[1111]: 2026/09/22 11:26:52 INFO Signed narinfos id=51 count=16754worker2 # [ 136.762659] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.939Z level=INFO msg="Uploading 1 narinfos"6755lb # [ 136.785780] niks3-server[1111]: 2026/09/22 11:26:52 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6756lb # [ 136.793772] niks3-server[1111]: 2026/09/22 11:26:52 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6757lb # [ 136.796185] niks3-server[1111]: 2026/09/22 11:26:52 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6758lb # [ 136.798951] niks3-server[1111]: 2026/09/22 11:26:52 INFO Completed upload id=516759worker2 # [ 136.786742] nix-grpc-daemon[1743]: time=2026-09-22T11:26:52.963Z level=INFO msg="Upload complete. (246ms)"6760worker2 # [ 136.789074] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=6406761worker2 # [ 136.791763] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206762worker2 # [ 136.798842] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6763worker2 # [ 136.801164] nix-daemon[868]: reaped child process 1851, status = succeeded6764client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6765client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6766worker2 # [ 136.806580] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6767client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6768client # farm-b> LOG-farm-b-during-drain6769worker2 # [ 137.622491] nix-daemon[868]: reaped child process 1865, status = succeeded6770worker2 # [ 137.697580] nix-daemon[868]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6771worker2 # [ 137.709348] nix-daemon[868]: reaped child process 1884, status = succeeded6772lb # [ 137.729059] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received uploads request method=POST path=/api/pending_closures6773worker2 # [ 137.725196] nix-grpc-daemon[1743]: time=2026-09-22T11:26:53.902Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6774worker2 # [ 137.726708] nix-grpc-daemon[1743]: time=2026-09-22T11:26:53.903Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6775lb # [ 137.759229] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6776lb # [ 137.776431] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6777lb # [ 137.791446] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6778lb # [ 137.795325] niks3-server[1111]: 2026/09/22 11:26:53 INFO Signed narinfos id=52 count=16779worker2 # [ 137.785907] nix-grpc-daemon[1743]: time=2026-09-22T11:26:53.962Z level=INFO msg="Uploading 1 narinfos"6780lb # [ 137.800431] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6781lb # [ 137.811248] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6782lb # [ 137.815271] niks3-server[1111]: 2026/09/22 11:26:53 INFO Completed upload id=526783worker2 # [ 137.803169] nix-grpc-daemon[1743]: time=2026-09-22T11:26:53.980Z level=INFO msg="Upload complete. (189ms)"6784lb # [ 137.818370] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6785worker2 # [ 137.811779] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16786worker2 # [ 137.821463] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:53Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06787worker2 # [ 137.825203] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:53Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16788worker2 # [ 137.834759] nix-daemon[868]: reaped child process 1863, status = succeeded6789worker2 # [ 137.864278] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206790worker2 # [ 137.876694] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206791worker2 # [ 137.889717] nix-grpc-daemon[1743]: time=2026-09-22T11:26:54.066Z level=INFO msg="All 2 paths already cached"6792client # copying 1 paths to 'lb:50051'...6793client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6794worker2 # [ 137.898291] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490206795worker2 # [ 137.906552] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6796worker2 # [ 138.002959] nix-daemon[868]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6797worker2 # [ 138.016734] nix-daemon[868]: reaped child process 1893, status = succeeded6798worker2 # [ 138.097377] nix-daemon[868]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6799worker2 # [ 138.110454] nix-daemon[868]: reaped child process 1898, status = succeeded6800lb # [ 138.129122] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received uploads request method=POST path=/api/pending_closures6801worker2 # [ 138.124372] nix-grpc-daemon[1743]: time=2026-09-22T11:26:54.301Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6802worker2 # [ 138.125875] nix-grpc-daemon[1743]: time=2026-09-22T11:26:54.302Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6803lb # [ 138.159573] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6804lb # [ 138.167270] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6805lb # [ 138.172859] niks3-server[1111]: 2026/09/22 11:26:53 INFO Signed narinfos id=53 count=16806worker2 # [ 138.164147] nix-grpc-daemon[1743]: time=2026-09-22T11:26:54.340Z level=INFO msg="Uploading 1 narinfos"6807lb # [ 138.189081] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6808lb # [ 138.193107] niks3-server[1111]: 2026/09/22 11:26:53 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6809lb # [ 138.200316] niks3-server[1111]: 2026/09/22 11:26:53 INFO Completed upload id=536810worker2 # [ 138.187888] nix-grpc-daemon[1743]: time=2026-09-22T11:26:54.364Z level=INFO msg="Upload complete. (268ms)"6811lb # [ 138.204304] niks3-server[1111]: 2026/09/22 11:26:53 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6812worker2 # [ 138.191478] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=7686813worker2 # [ 138.197252] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490206814worker2 # [ 138.207590] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6815worker2 # [ 138.213099] nix-daemon[868]: reaped child process 1888, status = succeeded6816worker2 # [ 138.216058] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6817client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6818client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6819client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6820client # farm-top> LOG-farm-top-during-drain6821worker2 # [ 139.234194] nix-daemon[868]: reaped child process 1902, status = succeeded6822worker2 # [ 139.306449] nix-daemon[868]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6823worker2 # [ 139.319362] nix-daemon[868]: reaped child process 1921, status = succeeded6824lb # [ 139.338314] niks3-server[1111]: 2026/09/22 11:26:54 INFO Received uploads request method=POST path=/api/pending_closures6825worker2 # [ 139.334272] nix-grpc-daemon[1743]: time=2026-09-22T11:26:55.511Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6826worker2 # [ 139.335805] nix-grpc-daemon[1743]: time=2026-09-22T11:26:55.512Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6827lb # [ 139.372153] niks3-server[1111]: 2026/09/22 11:26:54 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6828lb # [ 139.383675] niks3-server[1111]: 2026/09/22 11:26:54 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6829lb # [ 139.396211] niks3-server[1111]: 2026/09/22 11:26:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6830lb # [ 139.400273] niks3-server[1111]: 2026/09/22 11:26:54 INFO Signed narinfos id=54 count=16831worker2 # [ 139.388477] nix-grpc-daemon[1743]: time=2026-09-22T11:26:55.565Z level=INFO msg="Uploading 1 narinfos"6832lb # [ 139.408471] niks3-server[1111]: 2026/09/22 11:26:54 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6833lb # [ 139.416073] niks3-server[1111]: 2026/09/22 11:26:54 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6834lb # [ 139.420055] niks3-server[1111]: 2026/09/22 11:26:54 INFO Completed upload id=546835worker2 # [ 139.408079] nix-grpc-daemon[1743]: time=2026-09-22T11:26:55.585Z level=INFO msg="Upload complete. (184ms)"6836lb # [ 139.422565] niks3-server[1111]: 2026/09/22 11:26:54 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6837worker2 # [ 139.415898] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16838worker2 # [ 139.428423] nix-daemon[868]: reaped child process 1900, status = succeeded6839client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.35 seconds)6840client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6841client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6842worker1: must succeed: pkill -f 'read -t [9]0 x'6843worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6844client: waiting for success: ! systemctl is-active sw6845worker1 # [ 139.652565] nix-daemon[847]: reaped child process 1626, status = succeeded6846worker1 # [ 139.721947] nix-daemon[847]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)6847worker1 # [ 139.735537] nix-daemon[847]: reaped child process 1666, status = succeeded6848lb # [ 139.752320] niks3-server[1111]: 2026/09/22 11:26:55 INFO Received uploads request method=POST path=/api/pending_closures6849worker1 # [ 139.750157] nix-grpc-daemon[1521]: time=2026-09-22T11:26:54.960Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6850worker1 # [ 139.751710] nix-grpc-daemon[1521]: time=2026-09-22T11:26:54.962Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6851lb # [ 139.794225] niks3-server[1111]: 2026/09/22 11:26:55 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6852lb # [ 139.804646] niks3-server[1111]: 2026/09/22 11:26:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6853lb # [ 139.811344] niks3-server[1111]: 2026/09/22 11:26:55 INFO Signed narinfos id=55 count=16854worker1 # [ 139.804814] nix-grpc-daemon[1521]: time=2026-09-22T11:26:55.014Z level=INFO msg="Uploading 1 narinfos"6855lb # [ 139.817054] niks3-server[1111]: 2026/09/22 11:26:55 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6856lb # [ 139.820626] niks3-server[1111]: 2026/09/22 11:26:55 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6857lb # [ 139.831758] niks3-server[1111]: 2026/09/22 11:26:55 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6858lb # [ 139.834442] niks3-server[1111]: 2026/09/22 11:26:55 INFO Completed upload id=556859worker1 # [ 139.824307] nix-grpc-daemon[1521]: time=2026-09-22T11:26:55.034Z level=INFO msg="Upload complete. (182ms)"6860lb # [ 139.836284] niks3-server[1111]: 2026/09/22 11:26:55 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6861worker1 # [ 139.826965] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44318 duration_s=5 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16862worker1 # [ 139.834826] nix-daemon[847]: reaped child process 1624, status = succeeded6863client # [ 139.864559] systemd[1]: sw.service: Deactivated successfully.6864client # [ 139.868070] systemd[1]: sw.service: Consumed 153ms CPU time over 6.115s wall clock time, 41.5M memory peak, 4K written to disk, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6865worker1 # [ 140.163139] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:55Z level=info event=signal_exit6866client: (finished: waiting for success: ! systemctl is-active sw, in 1.04 seconds)6867client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6868client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.02 seconds)6869worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6870worker2 # [ 141.171274] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:490126871worker2 # [ 141.175171] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:57Z level=info event=worker_gone addr=192.168.1.3:500516872worker1 # [ 141.177508] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:56Z level=info event=scheduler_restarting6873worker1 # [ 141.186581] nix-grpc-daemon[1453]: ts=2026-09-22T11:26:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6874worker1 # [ 141.209246] nix-daemon[847]: reaped child process 1509, status = succeeded6875worker1 # [ 141.396616] nix-grpc-daemon[1453]: E0000 00:00:1790076416.606776 1498 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6876worker1 # [ 141.404342] nix-grpc-daemon[1453]: E0000 00:00:1790076416.614634 1498 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6877worker1 # [ 143.179781] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6878worker1 # [ 143.181528] systemd[1]: nix-grpc-daemon.service: Consumed 753ms CPU time over 24.138s wall clock time, 36.8M memory peak, 24.7K incoming IP traffic, 34.4K outgoing IP traffic.6879worker1 # [ 143.188429] systemd[1]: Starting Nix worker-protocol over gRPC...6880worker1 # [ 143.267739] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6881worker1 # [ 143.272116] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6882worker1 # [ 143.276970] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16883worker1 # [ 143.281102] nix-grpc-daemon[1680]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6884worker1 # [ 143.285433] nix-grpc-daemon[1680]: E0000 00:00:1790076418.485101 1680 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6885worker1 # [ 143.288659] nix-grpc-daemon[1680]: E0000 00:00:1790076418.487135 1680 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6886worker1 # [ 143.291583] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6887worker1 # [ 143.295144] systemd[1]: Started Nix worker-protocol over gRPC.6888worker1 # [ 143.296034] nix-grpc-daemon[1680]: E0000 00:00:1790076418.503881 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6889worker1 # [ 143.310070] nix-grpc-daemon[1680]: E0000 00:00:1790076418.518045 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6890worker1 # [ 143.313138] nix-grpc-daemon[1680]: E0000 00:00:1790076418.518138 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6891worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.67 seconds)6892lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6893lb: must succeed: curl -sf localhost:9901/clusters6894worker1 # [ 143.323322] nix-grpc-daemon[1680]: E0000 00:00:1790076418.533671 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6895worker1 # [ 143.336408] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:58Z level=info event=scheduler_connected addr=lb:500516896worker2 # [ 143.337671] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:490126897worker2 # [ 143.341641] nix-grpc-daemon[1666]: ts=2026-09-22T11:26:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6899lb: must succeed: curl -sf localhost:9901/clusters6900lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6901lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.08 seconds)6902lb: must succeed: curl -sf localhost:9901/clusters6903lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6904worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6905worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6906(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 10.76 seconds)6907subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6908client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16909worker2 # [ 144.608861] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490206910worker2 # [ 144.615292] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490206911worker2 # [ 144.653241] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06912worker2 # [ 144.655222] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16913worker2 # [ 144.657039] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06914worker2 # [ 144.658605] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:00Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06915lb # [ 144.695487] envoy[896]: conn peer=[2001:db8:1::1]:52794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554536:system_library:OPENSSL_internal:Connection_reset_by_peer:TLS_error_end" rx=2993 tx=914 ms=55506916lb # [ 144.701370] envoy[896]: conn peer=[2001:db8:1::1]:45092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9512 tx=4027 ms=40406917lb # [ 144.704956] envoy[896]: conn peer=[2001:db8:1::1]:45080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1402 ms=58256918lb # [ 144.708586] envoy[896]: conn peer=[2001:db8:1::3]:37894 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1762 tx=503 ms=219816919worker1 # [ 144.710396] nix-grpc-daemon[1680]: ts=2026-09-22T11:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428306920worker1 # [ 144.721677] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6921worker1 # [ 144.737886] nix-grpc-daemon[1680]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6922worker1 # [ 144.795651] nix-grpc-daemon[1680]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 2/5)6923worker1 # [ 144.842468] nix-grpc-daemon[1680]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 355 ms (attempt 3/5)6924worker1 # [ 145.199593] nix-grpc-daemon[1680]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 787 ms (attempt 4/5)6925worker1 # [ 145.989905] nix-grpc-daemon[1680]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6926worker1 # [ 145.993941] nix-grpc-daemon[1680]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6927worker1 # [ 146.011124] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428306928worker1 # [ 146.024331] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428306929worker1 # [ 146.035666] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6930worker1 # [ 146.054923] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:01Z level=info event=niks3_push_started pid=17406931worker1 # [ 146.155493] nix-grpc-daemon[1740]: time=2026-09-22T11:27:01.365Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6932worker1 # [ 146.244150] nix-daemon[847]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)6933worker1 # [ 146.257541] nix-daemon[847]: reaped child process 1756, status = succeeded6934worker1 # [ 146.338132] nix-daemon[847]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6935worker1 # [ 146.351903] nix-daemon[847]: reaped child process 1761, status = succeeded6936lb # [ 146.368900] niks3-server[1111]: 2026/09/22 11:27:01 INFO Received uploads request method=POST path=/api/pending_closures6937worker1 # [ 146.367769] nix-grpc-daemon[1740]: time=2026-09-22T11:27:01.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6938worker1 # [ 146.369153] nix-grpc-daemon[1740]: time=2026-09-22T11:27:01.577Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6939lb # [ 146.404176] niks3-server[1111]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6940lb # [ 146.410728] niks3-server[1111]: 2026/09/22 11:27:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6941worker1 # [ 146.406719] nix-grpc-daemon[1740]: time=2026-09-22T11:27:01.616Z level=INFO msg="Uploading 1 narinfos"6942lb # [ 146.416687] niks3-server[1111]: 2026/09/22 11:27:01 INFO Signed narinfos id=56 count=16943lb # [ 146.431089] niks3-server[1111]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6944lb # [ 146.433755] niks3-server[1111]: 2026/09/22 11:27:01 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6945lb # [ 146.438288] niks3-server[1111]: 2026/09/22 11:27:01 INFO Completed upload id=566946worker1 # [ 146.427670] nix-grpc-daemon[1740]: time=2026-09-22T11:27:01.638Z level=INFO msg="Upload complete. (268ms)"6947lb # [ 146.440238] niks3-server[1111]: 2026/09/22 11:27:01 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6948worker1 # [ 146.429770] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=5046949worker1 # [ 146.434490] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428306950worker1 # [ 146.441245] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6951worker1 # [ 146.442612] nix-daemon[847]: reaped child process 1738, status = succeeded6952worker1 # [ 146.447906] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6953worker1 # [ 147.693664] nix-daemon[847]: reaped child process 1765, status = succeeded6954worker1 # [ 147.768733] nix-daemon[847]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6955worker1 # [ 147.781846] nix-daemon[847]: reaped child process 1784, status = succeeded6956lb # [ 147.799827] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received uploads request method=POST path=/api/pending_closures6957worker1 # [ 147.797584] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6958worker1 # [ 147.799386] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.009Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6959lb # [ 147.831266] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6960lb # [ 147.846109] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6961lb # [ 147.856621] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6962worker1 # [ 147.853865] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.063Z level=INFO msg="Uploading 1 narinfos"6963lb # [ 147.863655] niks3-server[1111]: 2026/09/22 11:27:03 INFO Signed narinfos id=57 count=16964lb # [ 147.876324] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6965lb # [ 147.879751] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6966lb # [ 147.882927] niks3-server[1111]: 2026/09/22 11:27:03 INFO Completed upload id=576967worker1 # [ 147.873921] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.084Z level=INFO msg="Upload complete. (216ms)"6968lb # [ 147.885212] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6969worker1 # [ 147.877338] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=6 outputs=16970worker1 # [ 147.884764] nix-daemon[847]: reaped child process 1763, status = succeeded6971worker2 # [ 147.877837] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:04Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16972worker2 # [ 147.886866] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:04Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06973worker1 # [ 147.922553] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428306974worker1 # [ 147.930131] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428306975worker1 # [ 147.936786] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428306976worker1 # [ 147.941851] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6977worker1 # [ 148.029134] nix-daemon[847]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6978worker1 # [ 148.044054] nix-daemon[847]: reaped child process 1791, status = succeeded6979worker1 # [ 148.126266] nix-daemon[847]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6980worker1 # [ 148.140665] nix-daemon[847]: reaped child process 1796, status = succeeded6981lb # [ 148.156644] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received uploads request method=POST path=/api/pending_closures6982worker1 # [ 148.154249] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.364Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6983worker1 # [ 148.155689] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.364Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6984lb # [ 148.185550] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6985lb # [ 148.192122] niks3-server[1111]: 2026/09/22 11:27:03 INFO Signed narinfos id=58 count=16986worker1 # [ 148.185054] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.395Z level=INFO msg="Uploading 1 narinfos"6987lb # [ 148.200601] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6988lb # [ 148.207569] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6989lb # [ 148.219488] niks3-server[1111]: 2026/09/22 11:27:03 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6990worker1 # [ 148.213399] nix-grpc-daemon[1740]: time=2026-09-22T11:27:03.423Z level=INFO msg="Upload complete. (266ms)"6991lb # [ 148.223621] niks3-server[1111]: 2026/09/22 11:27:03 INFO Completed upload id=586992worker1 # [ 148.216075] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=5046993worker1 # [ 148.219133] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428306994lb # [ 148.231242] niks3-server[1111]: 2026/09/22 11:27:03 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6995worker1 # [ 148.226258] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6996worker1 # [ 148.229161] nix-daemon[847]: reaped child process 1786, status = succeeded6997worker1 # [ 148.234199] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6998worker1 # [ 149.452154] nix-daemon[847]: reaped child process 1800, status = succeeded6999worker1 # [ 149.525217] nix-daemon[847]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)7000worker1 # [ 149.539067] nix-daemon[847]: reaped child process 1819, status = succeeded7001lb # [ 149.555922] niks3-server[1111]: 2026/09/22 11:27:04 INFO Received uploads request method=POST path=/api/pending_closures7002worker1 # [ 149.552628] nix-grpc-daemon[1740]: time=2026-09-22T11:27:04.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7003worker1 # [ 149.554153] nix-grpc-daemon[1740]: time=2026-09-22T11:27:04.764Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7004lb # [ 149.586140] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7005lb # [ 149.595894] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7006lb # [ 149.606861] niks3-server[1111]: 2026/09/22 11:27:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign7007worker1 # [ 149.600898] nix-grpc-daemon[1740]: time=2026-09-22T11:27:04.811Z level=INFO msg="Uploading 1 narinfos"7008lb # [ 149.611444] niks3-server[1111]: 2026/09/22 11:27:05 INFO Signed narinfos id=59 count=17009lb # [ 149.616903] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7010lb # [ 149.622879] niks3-server[1111]: 2026/09/22 11:27:05 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete7011lb # [ 149.625566] niks3-server[1111]: 2026/09/22 11:27:05 INFO Completed upload id=597012worker1 # [ 149.615297] nix-grpc-daemon[1740]: time=2026-09-22T11:27:04.825Z level=INFO msg="Upload complete. (174ms)"7013worker1 # [ 149.619064] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=7 outputs=17014worker2 # [ 149.617780] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:05Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17015worker1 # [ 149.625113] nix-daemon[847]: reaped child process 1798, status = succeeded7016worker2 # [ 149.627374] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:05Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07017worker1 # [ 149.668936] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307018worker1 # [ 149.677087] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307019worker1 # [ 149.684679] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307020worker1 # [ 149.688453] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7021worker1 # [ 149.776324] nix-daemon[847]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)7022worker1 # [ 149.789645] nix-daemon[847]: reaped child process 1826, status = succeeded7023worker1 # [ 149.875194] nix-daemon[847]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)7024worker1 # [ 149.889502] nix-daemon[847]: reaped child process 1831, status = succeeded7025lb # [ 149.905656] niks3-server[1111]: 2026/09/22 11:27:05 INFO Received uploads request method=POST path=/api/pending_closures7026worker1 # [ 149.903620] nix-grpc-daemon[1740]: time=2026-09-22T11:27:05.113Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7027worker1 # [ 149.904975] nix-grpc-daemon[1740]: time=2026-09-22T11:27:05.113Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7028lb # [ 149.931528] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7029lb # [ 149.942105] niks3-server[1111]: 2026/09/22 11:27:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7030lb # [ 149.947680] niks3-server[1111]: 2026/09/22 11:27:05 INFO Signed narinfos id=60 count=17031worker1 # [ 149.940528] nix-grpc-daemon[1740]: time=2026-09-22T11:27:05.150Z level=INFO msg="Uploading 1 narinfos"7032lb # [ 149.957107] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7033lb # [ 149.968066] niks3-server[1111]: 2026/09/22 11:27:05 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7034worker1 # [ 149.965078] nix-grpc-daemon[1740]: time=2026-09-22T11:27:05.175Z level=INFO msg="Upload complete. (270ms)"7035lb # [ 149.975445] niks3-server[1111]: 2026/09/22 11:27:05 INFO Completed upload id=607036worker1 # [ 149.967109] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=5047037lb # [ 149.978481] niks3-server[1111]: 2026/09/22 11:27:05 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7038worker1 # [ 149.970772] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307039worker1 # [ 149.977755] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7040worker1 # [ 149.980132] nix-daemon[847]: reaped child process 1821, status = succeeded7041worker1 # [ 149.985257] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7042worker1 # [ 151.024597] nix-daemon[847]: reaped child process 1835, status = succeeded7043worker1 # [ 151.110081] nix-daemon[847]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)7044worker1 # [ 151.123947] nix-daemon[847]: reaped child process 1854, status = succeeded7045lb # [ 151.143506] niks3-server[1111]: 2026/09/22 11:27:06 INFO Received uploads request method=POST path=/api/pending_closures7046worker1 # [ 151.142548] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.352Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7047worker1 # [ 151.144614] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.354Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7048lb # [ 151.180898] niks3-server[1111]: 2026/09/22 11:27:06 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7049lb # [ 151.194708] niks3-server[1111]: 2026/09/22 11:27:06 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7050lb # [ 151.206582] niks3-server[1111]: 2026/09/22 11:27:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7051worker1 # [ 151.202474] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.412Z level=INFO msg="Uploading 1 narinfos"7052lb # [ 151.212971] niks3-server[1111]: 2026/09/22 11:27:06 INFO Signed narinfos id=61 count=17053lb # [ 151.218097] niks3-server[1111]: 2026/09/22 11:27:06 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7054lb # [ 151.227759] niks3-server[1111]: 2026/09/22 11:27:06 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7055lb # [ 151.232588] niks3-server[1111]: 2026/09/22 11:27:06 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7056lb # [ 151.235769] niks3-server[1111]: 2026/09/22 11:27:06 INFO Completed upload id=617057worker1 # [ 151.225673] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.436Z level=INFO msg="Upload complete. (233ms)"7058worker1 # [ 151.228471] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=8 outputs=17059worker2 # [ 151.228378] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:07Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17060worker1 # [ 151.237112] nix-daemon[847]: reaped child process 1833, status = succeeded7061worker2 # [ 151.239095] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:07Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07062worker1 # [ 151.280237] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307063worker1 # [ 151.288728] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307064worker1 # [ 151.297559] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.507Z level=INFO msg="All 1 paths already cached"7065worker1 # [ 151.301957] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307066worker1 # [ 151.305727] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7067worker1 # [ 151.392598] nix-daemon[847]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)7068worker1 # [ 151.408662] nix-daemon[847]: reaped child process 1861, status = succeeded7069worker1 # [ 151.486109] nix-daemon[847]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)7070worker1 # [ 151.500595] nix-daemon[847]: reaped child process 1866, status = succeeded7071lb # [ 151.516131] niks3-server[1111]: 2026/09/22 11:27:06 INFO Received uploads request method=POST path=/api/pending_closures7072worker1 # [ 151.513841] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.723Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7073worker1 # [ 151.515164] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.724Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7074lb # [ 151.542919] niks3-server[1111]: 2026/09/22 11:27:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7075worker1 # [ 151.539972] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.750Z level=INFO msg="Uploading 1 narinfos"7076lb # [ 151.550413] niks3-server[1111]: 2026/09/22 11:27:06 INFO Signed narinfos id=62 count=17077lb # [ 151.558713] niks3-server[1111]: 2026/09/22 11:27:06 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7078lb # [ 151.571373] niks3-server[1111]: 2026/09/22 11:27:07 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7079worker1 # [ 151.570095] nix-grpc-daemon[1740]: time=2026-09-22T11:27:06.780Z level=INFO msg="Upload complete. (258ms)"7080lb # [ 151.580601] niks3-server[1111]: 2026/09/22 11:27:07 INFO Completed upload id=627081worker1 # [ 151.572228] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=6247082lb # [ 151.584178] niks3-server[1111]: 2026/09/22 11:27:07 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7083worker1 # [ 151.578573] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307084lb # [ 151.588797] niks3-server[1111]: 2026/09/22 11:27:07 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7085worker1 # [ 151.583206] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7086worker1 # [ 151.585587] nix-daemon[847]: reaped child process 1856, status = succeeded7087worker1 # [ 151.590875] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7088worker1 # [ 152.830934] nix-daemon[847]: reaped child process 1870, status = succeeded7089worker1 # [ 152.910860] nix-daemon[847]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)7090worker1 # [ 152.923080] nix-daemon[847]: reaped child process 1889, status = succeeded7091lb # [ 152.939478] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received uploads request method=POST path=/api/pending_closures7092worker1 # [ 152.939473] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.149Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7093worker1 # [ 152.944062] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.151Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7094lb # [ 152.976330] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7095lb # [ 152.995303] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7096lb # [ 153.000780] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7097lb # [ 153.005389] niks3-server[1111]: 2026/09/22 11:27:08 INFO Signed narinfos id=63 count=17098worker1 # [ 152.995120] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.205Z level=INFO msg="Uploading 1 narinfos"7099lb # [ 153.013202] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7100lb # [ 153.021792] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7101lb # [ 153.024326] niks3-server[1111]: 2026/09/22 11:27:08 INFO Completed upload id=637102lb # [ 153.026135] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7103worker1 # [ 153.015479] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.225Z level=INFO msg="Upload complete. (217ms)"7104worker2 # [ 153.021261] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:09Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17105worker1 # [ 153.023441] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=9 outputs=17106worker2 # [ 153.029177] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:09Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07107worker1 # [ 153.037198] nix-daemon[847]: reaped child process 1868, status = succeeded7108worker1 # [ 153.065106] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307109worker1 # [ 153.076946] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307110worker1 # [ 153.090300] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.300Z level=INFO msg="All 1 paths already cached"7111worker1 # [ 153.097135] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307112worker1 # [ 153.105176] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7113worker1 # [ 153.205552] nix-daemon[847]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)7114worker1 # [ 153.218847] nix-daemon[847]: reaped child process 1896, status = succeeded7115worker1 # [ 153.304707] nix-daemon[847]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)7116worker1 # [ 153.317833] nix-daemon[847]: reaped child process 1901, status = succeeded7117lb # [ 153.335251] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received uploads request method=POST path=/api/pending_closures7118worker1 # [ 153.332480] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.542Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7119worker1 # [ 153.333854] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.542Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7120lb # [ 153.359073] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7121lb # [ 153.372839] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7122worker1 # [ 153.368636] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.578Z level=INFO msg="Uploading 1 narinfos"7123lb # [ 153.378596] niks3-server[1111]: 2026/09/22 11:27:08 INFO Signed narinfos id=64 count=17124lb # [ 153.383375] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7125lb # [ 153.396689] niks3-server[1111]: 2026/09/22 11:27:08 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7126worker1 # [ 153.391023] nix-grpc-daemon[1740]: time=2026-09-22T11:27:08.601Z level=INFO msg="Upload complete. (273ms)"7127lb # [ 153.401971] niks3-server[1111]: 2026/09/22 11:27:08 INFO Completed upload id=647128worker1 # [ 153.393441] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=6247129worker1 # [ 153.396862] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307130lb # [ 153.407479] niks3-server[1111]: 2026/09/22 11:27:08 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7131worker1 # [ 153.403317] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7132worker1 # [ 153.411345] nix-daemon[847]: reaped child process 1891, status = succeeded7133worker1 # [ 153.414792] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7134worker1 # [ 154.631745] nix-daemon[847]: reaped child process 1905, status = succeeded7135worker1 # [ 154.709910] nix-daemon[847]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)7136worker1 # [ 154.723675] nix-daemon[847]: reaped child process 1924, status = succeeded7137lb # [ 154.741642] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received uploads request method=POST path=/api/pending_closures7138worker1 # [ 154.739273] nix-grpc-daemon[1740]: time=2026-09-22T11:27:09.949Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7139worker1 # [ 154.740942] nix-grpc-daemon[1740]: time=2026-09-22T11:27:09.951Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7140lb # [ 154.776579] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7141lb # [ 154.790292] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7142lb # [ 154.801451] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7143lb # [ 154.806883] niks3-server[1111]: 2026/09/22 11:27:10 INFO Signed narinfos id=65 count=17144worker1 # [ 154.800138] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.010Z level=INFO msg="Uploading 1 narinfos"7145lb # [ 154.813725] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7146lb # [ 154.821875] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7147lb # [ 154.825837] niks3-server[1111]: 2026/09/22 11:27:10 INFO Completed upload id=657148worker1 # [ 154.815709] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.026Z level=INFO msg="Upload complete. (192ms)"7149lb # [ 154.827478] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7150worker1 # [ 154.820596] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=10 outputs=17151worker2 # [ 154.821199] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:10Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17152worker1 # [ 154.827734] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307153worker2 # [ 154.828605] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:11Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07154worker1 # [ 154.836594] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307155worker1 # [ 154.844690] nix-daemon[847]: reaped child process 1903, status = succeeded7156worker1 # [ 154.846201] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.056Z level=INFO msg="All 1 paths already cached"7157worker1 # [ 154.849756] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307158worker1 # [ 154.853578] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7159worker1 # [ 154.943038] nix-daemon[847]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)7160worker1 # [ 154.956581] nix-daemon[847]: reaped child process 1931, status = succeeded7161worker1 # [ 155.043168] nix-daemon[847]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)7162worker1 # [ 155.057137] nix-daemon[847]: reaped child process 1936, status = succeeded7163lb # [ 155.074073] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received uploads request method=POST path=/api/pending_closures7164worker1 # [ 155.070955] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.281Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7165worker1 # [ 155.072473] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.282Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7166lb # [ 155.104834] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7167lb # [ 155.112353] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7168lb # [ 155.116844] niks3-server[1111]: 2026/09/22 11:27:10 INFO Signed narinfos id=66 count=17169worker1 # [ 155.109465] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.319Z level=INFO msg="Uploading 1 narinfos"7170lb # [ 155.126317] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7171lb # [ 155.136078] niks3-server[1111]: 2026/09/22 11:27:10 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7172lb # [ 155.140536] niks3-server[1111]: 2026/09/22 11:27:10 INFO Completed upload id=667173worker1 # [ 155.133075] nix-grpc-daemon[1740]: time=2026-09-22T11:27:10.343Z level=INFO msg="Upload complete. (273ms)"7174worker1 # [ 155.135220] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=6247175lb # [ 155.146964] niks3-server[1111]: 2026/09/22 11:27:10 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7176worker1 # [ 155.141494] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307177worker1 # [ 155.146401] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7178worker1 # [ 155.148720] nix-daemon[847]: reaped child process 1926, status = succeeded7179worker1 # [ 155.153904] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7180worker1 # [ 155.769414] nix-daemon[847]: reaped child process 1940, status = succeeded7181worker1 # [ 155.840221] nix-daemon[847]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)7182worker1 # [ 155.853952] nix-daemon[847]: reaped child process 1959, status = succeeded7183lb # [ 155.868467] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received uploads request method=POST path=/api/pending_closures7184worker1 # [ 155.867042] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7185worker1 # [ 155.869545] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.078Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7186lb # [ 155.904132] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7187lb # [ 155.914736] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7188worker1 # [ 155.909510] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.119Z level=INFO msg="Uploading 1 narinfos"7189lb # [ 155.920448] niks3-server[1111]: 2026/09/22 11:27:11 INFO Signed narinfos id=67 count=17190lb # [ 155.926161] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7191lb # [ 155.933924] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7192lb # [ 155.943154] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7193lb # [ 155.947692] niks3-server[1111]: 2026/09/22 11:27:11 INFO Completed upload id=677194lb # [ 155.949613] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7195worker1 # [ 155.938989] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.149Z level=INFO msg="Upload complete. (178ms)"7196worker2 # [ 155.946379] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:12Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17197worker1 # [ 155.947986] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=11 outputs=17198worker1 # [ 155.954139] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307199worker2 # [ 155.953288] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:12Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07200worker1 # [ 155.965539] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307201worker1 # [ 155.973248] nix-daemon[847]: reaped child process 1938, status = succeeded7202worker1 # [ 155.976148] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.185Z level=INFO msg="All 2 paths already cached"7203worker1 # [ 155.979767] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307204worker1 # [ 155.985811] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7205worker1 # [ 156.065480] nix-daemon[847]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7206worker1 # [ 156.079225] nix-daemon[847]: reaped child process 1966, status = succeeded7207worker1 # [ 156.154259] nix-daemon[847]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)7208worker1 # [ 156.168567] nix-daemon[847]: reaped child process 1971, status = succeeded7209lb # [ 156.184226] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received uploads request method=POST path=/api/pending_closures7210worker1 # [ 156.182205] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.392Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7211worker1 # [ 156.183573] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.392Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7212lb # [ 156.214495] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7213lb # [ 156.220671] niks3-server[1111]: 2026/09/22 11:27:11 INFO Signed narinfos id=68 count=17214worker1 # [ 156.216726] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.426Z level=INFO msg="Uploading 1 narinfos"7215lb # [ 156.230635] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7216lb # [ 156.235261] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7217lb # [ 156.248250] niks3-server[1111]: 2026/09/22 11:27:11 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7218lb # [ 156.253343] niks3-server[1111]: 2026/09/22 11:27:11 INFO Completed upload id=687219worker1 # [ 156.245413] nix-grpc-daemon[1740]: time=2026-09-22T11:27:11.455Z level=INFO msg="Upload complete. (254ms)"7220worker1 # [ 156.248234] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=7527221lb # [ 156.259530] niks3-server[1111]: 2026/09/22 11:27:11 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7222worker1 # [ 156.253882] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307223worker1 # [ 156.262534] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7224worker1 # [ 156.270306] nix-daemon[847]: reaped child process 1961, status = succeeded7225worker1 # [ 156.273535] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7226worker1 # [ 157.490517] nix-daemon[847]: reaped child process 1975, status = succeeded7227worker1 # [ 157.563136] nix-daemon[847]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)7228worker1 # [ 157.577410] nix-daemon[847]: reaped child process 1994, status = succeeded7229lb # [ 157.593780] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received uploads request method=POST path=/api/pending_closures7230worker1 # [ 157.591210] nix-grpc-daemon[1740]: time=2026-09-22T11:27:12.801Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7231worker1 # [ 157.594059] nix-grpc-daemon[1740]: time=2026-09-22T11:27:12.803Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7232lb # [ 157.631719] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7233lb # [ 157.640222] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7234lb # [ 157.653891] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7235worker1 # [ 157.649404] nix-grpc-daemon[1740]: time=2026-09-22T11:27:12.859Z level=INFO msg="Uploading 1 narinfos"7236lb # [ 157.658985] niks3-server[1111]: 2026/09/22 11:27:13 INFO Signed narinfos id=69 count=17237lb # [ 157.666515] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7238lb # [ 157.677542] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7239lb # [ 157.680962] niks3-server[1111]: 2026/09/22 11:27:13 INFO Completed upload id=697240worker1 # [ 157.671085] nix-grpc-daemon[1740]: time=2026-09-22T11:27:12.881Z level=INFO msg="Upload complete. (190ms)"7241lb # [ 157.683729] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7242worker1 # [ 157.676572] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=12 outputs=17243worker2 # [ 157.675694] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:13Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17244worker1 # [ 157.691611] nix-daemon[847]: reaped child process 1973, status = succeeded7245worker1 # [ 157.720894] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307246worker1 # [ 157.732458] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307247worker1 # [ 157.744515] nix-grpc-daemon[1740]: time=2026-09-22T11:27:12.954Z level=INFO msg="All 2 paths already cached"7248worker1 # [ 157.751128] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307249worker1 # [ 157.760164] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7250worker1 # [ 157.858158] nix-daemon[847]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)7251worker1 # [ 157.872117] nix-daemon[847]: reaped child process 2001, status = succeeded7252worker1 # [ 157.954193] nix-daemon[847]: accepted connection from pid 2003, user nix-grpc-daemon (trusted)7253worker1 # [ 157.967045] nix-daemon[847]: reaped child process 2006, status = succeeded7254lb # [ 157.984347] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received uploads request method=POST path=/api/pending_closures7255worker1 # [ 157.982586] nix-grpc-daemon[1740]: time=2026-09-22T11:27:13.192Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7256worker1 # [ 157.983928] nix-grpc-daemon[1740]: time=2026-09-22T11:27:13.192Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7257lb # [ 158.011282] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7258lb # [ 158.021124] niks3-server[1111]: 2026/09/22 11:27:13 INFO Signed narinfos id=70 count=17259worker1 # [ 158.013956] nix-grpc-daemon[1740]: time=2026-09-22T11:27:13.224Z level=INFO msg="Uploading 1 narinfos"7260lb # [ 158.026643] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7261lb # [ 158.032232] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7262lb # [ 158.045607] niks3-server[1111]: 2026/09/22 11:27:13 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7263worker1 # [ 158.041051] nix-grpc-daemon[1740]: time=2026-09-22T11:27:13.251Z level=INFO msg="Upload complete. (269ms)"7264lb # [ 158.051272] niks3-server[1111]: 2026/09/22 11:27:13 INFO Completed upload id=707265worker1 # [ 158.043142] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=7527266lb # [ 158.057537] niks3-server[1111]: 2026/09/22 11:27:13 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7267worker1 # [ 158.047492] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307268worker1 # [ 158.055051] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7269worker1 # [ 158.062237] nix-daemon[847]: reaped child process 1996, status = succeeded7270worker1 # [ 158.065169] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7271worker1 # [ 159.104969] nix-daemon[847]: reaped child process 2010, status = succeeded7272worker1 # [ 159.170209] nix-daemon[847]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)7273worker1 # [ 159.184205] nix-daemon[847]: reaped child process 2029, status = succeeded7274lb # [ 159.201732] niks3-server[1111]: 2026/09/22 11:27:14 INFO Received uploads request method=POST path=/api/pending_closures7275worker1 # [ 159.199254] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.409Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7276worker1 # [ 159.200866] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.411Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7277lb # [ 159.237785] niks3-server[1111]: 2026/09/22 11:27:14 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7278lb # [ 159.249598] niks3-server[1111]: 2026/09/22 11:27:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7279lb # [ 159.255799] niks3-server[1111]: 2026/09/22 11:27:14 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7280worker1 # [ 159.252470] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.462Z level=INFO msg="Uploading 1 narinfos"7281lb # [ 159.262955] niks3-server[1111]: 2026/09/22 11:27:14 INFO Signed narinfos id=71 count=17282lb # [ 159.268423] niks3-server[1111]: 2026/09/22 11:27:14 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7283lb # [ 159.279455] niks3-server[1111]: 2026/09/22 11:27:14 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7284lb # [ 159.283884] niks3-server[1111]: 2026/09/22 11:27:14 INFO Completed upload id=717285lb # [ 159.285638] niks3-server[1111]: 2026/09/22 11:27:14 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7286worker1 # [ 159.275154] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.485Z level=INFO msg="Upload complete. (202ms)"7287worker2 # [ 159.279932] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:15Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17288worker1 # [ 159.280905] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=13 outputs=17289worker1 # [ 159.290390] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307290worker1 # [ 159.302549] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428307291worker1 # [ 159.308437] nix-daemon[847]: reaped child process 2008, status = succeeded7292worker1 # [ 159.314453] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.524Z level=INFO msg="All 2 paths already cached"7293worker1 # [ 159.320700] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428307294worker1 # [ 159.327275] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7295worker1 # [ 159.413893] nix-daemon[847]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)7296worker1 # [ 159.429573] nix-daemon[847]: reaped child process 2036, status = succeeded7297worker1 # [ 159.508826] nix-daemon[847]: accepted connection from pid 2038, user nix-grpc-daemon (trusted)7298worker1 # [ 159.522044] nix-daemon[847]: reaped child process 2041, status = succeeded7299lb # [ 159.539467] niks3-server[1111]: 2026/09/22 11:27:14 INFO Received uploads request method=POST path=/api/pending_closures7300worker1 # [ 159.536081] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.746Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7301worker1 # [ 159.537565] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.747Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7302lb # [ 159.569473] niks3-server[1111]: 2026/09/22 11:27:15 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7303lb # [ 159.578368] niks3-server[1111]: 2026/09/22 11:27:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7304lb # [ 159.584138] niks3-server[1111]: 2026/09/22 11:27:15 INFO Signed narinfos id=72 count=17305worker1 # [ 159.576675] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.787Z level=INFO msg="Uploading 1 narinfos"7306lb # [ 159.593211] niks3-server[1111]: 2026/09/22 11:27:15 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7307lb # [ 159.605293] niks3-server[1111]: 2026/09/22 11:27:15 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7308lb # [ 159.610540] niks3-server[1111]: 2026/09/22 11:27:15 INFO Completed upload id=727309worker1 # [ 159.600117] nix-grpc-daemon[1740]: time=2026-09-22T11:27:14.810Z level=INFO msg="Upload complete. (266ms)"7310lb # [ 159.614803] niks3-server[1111]: 2026/09/22 11:27:15 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7311worker1 # [ 159.603818] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 paths=1 nar_bytes_in=7527312worker1 # [ 159.610282] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428307313worker1 # [ 159.617475] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7314worker1 # [ 159.624057] nix-daemon[847]: reaped child process 2031, status = succeeded7315worker1 # [ 159.628911] nix-daemon[847]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)7316worker1 # [ 160.461405] nix-daemon[847]: reaped child process 2045, status = succeeded7317worker1 # [ 160.527127] nix-daemon[847]: accepted connection from pid 2061, user nix-grpc-daemon (trusted)7318worker1 # [ 160.540889] nix-daemon[847]: reaped child process 2064, status = succeeded7319lb # [ 160.557454] niks3-server[1111]: 2026/09/22 11:27:15 INFO Received uploads request method=POST path=/api/pending_closures7320worker1 # [ 160.551481] nix-grpc-daemon[1740]: time=2026-09-22T11:27:15.761Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7321worker1 # [ 160.552907] nix-grpc-daemon[1740]: time=2026-09-22T11:27:15.763Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7322lb # [ 160.576409] niks3-server[1111]: 2026/09/22 11:27:16 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7323lb # [ 160.580603] niks3-server[1111]: 2026/09/22 11:27:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7324lb # [ 160.582834] niks3-server[1111]: 2026/09/22 11:27:16 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7325lb # [ 160.584895] niks3-server[1111]: 2026/09/22 11:27:16 INFO Signed narinfos id=73 count=17326worker1 # [ 160.575995] nix-grpc-daemon[1740]: time=2026-09-22T11:27:15.786Z level=INFO msg="Uploading 1 narinfos"7327lb # [ 160.589273] niks3-server[1111]: 2026/09/22 11:27:16 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7328lb # [ 160.594053] niks3-server[1111]: 2026/09/22 11:27:16 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7329lb # [ 160.596981] niks3-server[1111]: 2026/09/22 11:27:16 INFO Completed upload id=737330worker1 # [ 160.586772] nix-grpc-daemon[1740]: time=2026-09-22T11:27:15.797Z level=INFO msg="Upload complete. (151ms)"7331worker1 # [ 160.589182] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42830 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=14 outputs=17332lb # [ 160.601096] niks3-server[1111]: 2026/09/22 11:27:16 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7333worker1 # [ 160.596130] nix-daemon[847]: reaped child process 2043, status = succeeded7334client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.17 seconds)7335worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7336worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)7337client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7338client # [ 160.718911] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7339client # Running as unit: gpu.service; invocation ID: b3d6a5ef21624818b36810d6fd3a439c7340client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7341lb: must succeed: curl -sf localhost:9901/clusters7342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7343lb: must succeed: curl -sf localhost:9901/clusters7344lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7345worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7346worker2 # [ 160.845763] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490207347client # [ 160.858754] nix[1473]: don't know how to build these paths:7348client # [ 160.859822] nix[1473]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7349worker2 # [ 160.854324] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490207350worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7351worker2 # [ 160.873275] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:17Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07352client # [ 160.884295] nix[1473]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7353lb: must succeed: curl -sf localhost:9901/clusters7354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7355lb: must succeed: curl -sf localhost:9901/clusters7356lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7357worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7358worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7359client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7360lb # [ 164.696446] envoy[896]: conn peer=[2001:db8:1::1]:33304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27068 tx=11312 ms=159987361client # [ 190.885515] nix[1473]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7362client # [ 190.893499] nix[1473]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7363client # [ 190.924233] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7364client # [ 190.927822] systemd[1]: gpu.service: Failed with result 'exit-code'.7365client # [ 190.931974] systemd[1]: gpu.service: Consumed 128ms CPU time over 30.209s wall clock time, 35.6M memory peak, 5.1K incoming IP traffic, 5.1K outgoing IP traffic.7366client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.35 seconds)7367client: must succeed: journalctl -u gpu -o cat7368client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7369lb: must succeed: curl -sf localhost:9901/clusters7370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7371lb: must succeed: curl -sf localhost:9901/clusters7372lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7373worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7374worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)7375(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.98 seconds)7376subtest: low disk drains a worker and builds go to the other7377worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7378worker1 # [ 192.298115] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:47Z level=info event=unhealthy reason=min_free7379worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)7380worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7381worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds)7382client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27383worker1 # [ 192.613304] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428307384client # don't know how to build these paths:7385client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7386worker2 # [ 192.624945] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490207387worker2 # [ 192.644516] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07388worker2 # [ 192.650349] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17389worker2 # [ 192.655283] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207390worker2 # [ 192.665967] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490207391client # copying 1 paths to 'lb:50051'...7392client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7393worker2 # [ 192.677789] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490207394worker2 # [ 192.689579] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7395worker2 # [ 192.795477] nix-daemon[868]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)7396worker2 # [ 192.812947] nix-daemon[868]: reaped child process 1958, status = succeeded7397worker2 # [ 192.905679] nix-daemon[868]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)7398worker2 # [ 192.922169] nix-daemon[868]: reaped child process 1963, status = succeeded7399lb # [ 192.941405] niks3-server[1111]: 2026/09/22 11:27:48 INFO Received uploads request method=POST path=/api/pending_closures7400worker2 # [ 192.937319] nix-grpc-daemon[1743]: time=2026-09-22T11:27:49.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7401worker2 # [ 192.939163] nix-grpc-daemon[1743]: time=2026-09-22T11:27:49.115Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7402lb # [ 192.973871] niks3-server[1111]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7403lb # [ 192.984644] niks3-server[1111]: 2026/09/22 11:27:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7404worker2 # [ 192.980938] nix-grpc-daemon[1743]: time=2026-09-22T11:27:49.157Z level=INFO msg="Uploading 1 narinfos"7405lb # [ 192.993060] niks3-server[1111]: 2026/09/22 11:27:48 INFO Signed narinfos id=74 count=17406lb # [ 193.003871] niks3-server[1111]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7407lb # [ 193.015441] niks3-server[1111]: 2026/09/22 11:27:48 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7408lb # [ 193.018467] niks3-server[1111]: 2026/09/22 11:27:48 INFO Completed upload id=747409worker2 # [ 193.007411] nix-grpc-daemon[1743]: time=2026-09-22T11:27:49.184Z level=INFO msg="Upload complete. (306ms)"7410lb # [ 193.020058] niks3-server[1111]: 2026/09/22 11:27:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7411worker2 # [ 193.009816] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=5127412worker2 # [ 193.012632] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207413worker2 # [ 193.020061] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7414worker2 # [ 193.025624] nix-daemon[868]: reaped child process 1953, status = succeeded7415worker2 # [ 193.029967] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7416client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7417client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7418client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7419client # farm-a> LOG-farm-a-drain7420worker2 # [ 194.076513] nix-daemon[868]: reaped child process 1967, status = succeeded7421worker2 # [ 194.152990] nix-daemon[868]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)7422worker2 # [ 194.168248] nix-daemon[868]: reaped child process 1986, status = succeeded7423lb # [ 194.186455] niks3-server[1111]: 2026/09/22 11:27:49 INFO Received uploads request method=POST path=/api/pending_closures7424worker2 # [ 194.182525] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7425worker2 # [ 194.184034] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.361Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7426lb # [ 194.227698] niks3-server[1111]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7427lb # [ 194.237045] niks3-server[1111]: 2026/09/22 11:27:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7428lb # [ 194.242663] niks3-server[1111]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7429lb # [ 194.247628] niks3-server[1111]: 2026/09/22 11:27:49 INFO Signed narinfos id=75 count=17430worker2 # [ 194.238885] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.415Z level=INFO msg="Uploading 1 narinfos"7431lb # [ 194.254451] niks3-server[1111]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7432lb # [ 194.259069] niks3-server[1111]: 2026/09/22 11:27:49 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7433lb # [ 194.264592] niks3-server[1111]: 2026/09/22 11:27:49 INFO Completed upload id=757434lb # [ 194.265848] niks3-server[1111]: 2026/09/22 11:27:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7435worker2 # [ 194.252393] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.429Z level=INFO msg="Upload complete. (213ms)"7436worker2 # [ 194.258631] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=15 outputs=17437worker2 # [ 194.272697] nix-daemon[868]: reaped child process 1965, status = succeeded7438worker2 # [ 194.305928] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07439worker2 # [ 194.310927] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17440worker2 # [ 194.351293] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207441worker2 # [ 194.364683] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490207442worker2 # [ 194.378255] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.555Z level=INFO msg="All 1 paths already cached"7443client # copying 1 paths to 'lb:50051'...7444client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7445worker2 # [ 194.386394] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490207446worker2 # [ 194.397127] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7447worker2 # [ 194.500180] nix-daemon[868]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)7448worker2 # [ 194.516673] nix-daemon[868]: reaped child process 1995, status = succeeded7449worker2 # [ 194.599459] nix-daemon[868]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)7450worker2 # [ 194.613666] nix-daemon[868]: reaped child process 2000, status = succeeded7451lb # [ 194.632904] niks3-server[1111]: 2026/09/22 11:27:50 INFO Received uploads request method=POST path=/api/pending_closures7452worker2 # [ 194.630036] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.807Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7453worker2 # [ 194.632060] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.808Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7454lb # [ 194.662775] niks3-server[1111]: 2026/09/22 11:27:50 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7455lb # [ 194.666438] niks3-server[1111]: 2026/09/22 11:27:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7456lb # [ 194.669508] niks3-server[1111]: 2026/09/22 11:27:50 INFO Signed narinfos id=76 count=17457worker2 # [ 194.657575] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.834Z level=INFO msg="Uploading 1 narinfos"7458lb # [ 194.672400] niks3-server[1111]: 2026/09/22 11:27:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7459lb # [ 194.678814] niks3-server[1111]: 2026/09/22 11:27:50 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7460lb # [ 194.681718] niks3-server[1111]: 2026/09/22 11:27:50 INFO Completed upload id=767461worker2 # [ 194.670739] nix-grpc-daemon[1743]: time=2026-09-22T11:27:50.847Z level=INFO msg="Upload complete. (260ms)"7462lb # [ 194.683755] niks3-server[1111]: 2026/09/22 11:27:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7463worker2 # [ 194.672946] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=6247464worker2 # [ 194.676189] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207465worker2 # [ 194.683830] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7466lb # [ 194.697858] envoy[896]: conn peer=[2001:db8:1::1]:43006 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300547467worker2 # [ 194.687595] nix-daemon[868]: reaped child process 1990, status = succeeded7468worker2 # [ 194.692924] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7469client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7470client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7471client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7472client # farm-b> LOG-farm-b-drain7473worker2 # [ 195.732532] nix-daemon[868]: reaped child process 2004, status = succeeded7474worker2 # [ 195.819101] nix-daemon[868]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)7475worker2 # [ 195.832871] nix-daemon[868]: reaped child process 2023, status = succeeded7476lb # [ 195.851734] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received uploads request method=POST path=/api/pending_closures7477worker2 # [ 195.848610] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.025Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7478worker2 # [ 195.850247] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.027Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7479lb # [ 195.897462] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7480lb # [ 195.906405] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7481lb # [ 195.912635] niks3-server[1111]: 2026/09/22 11:27:51 INFO Signed narinfos id=77 count=17482worker2 # [ 195.903862] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.080Z level=INFO msg="Uploading 1 narinfos"7483lb # [ 195.918763] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7484lb # [ 195.928334] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7485lb # [ 195.937238] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7486lb # [ 195.942183] niks3-server[1111]: 2026/09/22 11:27:51 INFO Completed upload id=777487lb # [ 195.944224] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7488worker2 # [ 195.931492] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.108Z level=INFO msg="Upload complete. (229ms)"7489worker2 # [ 195.939900] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=16 outputs=17490worker2 # [ 195.950079] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07491worker2 # [ 195.954405] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17492worker2 # [ 195.964986] nix-daemon[868]: reaped child process 2002, status = succeeded7493worker2 # [ 195.994893] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207494worker2 # [ 196.008256] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490207495worker2 # [ 196.021351] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.198Z level=INFO msg="All 2 paths already cached"7496client # copying 1 paths to 'lb:50051'...7497client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7498worker2 # [ 196.029813] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490207499worker2 # [ 196.039924] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7500worker2 # [ 196.144136] nix-daemon[868]: accepted connection from pid 2029, user nix-grpc-daemon (trusted)7501worker2 # [ 196.160202] nix-daemon[868]: reaped child process 2032, status = succeeded7502worker2 # [ 196.245620] nix-daemon[868]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7503worker2 # [ 196.261187] nix-daemon[868]: reaped child process 2037, status = succeeded7504lb # [ 196.278229] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received uploads request method=POST path=/api/pending_closures7505worker2 # [ 196.273722] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.450Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7506worker2 # [ 196.275291] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.450Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7507lb # [ 196.303321] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7508lb # [ 196.316440] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7509lb # [ 196.322438] niks3-server[1111]: 2026/09/22 11:27:51 INFO Signed narinfos id=78 count=17510worker2 # [ 196.313777] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.490Z level=INFO msg="Uploading 1 narinfos"7511lb # [ 196.329310] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7512lb # [ 196.342613] niks3-server[1111]: 2026/09/22 11:27:51 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7513worker2 # [ 196.335757] nix-grpc-daemon[1743]: time=2026-09-22T11:27:52.512Z level=INFO msg="Upload complete. (282ms)"7514lb # [ 196.347706] niks3-server[1111]: 2026/09/22 11:27:51 INFO Completed upload id=787515worker2 # [ 196.338293] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=0 paths=1 nar_bytes_in=7527516lb # [ 196.352559] niks3-server[1111]: 2026/09/22 11:27:51 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7517worker2 # [ 196.344630] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490207518worker2 # [ 196.356525] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7519worker2 # [ 196.359659] nix-daemon[868]: reaped child process 2027, status = succeeded7520worker2 # [ 196.363379] nix-daemon[868]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)7521client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7522client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7523client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7524client # farm-top> LOG-farm-top-drain7525worker2 # [ 197.579830] nix-daemon[868]: reaped child process 2041, status = succeeded7526worker2 # [ 197.659713] nix-daemon[868]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)7527worker2 # [ 197.675426] nix-daemon[868]: reaped child process 2060, status = succeeded7528lb # [ 197.691967] niks3-server[1111]: 2026/09/22 11:27:53 INFO Received uploads request method=POST path=/api/pending_closures7529worker2 # [ 197.688620] nix-grpc-daemon[1743]: time=2026-09-22T11:27:53.865Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7530worker2 # [ 197.691158] nix-grpc-daemon[1743]: time=2026-09-22T11:27:53.867Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7531lb # [ 197.724097] niks3-server[1111]: 2026/09/22 11:27:53 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7532lb # [ 197.739642] niks3-server[1111]: 2026/09/22 11:27:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7533lb # [ 197.746061] niks3-server[1111]: 2026/09/22 11:27:53 INFO Signed narinfos id=79 count=17534worker2 # [ 197.736907] nix-grpc-daemon[1743]: time=2026-09-22T11:27:53.913Z level=INFO msg="Uploading 1 narinfos"7535lb # [ 197.757794] niks3-server[1111]: 2026/09/22 11:27:53 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7536lb # [ 197.763101] niks3-server[1111]: 2026/09/22 11:27:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7537lb # [ 197.769775] niks3-server[1111]: 2026/09/22 11:27:53 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7538lb # [ 197.773560] niks3-server[1111]: 2026/09/22 11:27:53 INFO Completed upload id=797539worker2 # [ 197.761522] nix-grpc-daemon[1743]: time=2026-09-22T11:27:53.938Z level=INFO msg="Upload complete. (191ms)"7540lb # [ 197.775608] niks3-server[1111]: 2026/09/22 11:27:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7541worker2 # [ 197.770547] nix-grpc-daemon[1666]: ts=2026-09-22T11:27:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49020 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=17 outputs=17542worker2 # [ 197.784653] nix-daemon[868]: reaped child process 2039, status = succeeded7543client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.33 seconds)7544client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7545client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7546worker1: must succeed: rm /nix/.rw-store/fill7547worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7548worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7549worker1 # [ 198.300277] nix-grpc-daemon[1680]: ts=2026-09-22T11:27:53Z level=info event=healthy reason=min_free7550worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.06 seconds)7551(finished: subtest: low disk drains a worker and builds go to the other, in 7.70 seconds)7552(finished: run the VM test script, in 200.03 seconds)7553test script finished in 200.20s7554cleanup7555kill QemuMachine (pid 49)7556client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7557client # [2026-09-22T11:27:55Z INFO virtiofsd] Client disconnected, shutting down7558client # [2026-09-22T11:27:55Z INFO virtiofsd] Client disconnected, shutting down7559client # [2026-09-22T11:27:55Z INFO virtiofsd] Client disconnected, shutting down7560kill QemuMachine (pid 51)7561lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7562lb # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7563lb # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7564lb # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7565kill QemuMachine (pid 50)7566worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7567worker1 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7568worker1 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7569worker1 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7570kill QemuMachine (pid 52)7571worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7572worker2 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7573worker2 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7574worker2 # [2026-09-22T11:27:56Z INFO virtiofsd] Client disconnected, shutting down7575(finished: cleanup, in 1.26 seconds)7576additionally exposed symbols:7577 client, lb, worker1, worker2,7578 vlan1,7579 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh