vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #518
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14client # Formatting '/build/vm-state-client/tmp.Lfp6ShbdSN', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: 25cfa1e7-df3e-4523-9fd0-db9134629aa619client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26client # 27worker1 # Disk image does not exist, creating the virtualisation disk image...28worker1: QEMU running (pid 51)29worker1 # Formatting '/build/vm-state-worker1/tmp.1DHARLjqYu', fmt=raw size=107374182430worker2: QEMU running (pid 52)31worker1 # mke2fs 1.47.4 (6-Mar-2025)32worker2 # Disk image does not exist, creating the virtualisation disk image...33worker1 # Discarding device blocks: 0/262144 done34worker2 # Formatting '/build/vm-state-worker2/tmp.Nz0xsFCe9I', fmt=raw size=107374182435worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes36worker2 # mke2fs 1.47.4 (6-Mar-2025)37worker1 # Filesystem UUID: 532642d2-c6f1-49f2-a2b4-3782e76d531f38worker2 # Discarding device blocks: 0/262144 done39worker1 # Superblock backups stored on blocks:40worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes41worker1 # 32768, 98304, 163840, 22937642worker2 # Filesystem UUID: 55846195-6ef6-4740-bf73-4ddfc108f09a43worker1 # 44worker2 # Superblock backups stored on blocks:45worker1 # Allocating group tables: 0/8 done46worker2 # 32768, 98304, 163840, 22937647worker1 # Writing inode tables: 0/8 done48worker2 # 49worker1 # Creating journal (8192 blocks): done50worker1 # Writing superblocks and filesystem accounting information: 0/8 done51worker1 # 52worker1 # Virtualisation disk image created.53worker1 # Starting virtiofs daemons...54worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)55worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether56worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)57worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether58worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...59worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...60worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61worker1 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether62worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...63worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests64worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests65worker1 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests66worker2 # Allocating group tables: 0/8 done67client # Virtualisation disk image created.68worker2 # Writing inode tables: 0/8 done69client # Starting virtiofs daemons...70worker2 # Creating journal (8192 blocks): done71client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73client # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...74client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76client # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...77client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78client # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...80client # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests81client # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests82client # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests83worker2 # Writing superblocks and filesystem accounting information: 0/8 done84worker2 # 85worker2 # Virtualisation disk image created.86worker2 # Starting virtiofs daemons...87worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)93worker2 # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...95worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests97worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests98worker2 # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests99lb # Disk image does not exist, creating the virtualisation disk image...100lb: QEMU running (pid 50)101lb # Formatting '/build/vm-state-lb/tmp.tkitIUtndL', fmt=raw size=1073741824102lb # mke2fs 1.47.4 (6-Mar-2025)103lb # Discarding device blocks: 0/262144 done104lb # Creating filesystem with 262144 4k blocks and 65536 inodes105lb # Filesystem UUID: 8ebeaf7c-8052-4a07-b2b8-f23a52899149106lb # Superblock backups stored on blocks:107lb # 32768, 98304, 163840, 229376108lb # 109(finished: start all VMs, in 0.34 seconds)110client: waiting for unit network-addresses-eth1.service111lb # Allocating group tables: 0/8 done112client: waiting for the VM to finish booting113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-23T05:11:32Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-23T05:11:32Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T05:11:32Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137worker2 # 138worker2 # 139client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142client # 143client # 144client # 145client # 146worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147worker2 # 148worker2 # 149worker2 # 150worker2 # 151worker1 # 152worker1 # 153client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00154worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00155worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157client # 158client # 159worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...160worker1 # 161worker1 # 162worker1 # 163worker1 # 164worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...165worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00166worker2 # 167worker2 # 168worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker1 # 170worker1 # 171lb # 172lb # 173lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175lb # 176lb # 177lb # 178lb # 179lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183lb # Booting from ROM...184client # Booting from ROM...185worker2 # Booting from ROM...186worker1 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xm7zpmh78xsj3jfdbh2jw86x9ziaswkw-nixos-system-client-test/init regInfo=/nix/store/hmvlr8b2278pc080sma8byzfjan2y23p-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200client # [ 0.000000] NX (Execute Disable) protection: active201client # [ 0.000000] APIC: Static calls initialized202client # [ 0.000000] SMBIOS 2.8 present.203client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204client # [ 0.000000] DMI: Memory slots populated: 1/1205worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026206client # [ 0.000000] Hypervisor detected: KVM207client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00209worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026210client # [ 0.000000] kvm-clock: using sched offset of 459410443 cycles211client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns212worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vhp4sld1igrws2rdxgli0qdljgmsnza3-nixos-system-worker2-test/init regInfo=/nix/store/7r6pyv5kjf3bmhc3psizy6pjwb2p8ghw-closure-info/registration console=ttyS0,115200n8 console=tty0213client # [ 0.000005] tsc: Detected 2400.012 MHz processor214worker2 # [ 0.000000] BIOS-provided physical RAM map:215client # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000216worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable217client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs218worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ckamkygndjknysls7i47y9vrkanbxzr1-nixos-system-worker1-test/init regInfo=/nix/store/5psc4qmnnvn37sl0sb72chfm0rpi87nv-closure-info/registration console=ttyS0,115200n8 console=tty0219worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved220worker1 # [ 0.000000] BIOS-provided physical RAM map:221client # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT222worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved223worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable224client # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]225worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable226client # [ 0.002743] Using GB pages for direct mapping227worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved228client # [ 0.002851] RAMDISK: [mem 0x5e370000-0x5ffcffff]229worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved230worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved231client # [ 0.002859] ACPI: Early table checksum verification disabled232worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved233client # [ 0.002861] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )234worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable235worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved236worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved237client # [ 0.002865] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved239worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved240client # [ 0.002868] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved242worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved243worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved244client # [ 0.002875] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)245worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved246worker2 # [ 0.000000] NX (Execute Disable) protection: active247client # [ 0.002877] ACPI: FACS 0x000000005FFE0000 000040248worker2 # [ 0.000000] APIC: Static calls initialized249worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved250worker2 # [ 0.000000] SMBIOS 2.8 present.251client # [ 0.002878] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)252worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved253worker1 # [ 0.000000] NX (Execute Disable) protection: active254worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014255client # [ 0.002880] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)256worker1 # [ 0.000000] APIC: Static calls initialized257worker2 # [ 0.000000] DMI: Memory slots populated: 1/1258worker1 # [ 0.000000] SMBIOS 2.8 present.259worker2 # [ 0.000000] Hypervisor detected: KVM260client # [ 0.002882] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)261worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000262client # [ 0.002883] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)263worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014264worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00265worker1 # [ 0.000000] DMI: Memory slots populated: 1/1266client # [ 0.002884] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]267worker2 # [ 0.000000] kvm-clock: using sched offset of 458893125 cycles268worker1 # [ 0.000000] Hypervisor detected: KVM269client # [ 0.002886] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]270worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000271worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns272worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00273client # [ 0.002886] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]274worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor275worker1 # [ 0.000001] kvm-clock: using sched offset of 462330939 cycles276client # [ 0.002887] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]277worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000278client # [ 0.002887] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]279worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns280worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs281worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor282client # [ 0.002888] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]283worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT284worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000285client # [ 0.002888] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]286worker2 # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]287client # [ 0.003105] No NUMA configuration found288worker2 # [ 0.002744] Using GB pages for direct mapping289worker1 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs290worker2 # [ 0.002820] RAMDISK: [mem 0x5e36f000-0x5ffcffff]291client # [ 0.003106] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]292worker1 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT293worker2 # [ 0.002829] ACPI: Early table checksum verification disabled294client # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]295worker1 # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f]296client # [ 0.005382] Zone ranges:297worker2 # [ 0.002832] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )298worker1 # [ 0.002736] Using GB pages for direct mapping299client # [ 0.005382] DMA [mem 0x0000000000001000-0x0000000000ffffff]300worker1 # [ 0.002819] RAMDISK: [mem 0x5e370000-0x5ffcffff]301worker2 # [ 0.002836] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)302client # [ 0.005384] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]303worker1 # [ 0.002829] ACPI: Early table checksum verification disabled304client # [ 0.005385] Normal empty305lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026306client # [ 0.005386] Device empty307worker1 # [ 0.002832] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )308worker2 # [ 0.002840] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)309client # [ 0.005386] Movable zone start for each node310client # [ 0.005387] Early memory node ranges311worker2 # [ 0.002847] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)312worker1 # [ 0.002836] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)313client # [ 0.005387] node 0: [mem 0x0000000000001000-0x000000000009efff]314worker2 # [ 0.002849] ACPI: FACS 0x000000005FFE0000 000040315worker1 # [ 0.002840] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)316client # [ 0.005388] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]317worker2 # [ 0.002851] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)318worker1 # [ 0.002847] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)319client # [ 0.005390] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]320lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0321worker2 # [ 0.002852] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)322worker1 # [ 0.002850] ACPI: FACS 0x000000005FFE0000 000040323lb # [ 0.000000] BIOS-provided physical RAM map:324client # [ 0.005409] On node 0, zone DMA: 1 pages in unavailable ranges325client # [ 0.005673] On node 0, zone DMA: 97 pages in unavailable ranges326worker2 # [ 0.002854] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)327lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable328worker1 # [ 0.002851] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)329lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved330client # [ 0.032795] On node 0, zone DMA32: 40 pages in unavailable ranges331worker2 # [ 0.002855] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)332worker1 # [ 0.002853] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)333client # [ 0.033246] ACPI: PM-Timer IO Port: 0x608334lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved335worker2 # [ 0.002857] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]336client # [ 0.033257] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])337worker1 # [ 0.002854] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)338lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable339worker2 # [ 0.002858] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]340client # [ 0.033283] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23341lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved342worker1 # [ 0.002856] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)343client # [ 0.033286] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)344worker2 # [ 0.002858] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]345worker1 # [ 0.002857] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]346lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved347client # [ 0.033287] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)348worker2 # [ 0.002859] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]349lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved350worker1 # [ 0.002858] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]351client # [ 0.033289] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)352worker2 # [ 0.002859] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]353lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved354worker1 # [ 0.002859] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]355client # [ 0.033290] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)356worker2 # [ 0.002860] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]357lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved358worker1 # [ 0.002859] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]359client # [ 0.033290] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)360worker2 # [ 0.002860] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved362worker1 # [ 0.002860] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]363worker2 # [ 0.003090] No NUMA configuration found364client # [ 0.033293] ACPI: Using ACPI (MADT) for SMP configuration information365lb # [ 0.000000] NX (Execute Disable) protection: active366client # [ 0.033294] ACPI: HPET id: 0x8086a201 base: 0xfed00000367worker2 # [ 0.003091] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]368worker1 # [ 0.002860] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]369lb # [ 0.000000] APIC: Static calls initialized370client # [ 0.033297] TSC deadline timer available371lb # [ 0.000000] SMBIOS 2.8 present.372worker2 # [ 0.003094] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]373client # [ 0.033301] CPU topo: Max. logical packages: 1374worker1 # [ 0.002860] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]375worker2 # [ 0.005534] Zone ranges:376worker1 # [ 0.003093] No NUMA configuration found377client # [ 0.033302] CPU topo: Max. logical dies: 1378lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014379worker2 # [ 0.005535] DMA [mem 0x0000000000001000-0x0000000000ffffff]380client # [ 0.033302] CPU topo: Max. dies per package: 1381worker1 # [ 0.003094] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]382lb # [ 0.000000] DMI: Memory slots populated: 1/1383worker2 # [ 0.005537] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]384client # [ 0.033305] CPU topo: Max. threads per core: 1385lb # [ 0.000000] Hypervisor detected: KVM386worker2 # [ 0.005538] Normal empty387worker1 # [ 0.003097] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]388client # [ 0.033306] CPU topo: Num. cores per package: 1389worker1 # [ 0.005526] Zone ranges:390worker2 # [ 0.005539] Device empty391lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000392client # [ 0.033306] CPU topo: Num. threads per package: 1393worker2 # [ 0.005539] Movable zone start for each node394lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00395worker1 # [ 0.005527] DMA [mem 0x0000000000001000-0x0000000000ffffff]396worker2 # [ 0.005540] Early memory node ranges397client # [ 0.033307] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs398lb # [ 0.000001] kvm-clock: using sched offset of 470398907 cycles399worker1 # [ 0.005529] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]400worker2 # [ 0.005540] node 0: [mem 0x0000000000001000-0x000000000009efff]401worker1 # [ 0.005530] Normal empty402client # [ 0.033325] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()403worker1 # [ 0.005531] Device empty404worker2 # [ 0.005542] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]405lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns406worker1 # [ 0.005531] Movable zone start for each node407client # [ 0.033354] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]408worker1 # [ 0.005532] Early memory node ranges409lb # [ 0.000005] tsc: Detected 2400.012 MHz processor410worker2 # [ 0.005543] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]411lb # [ 0.000820] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000412client # [ 0.033356] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]413worker1 # [ 0.005533] node 0: [mem 0x0000000000001000-0x000000000009efff]414worker2 # [ 0.005563] On node 0, zone DMA: 1 pages in unavailable ranges415client # [ 0.033357] [mem 0x60000000-0xafffffff] available for PCI devices416worker1 # [ 0.005534] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]417worker2 # [ 0.005838] On node 0, zone DMA: 97 pages in unavailable ranges418lb # [ 0.000847] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs419client # [ 0.033358] Booting paravirtualized kernel on KVM420worker2 # [ 0.034494] On node 0, zone DMA32: 40 pages in unavailable ranges421worker1 # [ 0.005535] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]422lb # [ 0.000850] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT423worker2 # [ 0.034952] ACPI: PM-Timer IO Port: 0x608424worker1 # [ 0.005556] On node 0, zone DMA: 1 pages in unavailable ranges425lb # [ 0.002762] found SMP MP-table at [mem 0x000f5450-0x000f545f]426client # [ 0.033361] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns427worker2 # [ 0.034963] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])428lb # [ 0.002774] Using GB pages for direct mapping429worker1 # [ 0.005837] On node 0, zone DMA: 97 pages in unavailable ranges430client # [ 0.037875] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1431worker2 # [ 0.034994] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23432lb # [ 0.002888] RAMDISK: [mem 0x5e36e000-0x5ffcffff]433worker1 # [ 0.034073] On node 0, zone DMA32: 40 pages in unavailable ranges434lb # [ 0.002898] ACPI: Early table checksum verification disabled435worker1 # [ 0.034525] ACPI: PM-Timer IO Port: 0x608436worker2 # [ 0.034997] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)437client # [ 0.040061] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152438lb # [ 0.002900] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )439client # [ 0.040100] kvm-guest: PV spinlocks disabled, single CPU440worker1 # [ 0.034537] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])441worker2 # [ 0.034999] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)442worker1 # [ 0.034563] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23443lb # [ 0.002904] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)444worker2 # [ 0.035000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)445worker1 # [ 0.034566] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)446lb # [ 0.002908] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)447worker2 # [ 0.035001] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)448worker1 # [ 0.034568] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)449worker2 # [ 0.035002] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)450lb # [ 0.002916] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)451worker1 # [ 0.034569] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)452lb # [ 0.002918] ACPI: FACS 0x000000005FFE0000 000040453worker2 # [ 0.035004] ACPI: Using ACPI (MADT) for SMP configuration information454client # [ 0.040101] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xm7zpmh78xsj3jfdbh2jw86x9ziaswkw-nixos-system-client-test/init regInfo=/nix/store/hmvlr8b2278pc080sma8byzfjan2y23p-closure-info/registration console=ttyS0,115200n8 console=tty0455worker2 # [ 0.035005] ACPI: HPET id: 0x8086a201 base: 0xfed00000456worker1 # [ 0.034570] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)457lb # [ 0.002920] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)458worker2 # [ 0.035009] TSC deadline timer available459worker1 # [ 0.034570] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)460worker2 # [ 0.035013] CPU topo: Max. logical packages: 1461client # [ 0.040194] Unknown kernel command line parameters "regInfo=/nix/store/hmvlr8b2278pc080sma8byzfjan2y23p-closure-info/registration", will be passed to user space.462lb # [ 0.002921] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)463worker2 # [ 0.035014] CPU topo: Max. logical dies: 1464worker1 # [ 0.034573] ACPI: Using ACPI (MADT) for SMP configuration information465client # [ 0.040207] random: crng init done466worker2 # [ 0.035014] CPU topo: Max. dies per package: 1467worker1 # [ 0.034574] ACPI: HPET id: 0x8086a201 base: 0xfed00000468lb # [ 0.002923] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)469client # [ 0.040207] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes470worker2 # [ 0.035017] CPU topo: Max. threads per core: 1471worker1 # [ 0.034577] TSC deadline timer available472worker1 # [ 0.034581] CPU topo: Max. logical packages: 1473worker2 # [ 0.035018] CPU topo: Num. cores per package: 1474lb # [ 0.002924] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)475client # [ 0.042366] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)476worker2 # [ 0.035018] CPU topo: Num. threads per package: 1477worker1 # [ 0.034582] CPU topo: Max. logical dies: 1478lb # [ 0.002926] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]479worker1 # [ 0.034582] CPU topo: Max. dies per package: 1480client # [ 0.043756] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)481worker2 # [ 0.035018] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs482worker1 # [ 0.034585] CPU topo: Max. threads per core: 1483client # [ 0.043787] Fallback order for Node 0: 0484lb # [ 0.002927] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]485worker2 # [ 0.035040] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()486worker1 # [ 0.034586] CPU topo: Num. cores per package: 1487client # [ 0.043789] Built 1 zonelists, mobility grouping on. Total pages: 393078488lb # [ 0.002927] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]489worker1 # [ 0.034586] CPU topo: Num. threads per package: 1490client # [ 0.043790] Policy zone: DMA32491worker2 # [ 0.035071] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]492lb # [ 0.002928] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]493worker1 # [ 0.034587] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs494client # [ 0.046311] mem auto-init: stack:all(zero), heap alloc:on, heap free:off495worker2 # [ 0.035073] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]496lb # [ 0.002928] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]497worker1 # [ 0.034605] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()498client # [ 0.049436] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1499worker2 # [ 0.035075] [mem 0x60000000-0xafffffff] available for PCI devices500client # [ 0.052793] allocated 3145728 bytes of page_ext501lb # [ 0.002929] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]502worker2 # [ 0.035076] Booting paravirtualized kernel on KVM503worker1 # [ 0.034636] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]504client # [ 0.062585] ftrace: allocating 48787 entries in 192 pages505lb # [ 0.002929] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]506worker1 # [ 0.034638] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]507client # [ 0.062586] ftrace: allocated 192 pages with 2 groups508lb # [ 0.003160] No NUMA configuration found509worker2 # [ 0.035079] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns510client # [ 0.063421] Dynamic Preempt: lazy511worker1 # [ 0.034639] [mem 0x60000000-0xafffffff] available for PCI devices512lb # [ 0.003161] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]513client # [ 0.063548] rcu: Preemptible hierarchical RCU implementation.514worker2 # [ 0.039582] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1515worker1 # [ 0.034640] Booting paravirtualized kernel on KVM516lb # [ 0.003164] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]517client # [ 0.063548] rcu: RCU event tracing is enabled.518lb # [ 0.005597] Zone ranges:519worker2 # [ 0.041892] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152520client # [ 0.063549] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.521worker1 # [ 0.034644] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns522worker2 # [ 0.041938] kvm-guest: PV spinlocks disabled, single CPU523lb # [ 0.005598] DMA [mem 0x0000000000001000-0x0000000000ffffff]524client # [ 0.063551] Trampoline variant of Tasks RCU enabled.525lb # [ 0.005599] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]526worker1 # [ 0.039135] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1527client # [ 0.063551] Rude variant of Tasks RCU enabled.528lb # [ 0.005601] Normal empty529lb # [ 0.005602] Device empty530worker1 # [ 0.041429] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152531client # [ 0.063551] Tracing variant of Tasks RCU enabled.532lb # [ 0.005602] Movable zone start for each node533worker1 # [ 0.041470] kvm-guest: PV spinlocks disabled, single CPU534lb # [ 0.005603] Early memory node ranges535client # [ 0.063552] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.536lb # [ 0.005603] node 0: [mem 0x0000000000001000-0x000000000009efff]537client # [ 0.063553] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1538worker2 # [ 0.041940] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/vhp4sld1igrws2rdxgli0qdljgmsnza3-nixos-system-worker2-test/init regInfo=/nix/store/7r6pyv5kjf3bmhc3psizy6pjwb2p8ghw-closure-info/registration console=ttyS0,115200n8 console=tty0539lb # [ 0.005604] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]540client # [ 0.063562] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.541lb # [ 0.005605] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]542lb # [ 0.005626] On node 0, zone DMA: 1 pages in unavailable ranges543client # [ 0.063563] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.544worker2 # [ 0.042036] Unknown kernel command line parameters "regInfo=/nix/store/7r6pyv5kjf3bmhc3psizy6pjwb2p8ghw-closure-info/registration", will be passed to user space.545worker2 # [ 0.042048] random: crng init done546lb # [ 0.005915] On node 0, zone DMA: 97 pages in unavailable ranges547worker1 # [ 0.041472] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ckamkygndjknysls7i47y9vrkanbxzr1-nixos-system-worker1-test/init regInfo=/nix/store/5psc4qmnnvn37sl0sb72chfm0rpi87nv-closure-info/registration console=ttyS0,115200n8 console=tty0548client # [ 0.063564] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.549lb # [ 0.034421] On node 0, zone DMA32: 40 pages in unavailable ranges550worker2 # [ 0.042049] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes551lb # [ 0.034892] ACPI: PM-Timer IO Port: 0x608552client # [ 0.068302] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16553worker2 # [ 0.044315] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)554worker1 # [ 0.041566] Unknown kernel command line parameters "regInfo=/nix/store/5psc4qmnnvn37sl0sb72chfm0rpi87nv-closure-info/registration", will be passed to user space.555lb # [ 0.034903] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])556client # [ 0.068570] rcu: srcu_init: Setting srcu_struct sizes based on contention.557worker1 # [ 0.041578] random: crng init done558lb # [ 0.034932] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23559worker2 # [ 0.045739] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)560worker1 # [ 0.041579] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes561worker2 # [ 0.045770] Fallback order for Node 0: 0562client # [ 0.068575] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns563lb # [ 0.034935] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)564worker2 # [ 0.045773] Built 1 zonelists, mobility grouping on. Total pages: 393078565worker1 # [ 0.043843] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)566lb # [ 0.034937] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)567client # [ 0.068680] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)568worker2 # [ 0.045774] Policy zone: DMA32569client # [ 0.072211] Console: colour VGA+ 80x25570worker1 # [ 0.045261] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)571lb # [ 0.034938] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)572worker2 # [ 0.048429] mem auto-init: stack:all(zero), heap alloc:on, heap free:off573client # [ 0.072214] printk: legacy console [tty0] enabled574worker1 # [ 0.045294] Fallback order for Node 0: 0575lb # [ 0.034939] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)576worker2 # [ 0.051843] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1577client # [ 0.101725] printk: legacy console [ttyS0] enabled578worker1 # [ 0.045297] Built 1 zonelists, mobility grouping on. Total pages: 393078579worker2 # [ 0.055339] allocated 3145728 bytes of page_ext580lb # [ 0.034940] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)581worker1 # [ 0.045298] Policy zone: DMA32582client # [ 0.197412] ACPI: Core revision 20250807583worker2 # [ 0.065694] ftrace: allocating 48787 entries in 192 pages584lb # [ 0.034942] ACPI: Using ACPI (MADT) for SMP configuration information585worker1 # [ 0.047926] mem auto-init: stack:all(zero), heap alloc:on, heap free:off586worker2 # [ 0.065696] ftrace: allocated 192 pages with 2 groups587lb # [ 0.034943] ACPI: HPET id: 0x8086a201 base: 0xfed00000588worker2 # [ 0.066600] Dynamic Preempt: lazy589worker1 # [ 0.051373] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1590lb # [ 0.034947] TSC deadline timer available591client # [ 0.198320] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns592worker1 # [ 0.054866] allocated 3145728 bytes of page_ext593worker2 # [ 0.066734] rcu: Preemptible hierarchical RCU implementation.594lb # [ 0.034951] CPU topo: Max. logical packages: 1595client # [ 0.199847] APIC: Switch to symmetric I/O mode setup596lb # [ 0.034952] CPU topo: Max. logical dies: 1597worker2 # [ 0.066735] rcu: RCU event tracing is enabled.598worker1 # [ 0.065150] ftrace: allocating 48787 entries in 192 pages599worker1 # [ 0.065152] ftrace: allocated 192 pages with 2 groups600client # [ 0.200772] x2apic enabled601lb # [ 0.034952] CPU topo: Max. dies per package: 1602worker2 # [ 0.066736] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.603worker1 # [ 0.066012] Dynamic Preempt: lazy604lb # [ 0.034956] CPU topo: Max. threads per core: 1605worker2 # [ 0.066737] Trampoline variant of Tasks RCU enabled.606lb # [ 0.034956] CPU topo: Num. cores per package: 1607worker1 # [ 0.066188] rcu: Preemptible hierarchical RCU implementation.608client # [ 0.201496] APIC: Switched APIC routing to: physical x2apic609worker2 # [ 0.066738] Rude variant of Tasks RCU enabled.610lb # [ 0.034957] CPU topo: Num. threads per package: 1611worker1 # [ 0.066188] rcu: RCU event tracing is enabled.612worker2 # [ 0.066738] Tracing variant of Tasks RCU enabled.613lb # [ 0.034957] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs614worker1 # [ 0.066189] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.615worker2 # [ 0.066739] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.616worker1 # [ 0.066191] Trampoline variant of Tasks RCU enabled.617client # [ 0.203426] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1618lb # [ 0.034976] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()619worker2 # [ 0.066739] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1620worker1 # [ 0.066191] Rude variant of Tasks RCU enabled.621lb # [ 0.035008] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]622worker1 # [ 0.066191] Tracing variant of Tasks RCU enabled.623worker2 # [ 0.066797] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.624client # [ 0.204365] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns625lb # [ 0.035010] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]626worker1 # [ 0.066192] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.627lb # [ 0.035011] [mem 0x60000000-0xafffffff] available for PCI devices628worker2 # [ 0.066799] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629worker1 # [ 0.066193] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1630lb # [ 0.035013] Booting paravirtualized kernel on KVM631client # [ 0.205996] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)632worker2 # [ 0.066800] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.633worker1 # [ 0.066214] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634client # [ 0.207080] x86/cpu: User Mode Instruction Prevention (UMIP) activated635lb # [ 0.035015] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns636worker2 # [ 0.071246] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16637worker1 # [ 0.066216] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.638worker2 # [ 0.071536] rcu: srcu_init: Setting srcu_struct sizes based on contention.639lb # [ 0.039551] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1640client # [ 0.208133] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127641lb # [ 0.041746] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152642worker1 # [ 0.066217] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.643client # [ 0.208994] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0644worker2 # [ 0.071543] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns645lb # [ 0.041789] kvm-guest: PV spinlocks disabled, single CPU646worker1 # [ 0.070955] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16647worker2 # [ 0.071653] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)648worker1 # [ 0.071223] rcu: srcu_init: Setting srcu_struct sizes based on contention.649worker2 # [ 0.075517] Console: colour VGA+ 80x25650client # [ 0.209998] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto651worker2 # [ 0.075520] printk: legacy console [tty0] enabled652worker1 # [ 0.071230] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns653worker2 # [ 0.105030] printk: legacy console [ttyS0] enabled654client # [ 0.211994] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl655worker2 # [ 0.200915] ACPI: Core revision 20250807656worker1 # [ 0.071335] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)657lb # [ 0.041791] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0658client # [ 0.212994] Transient Scheduler Attacks: Vulnerable: No microcode659worker1 # [ 0.074883] Console: colour VGA+ 80x25660worker1 # [ 0.074886] printk: legacy console [tty0] enabled661client # [ 0.213994] Spectre V2 : Mitigation: Enhanced / Automatic IBRS662worker2 # [ 0.201776] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns663worker1 # [ 0.104419] printk: legacy console [ttyS0] enabled664lb # [ 0.041887] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.665client # [ 0.214994] Speculative Return Stack Overflow: Mitigation: Safe RET666lb # [ 0.041901] random: crng init done667worker2 # [ 0.203280] APIC: Switch to symmetric I/O mode setup668worker1 # [ 0.200749] ACPI: Core revision 20250807669worker2 # [ 0.204218] x2apic enabled670lb # [ 0.041901] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes671worker1 # [ 0.201644] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns672lb # [ 0.044313] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)673worker2 # [ 0.204987] APIC: Switched APIC routing to: physical x2apic674worker1 # [ 0.203226] APIC: Switch to symmetric I/O mode setup675lb # [ 0.045383] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)676lb # [ 0.045417] Fallback order for Node 0: 0677worker1 # [ 0.204236] x2apic enabled678worker2 # [ 0.206795] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1679client # [ 0.215994] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization680lb # [ 0.045421] Built 1 zonelists, mobility grouping on. Total pages: 393078681lb # [ 0.045422] Policy zone: DMA32682worker1 # [ 0.204999] APIC: Switched APIC routing to: physical x2apic683client # [ 0.218000] Spectre V2 : Enabling IBPB for BPF684lb # [ 0.047919] mem auto-init: stack:all(zero), heap alloc:on, heap free:off685worker2 # [ 0.207764] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns686client # [ 0.218994] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier687lb # [ 0.051333] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1688worker1 # [ 0.206809] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1689lb # [ 0.054674] allocated 3145728 bytes of page_ext690client # [ 0.219994] active return thunk: srso_alias_return_thunk691worker2 # [ 0.209420] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)692lb # [ 0.065106] ftrace: allocating 48787 entries in 192 pages693lb # [ 0.065108] ftrace: allocated 192 pages with 2 groups694worker2 # [ 0.210506] x86/cpu: User Mode Instruction Prevention (UMIP) activated695client # [ 0.221015] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'696worker1 # [ 0.207787] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns697lb # [ 0.065984] Dynamic Preempt: lazy698lb # [ 0.066125] rcu: Preemptible hierarchical RCU implementation.699client # [ 0.221994] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'700worker2 # [ 0.211548] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127701lb # [ 0.066126] rcu: RCU event tracing is enabled.702worker1 # [ 0.209465] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)703client # [ 0.222994] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'704worker2 # [ 0.212418] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0705lb # [ 0.066127] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.706worker1 # [ 0.210558] x86/cpu: User Mode Instruction Prevention (UMIP) activated707lb # [ 0.066128] Trampoline variant of Tasks RCU enabled.708client # [ 0.223994] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'709lb # [ 0.066129] Rude variant of Tasks RCU enabled.710worker1 # [ 0.211594] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127711worker2 # [ 0.213420] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto712lb # [ 0.066129] Tracing variant of Tasks RCU enabled.713client # [ 0.224994] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'714worker1 # [ 0.213132] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0715lb # [ 0.066130] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.716client # [ 0.225994] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'717worker2 # [ 0.214417] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl718lb # [ 0.066131] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1719worker2 # [ 0.215417] Transient Scheduler Attacks: Vulnerable: No microcode720client # [ 0.226993] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'721worker1 # [ 0.213466] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto722lb # [ 0.066147] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.723worker2 # [ 0.216417] Spectre V2 : Mitigation: Enhanced / Automatic IBRS724client # [ 0.227994] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256725lb # [ 0.066148] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.726worker1 # [ 0.215462] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl727worker2 # [ 0.217417] Speculative Return Stack Overflow: Mitigation: Safe RET728client # [ 0.228994] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64729lb # [ 0.066149] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.730worker1 # [ 0.216463] Transient Scheduler Attacks: Vulnerable: No microcode731client # [ 0.229994] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512732lb # [ 0.070624] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16733worker1 # [ 0.217462] Spectre V2 : Mitigation: Enhanced / Automatic IBRS734client # [ 0.230994] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024735lb # [ 0.070916] rcu: srcu_init: Setting srcu_struct sizes based on contention.736worker1 # [ 0.218462] Speculative Return Stack Overflow: Mitigation: Safe RET737client # [ 0.231994] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8738lb # [ 0.070922] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns739worker2 # [ 0.218417] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization740client # [ 0.232994] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.741lb # [ 0.071033] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)742worker2 # [ 0.220425] Spectre V2 : Enabling IBPB for BPF743lb # [ 0.074522] Console: colour VGA+ 80x25744lb # [ 0.074525] printk: legacy console [tty0] enabled745worker2 # [ 0.221418] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier746lb # [ 0.104168] printk: legacy console [ttyS0] enabled747worker1 # [ 0.219462] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization748worker2 # [ 0.222418] active return thunk: srso_alias_return_thunk749lb # [ 0.198473] ACPI: Core revision 20250807750worker1 # [ 0.221470] Spectre V2 : Enabling IBPB for BPF751worker2 # [ 0.223438] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'752lb # [ 0.199390] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns753worker1 # [ 0.223037] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier754worker2 # [ 0.224417] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'755lb # [ 0.200930] APIC: Switch to symmetric I/O mode setup756worker1 # [ 0.223463] active return thunk: srso_alias_return_thunk757worker2 # [ 0.225417] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'758lb # [ 0.201879] x2apic enabled759worker2 # [ 0.226417] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'760worker1 # [ 0.224340] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'761lb # [ 0.202673] APIC: Switched APIC routing to: physical x2apic762worker2 # [ 0.227417] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'763worker1 # [ 0.225462] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'764worker2 # [ 0.228417] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'765worker1 # [ 0.226462] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'766lb # [ 0.204479] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1767worker1 # [ 0.227462] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'768worker2 # [ 0.229417] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'769lb # [ 0.205511] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns770worker1 # [ 0.228463] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'771worker2 # [ 0.231310] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256772worker2 # [ 0.232124] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64773lb # [ 0.207205] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)774worker1 # [ 0.229462] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'775worker2 # [ 0.233122] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512776lb # [ 0.208290] x86/cpu: User Mode Instruction Prevention (UMIP) activated777worker1 # [ 0.230462] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'778worker2 # [ 0.234130] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024779lb # [ 0.209354] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127780worker1 # [ 0.231463] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256781worker2 # [ 0.234417] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8782lb # [ 0.210873] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0783worker1 # [ 0.232462] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64784worker1 # [ 0.233462] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512785worker2 # [ 0.235417] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.786lb # [ 0.211958] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto787worker1 # [ 0.234462] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024788lb # [ 0.213202] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl789worker1 # [ 0.235462] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8790lb # [ 0.214202] Transient Scheduler Attacks: Vulnerable: No microcode791worker1 # [ 0.236462] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.792lb # [ 0.215948] Spectre V2 : Mitigation: Enhanced / Automatic IBRS793lb # [ 0.216897] Speculative Return Stack Overflow: Mitigation: Safe RET794lb # [ 0.217997] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization795lb # [ 0.219209] Spectre V2 : Enabling IBPB for BPF796lb # [ 0.220202] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier797lb # [ 0.221202] active return thunk: srso_alias_return_thunk798lb # [ 0.222111] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799lb # [ 0.223201] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800lb # [ 0.224201] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801lb # [ 0.225201] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802client # [ 0.267610] Freeing SMP alternatives memory: 44K803lb # [ 0.226201] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'804client # [ 0.267997] pid_max: default: 32768 minimum: 301805lb # [ 0.227202] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'806client # [ 0.269079] LSM: initializing lsm=capability,landlock,yama,bpf,ima807lb # [ 0.228201] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'808client # [ 0.270098] landlock: Up and running.809client # [ 0.270711] Yama: becoming mindful.810lb # [ 0.229202] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256811client # [ 0.271615] LSM support for eBPF active812lb # [ 0.230937] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64813client # [ 0.272103] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)814lb # [ 0.231949] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512815lb # [ 0.232939] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024816client # [ 0.273031] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)817lb # [ 0.233932] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8818client # [ 0.275885] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)819lb # [ 0.234981] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.820client # [ 0.276505] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.276998] ... version: 2822client # [ 0.277690] ... bit width: 48823client # [ 0.277996] ... generic counters: 6824client # [ 0.278677] ... generic bitmap: 000000000000003f825client # [ 0.279040] ... fixed-purpose counters: 0826client # [ 0.279741] ... fixed-purpose bitmap: 0000000000000000827client # [ 0.279996] ... value mask: 0000ffffffffffff828client # [ 0.280867] ... max period: 00007fffffffffff829worker2 # [ 0.271356] Freeing SMP alternatives memory: 44K830client # [ 0.281657] ... global_ctrl mask: 000000000000003f831worker2 # [ 0.271992] pid_max: default: 32768 minimum: 301832client # [ 0.282107] signal: max sigframe size: 3376833worker2 # [ 0.272520] LSM: initializing lsm=capability,landlock,yama,bpf,ima834client # [ 0.282850] rcu: Hierarchical SRCU implementation.835worker2 # [ 0.273525] landlock: Up and running.836client # [ 0.283550] rcu: Max phase no-delay instances is 400.837worker1 # [ 0.271888] Freeing SMP alternatives memory: 44K838worker2 # [ 0.274418] Yama: becoming mindful.839worker1 # [ 0.272465] pid_max: default: 32768 minimum: 301840worker2 # [ 0.275621] LSM support for eBPF active841worker1 # [ 0.273558] LSM: initializing lsm=capability,landlock,yama,bpf,ima842worker2 # [ 0.276355] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843worker1 # [ 0.274567] landlock: Up and running.844worker1 # [ 0.275212] Yama: becoming mindful.845worker2 # [ 0.277457] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)846worker1 # [ 0.276104] LSM support for eBPF active847client # [ 0.288564] smp: Bringing up secondary CPUs ...848worker1 # [ 0.276592] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849client # [ 0.289010] smp: Brought up 1 node, 1 CPU850worker2 # [ 0.279911] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)851client # [ 0.289675] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)852worker1 # [ 0.277506] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)853worker2 # [ 0.281006] Performance Events: Fam17h+ core perfctr, AMD PMU driver.854worker2 # [ 0.281422] ... version: 2855client # [ 0.290155] Memory: 1454640K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)856worker2 # [ 0.282138] ... bit width: 48857worker1 # [ 0.280760] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)858client # [ 0.291265] devtmpfs: initialized859worker2 # [ 0.282420] ... generic counters: 6860worker2 # [ 0.283099] ... generic bitmap: 000000000000003f861client # [ 0.292109] x86/mm: Memory block size: 128MB862worker1 # [ 0.282056] Performance Events: Fam17h+ core perfctr, AMD PMU driver.863worker2 # [ 0.283430] ... fixed-purpose counters: 0864worker1 # [ 0.282471] ... version: 2865worker2 # [ 0.284142] ... fixed-purpose bitmap: 0000000000000000866worker1 # [ 0.283261] ... bit width: 48867client # [ 0.293911] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)868worker1 # [ 0.283465] ... generic counters: 6869worker2 # [ 0.284419] ... value mask: 0000ffffffffffff870worker2 # [ 0.285298] ... max period: 00007fffffffffff871worker1 # [ 0.284157] ... generic bitmap: 000000000000003f872client # [ 0.294886] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).873worker1 # [ 0.284465] ... fixed-purpose counters: 0874worker2 # [ 0.286064] ... global_ctrl mask: 000000000000003f875client # [ 0.296076] pinctrl core: initialized pinctrl subsystem876worker2 # [ 0.286533] signal: max sigframe size: 3376877worker1 # [ 0.285194] ... fixed-purpose bitmap: 0000000000000000878client # [ 0.297199] PM: RTC time: 05:11:32, date: 2026-09-23879worker2 # [ 0.287363] rcu: Hierarchical SRCU implementation.880worker1 # [ 0.285464] ... value mask: 0000ffffffffffff881worker2 # [ 0.288021] rcu: Max phase no-delay instances is 400.882worker1 # [ 0.286366] ... max period: 00007fffffffffff883worker1 # [ 0.287189] ... global_ctrl mask: 000000000000003f884worker1 # [ 0.287577] signal: max sigframe size: 3376885client # [ 0.300746] NET: Registered PF_NETLINK/PF_ROUTE protocol family886worker1 # [ 0.288438] rcu: Hierarchical SRCU implementation.887worker1 # [ 0.289074] rcu: Max phase no-delay instances is 400.888client # [ 0.302322] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations889lb # [ 0.270275] Freeing SMP alternatives memory: 44K890client # [ 0.303022] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations891worker2 # [ 0.293246] smp: Bringing up secondary CPUs ...892worker2 # [ 0.293995] smp: Brought up 1 node, 1 CPU893lb # [ 0.271032] pid_max: default: 32768 minimum: 301894client # [ 0.304280] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations895worker2 # [ 0.294420] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)896lb # [ 0.271828] LSM: initializing lsm=capability,landlock,yama,bpf,ima897client # [ 0.305006] audit: initializing netlink subsys (disabled)898lb # [ 0.272317] landlock: Up and running.899lb # [ 0.273624] Yama: becoming mindful.900worker1 # [ 0.294219] smp: Bringing up secondary CPUs ...901worker2 # [ 0.295538] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)902client # [ 0.305998] audit: type=2000 audit(1790140293.389:1): state=initialized audit_enabled=0 res=1903lb # [ 0.274417] LSM support for eBPF active904worker1 # [ 0.294480] smp: Brought up 1 node, 1 CPU905worker2 # [ 0.296731] devtmpfs: initialized906client # [ 0.307225] thermal_sys: Registered thermal governor 'fair_share'907lb # [ 0.275255] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)908worker1 # [ 0.295150] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)909worker2 # [ 0.297483] x86/mm: Memory block size: 128MB910client # [ 0.307227] thermal_sys: Registered thermal governor 'bang_bang'911lb # [ 0.276241] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)912client # [ 0.307997] thermal_sys: Registered thermal governor 'step_wise'913worker1 # [ 0.295677] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)914worker2 # [ 0.299290] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)915client # [ 0.308974] thermal_sys: Registered thermal governor 'user_space'916worker1 # [ 0.296756] devtmpfs: initialized917client # [ 0.309716] thermal_sys: Registered thermal governor 'power_allocator'918worker2 # [ 0.300301] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).919lb # [ 0.278921] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)920worker1 # [ 0.297608] x86/mm: Memory block size: 128MB921client # [ 0.310009] cpuidle: using governor menu922worker2 # [ 0.301529] pinctrl core: initialized pinctrl subsystem923lb # [ 0.279789] Performance Events: Fam17h+ core perfctr, AMD PMU driver.924client # [ 0.312202] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5925worker2 # [ 0.302674] PM: RTC time: 05:11:32, date: 2026-09-23926lb # [ 0.280206] ... version: 2927worker1 # [ 0.299491] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)928lb # [ 0.280923] ... bit width: 48929client # [ 0.313218] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]930lb # [ 0.281204] ... generic counters: 6931worker1 # [ 0.300490] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).932client # [ 0.313999] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry933lb # [ 0.281935] ... generic bitmap: 000000000000003f934worker1 # [ 0.301549] pinctrl core: initialized pinctrl subsystem935lb # [ 0.282258] ... fixed-purpose counters: 0936client # [ 0.315009] PCI: Using configuration type 1 for base access937worker2 # [ 0.306288] NET: Registered PF_NETLINK/PF_ROUTE protocol family938worker1 # [ 0.302673] PM: RTC time: 05:11:32, date: 2026-09-23939lb # [ 0.282945] ... fixed-purpose bitmap: 0000000000000000940client # [ 0.316088] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.941lb # [ 0.283205] ... value mask: 0000ffffffffffff942worker2 # [ 0.307659] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations943lb # [ 0.284096] ... max period: 00007fffffffffff944worker2 # [ 0.308445] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations945lb # [ 0.284923] ... global_ctrl mask: 000000000000003f946lb # [ 0.285333] signal: max sigframe size: 3376947worker1 # [ 0.306388] NET: Registered PF_NETLINK/PF_ROUTE protocol family948worker2 # [ 0.309710] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations949lb # [ 0.286149] rcu: Hierarchical SRCU implementation.950worker2 # [ 0.310429] audit: initializing netlink subsys (disabled)951worker1 # [ 0.307723] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations952lb # [ 0.286829] rcu: Max phase no-delay instances is 400.953worker2 # [ 0.311542] thermal_sys: Registered thermal governor 'fair_share'954worker1 # [ 0.308490] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations955worker2 # [ 0.311544] thermal_sys: Registered thermal governor 'bang_bang'956client # [ 0.323020] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages957worker2 # [ 0.312420] thermal_sys: Registered thermal governor 'step_wise'958worker1 # [ 0.309754] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations959worker2 # [ 0.313351] thermal_sys: Registered thermal governor 'user_space'960client # [ 0.323997] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page961worker1 # [ 0.310475] audit: initializing netlink subsys (disabled)962worker2 # [ 0.314152] audit: type=2000 audit(1790140293.406:1): state=initialized audit_enabled=0 res=1963worker1 # [ 0.311624] thermal_sys: Registered thermal governor 'fair_share'964lb # [ 0.291815] smp: Bringing up secondary CPUs ...965worker2 # [ 0.315423] thermal_sys: Registered thermal governor 'power_allocator'966worker1 # [ 0.311626] thermal_sys: Registered thermal governor 'bang_bang'967lb # [ 0.292218] smp: Brought up 1 node, 1 CPU968worker2 # [ 0.315443] cpuidle: using governor menu969worker1 # [ 0.312466] thermal_sys: Registered thermal governor 'step_wise'970lb # [ 0.292921] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)971worker2 # [ 0.317632] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5972worker1 # [ 0.313449] thermal_sys: Registered thermal governor 'user_space'973client # [ 0.328997] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages974worker1 # [ 0.314223] audit: type=2000 audit(1790140293.408:1): state=initialized audit_enabled=0 res=1975client # [ 0.329997] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page976worker2 # [ 0.318663] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]977lb # [ 0.293420] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)978worker1 # [ 0.316467] thermal_sys: Registered thermal governor 'power_allocator'979lb # [ 0.294502] devtmpfs: initialized980worker2 # [ 0.319423] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry981worker1 # [ 0.316482] cpuidle: using governor menu982worker2 # [ 0.320433] PCI: Using configuration type 1 for base access983lb # [ 0.295278] x86/mm: Memory block size: 128MB984worker1 # [ 0.318735] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5985worker2 # [ 0.321476] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.986lb # [ 0.297327] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)987worker1 # [ 0.319705] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]988lb # [ 0.298236] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).989worker1 # [ 0.320468] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry990worker1 # [ 0.321476] PCI: Using configuration type 1 for base access991lb # [ 0.299294] pinctrl core: initialized pinctrl subsystem992lb # [ 0.300461] PM: RTC time: 05:11:32, date: 2026-09-23993worker1 # [ 0.322602] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.994worker2 # [ 0.326712] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages995worker2 # [ 0.327421] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page996client # [ 0.341241] ACPI: Added _OSI(Module Device)997client # [ 0.341971] ACPI: Added _OSI(Processor Device)998lb # [ 0.304420] NET: Registered PF_NETLINK/PF_ROUTE protocol family999client # [ 0.342575] ACPI: Added _OSI(Processor Aggregator Device)1000worker1 # [ 0.327741] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1001lb # [ 0.305769] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1002worker2 # [ 0.332421] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1003worker1 # [ 0.328466] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1004lb # [ 0.306232] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1005worker2 # [ 0.333420] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1006lb # [ 0.307500] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1007lb # [ 0.308215] audit: initializing netlink subsys (disabled)1008client # [ 0.348355] ACPI: 1 ACPI AML tables successfully acquired and loaded1009lb # [ 0.309403] thermal_sys: Registered thermal governor 'fair_share'1010worker1 # [ 0.333466] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1011lb # [ 0.309406] thermal_sys: Registered thermal governor 'bang_bang'1012worker1 # [ 0.334464] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1013lb # [ 0.310208] audit: type=2000 audit(1790140293.434:1): state=initialized audit_enabled=0 res=11014lb # [ 0.312206] thermal_sys: Registered thermal governor 'step_wise'1015client # [ 0.352421] ACPI: Interpreter enabled1016lb # [ 0.312207] thermal_sys: Registered thermal governor 'user_space'1017client # [ 0.353011] ACPI: PM: (supports S0 S3 S4 S5)1018client # [ 0.353745] ACPI: Using IOAPIC for interrupt routing1019lb # [ 0.313204] thermal_sys: Registered thermal governor 'power_allocator'1020lb # [ 0.314242] cpuidle: using governor menu1021worker2 # [ 0.343904] ACPI: Added _OSI(Module Device)1022worker2 # [ 0.344422] ACPI: Added _OSI(Processor Device)1023lb # [ 0.316475] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51024worker2 # [ 0.345116] ACPI: Added _OSI(Processor Aggregator Device)1025client # [ 0.356051] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1026lb # [ 0.317477] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1027client # [ 0.356997] PCI: Using E820 reservations for host bridge windows1028lb # [ 0.318208] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1029lb # [ 0.319216] PCI: Using configuration type 1 for base access1030worker1 # [ 0.344848] ACPI: Added _OSI(Module Device)1031worker1 # [ 0.345466] ACPI: Added _OSI(Processor Device)1032client # [ 0.360123] ACPI: Enabled 2 GPEs in block 00 to 3F1033lb # [ 0.320408] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1034worker1 # [ 0.346220] ACPI: Added _OSI(Processor Aggregator Device)1035worker2 # [ 0.352845] ACPI: 1 ACPI AML tables successfully acquired and loaded1036lb # [ 0.325513] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1037lb # [ 0.326205] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1038client # [ 0.367637] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1039worker1 # [ 0.352381] ACPI: 1 ACPI AML tables successfully acquired and loaded1040worker2 # [ 0.356829] ACPI: Interpreter enabled1041worker2 # [ 0.357436] ACPI: PM: (supports S0 S3 S4 S5)1042client # [ 0.368003] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1043worker2 # [ 0.358110] ACPI: Using IOAPIC for interrupt routing1044client # [ 0.369075] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1045client # [ 0.370120] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1046lb # [ 0.331205] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1047client # [ 0.371388] PCI host bridge to bus 0000:001048worker2 # [ 0.360491] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049lb # [ 0.332205] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1050worker1 # [ 0.358436] ACPI: Interpreter enabled1051client # [ 0.372001] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1052worker2 # [ 0.361421] PCI: Using E820 reservations for host bridge windows1053worker1 # [ 0.358932] ACPI: PM: (supports S0 S3 S4 S5)1054worker1 # [ 0.359466] ACPI: Using IOAPIC for interrupt routing1055client # [ 0.372997] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1056client # [ 0.373996] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1057worker1 # [ 0.360343] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1058worker2 # [ 0.364499] ACPI: Enabled 2 GPEs in block 00 to 3F1059client # [ 0.374996] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060client # [ 0.375997] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1061client # [ 0.376996] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1062client # [ 0.377997] pci_bus 0000:00: root bus resource [bus 00-ff]1063worker1 # [ 0.365465] PCI: Using E820 reservations for host bridge windows1064worker1 # [ 0.366600] ACPI: Enabled 2 GPEs in block 00 to 3F1065client # [ 0.378998] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1066lb # [ 0.342667] ACPI: Added _OSI(Module Device)1067lb # [ 0.343206] ACPI: Added _OSI(Processor Device)1068client # [ 0.380464] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1069lb # [ 0.343979] ACPI: Added _OSI(Processor Aggregator Device)1070worker2 # [ 0.372714] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1071worker2 # [ 0.373428] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1072worker2 # [ 0.374512] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1073worker2 # [ 0.375542] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1074worker2 # [ 0.376777] PCI host bridge to bus 0000:001075worker1 # [ 0.374487] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1076worker2 # [ 0.377424] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1077worker1 # [ 0.375470] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1078worker2 # [ 0.378420] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1079client # [ 0.383086] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1080worker1 # [ 0.376543] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1081worker2 # [ 0.379420] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1082client # [ 0.384008] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1083lb # [ 0.352591] ACPI: 1 ACPI AML tables successfully acquired and loaded1084worker1 # [ 0.377587] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1085worker2 # [ 0.380420] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1086client # [ 0.384958] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1087worker1 # [ 0.378812] PCI host bridge to bus 0000:001088worker2 # [ 0.381420] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1089worker1 # [ 0.379469] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1090client # [ 0.386298] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1091worker2 # [ 0.382420] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1092worker1 # [ 0.380465] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1093lb # [ 0.356627] ACPI: Interpreter enabled1094worker2 # [ 0.383420] pci_bus 0000:00: root bus resource [bus 00-ff]1095worker1 # [ 0.381464] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1096lb # [ 0.357220] ACPI: PM: (supports S0 S3 S4 S5)1097client # [ 0.387734] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098worker2 # [ 0.384355] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1099lb # [ 0.357937] ACPI: Using IOAPIC for interrupt routing1100worker1 # [ 0.382465] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1101worker1 # [ 0.383465] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1102worker2 # [ 0.385839] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1103client # [ 0.391035] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1104worker1 # [ 0.384465] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1105client # [ 0.391893] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1106lb # [ 0.360273] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1107worker1 # [ 0.385474] pci_bus 0000:00: root bus resource [bus 00-ff]1108client # [ 0.392766] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1109lb # [ 0.361206] PCI: Using E820 reservations for host bridge windows1110worker1 # [ 0.386468] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1111client # [ 0.393860] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1112worker1 # [ 0.387911] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1113lb # [ 0.364321] ACPI: Enabled 2 GPEs in block 00 to 3F1114client # [ 0.395691] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1115client # [ 0.398036] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1116worker2 # [ 0.388540] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1117client # [ 0.398854] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1118worker2 # [ 0.389503] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1119client # [ 0.399732] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1120worker2 # [ 0.390443] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1121client # [ 0.400578] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1122worker2 # [ 0.391784] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1123lb # [ 0.372026] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1124worker1 # [ 0.390567] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1125lb # [ 0.373003] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1126worker2 # [ 0.393153] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1127worker1 # [ 0.391519] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1128lb # [ 0.374287] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1129client # [ 0.403005] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1130worker1 # [ 0.392488] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1131client # [ 0.403865] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1132lb # [ 0.375334] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1133worker2 # [ 0.395430] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1134client # [ 0.404760] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1135lb # [ 0.376573] PCI host bridge to bus 0000:001136worker1 # [ 0.393782] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1137worker2 # [ 0.396324] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1138lb # [ 0.377209] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1139worker2 # [ 0.397196] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1140client # [ 0.405581] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1141worker1 # [ 0.395190] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1142lb # [ 0.378204] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1143worker2 # [ 0.398264] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1144lb # [ 0.379205] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1145client # [ 0.407035] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1146lb # [ 0.380205] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1147worker2 # [ 0.399982] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1148worker1 # [ 0.397483] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1149client # [ 0.407878] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1150lb # [ 0.381204] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1151worker1 # [ 0.398315] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1152client # [ 0.408735] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1153worker1 # [ 0.399205] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1154lb # [ 0.382204] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1155worker2 # [ 0.401458] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1156client # [ 0.409003] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1157lb # [ 0.383205] pci_bus 0000:00: root bus resource [bus 00-ff]1158worker1 # [ 0.399471] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1159worker2 # [ 0.402304] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1160worker2 # [ 0.403166] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1161lb # [ 0.384204] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1162client # [ 0.410548] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1163worker1 # [ 0.401022] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1164lb # [ 0.385638] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1165worker2 # [ 0.404034] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166client # [ 0.412009] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1167worker1 # [ 0.402508] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1168client # [ 0.413002] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1169worker1 # [ 0.403416] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1170client # [ 0.414590] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1171worker1 # [ 0.404285] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1172worker2 # [ 0.406458] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1173worker2 # [ 0.407317] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1174worker1 # [ 0.405902] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1175worker2 # [ 0.408219] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1176client # [ 0.416009] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1177client # [ 0.417019] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1178worker2 # [ 0.409840] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1179client # [ 0.418577] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180worker1 # [ 0.408489] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1181lb # [ 0.388348] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1182worker1 # [ 0.409363] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1183worker2 # [ 0.411456] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1184lb # [ 0.389257] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1185client # [ 0.420009] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1186worker1 # [ 0.410231] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1187worker2 # [ 0.412293] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1188lb # [ 0.390228] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1189client # [ 0.421007] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1190worker2 # [ 0.413129] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1191worker1 # [ 0.411037] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1192lb # [ 0.391536] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1193worker2 # [ 0.413426] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194client # [ 0.422570] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1195worker1 # [ 0.412473] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1196lb # [ 0.392926] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1197worker2 # [ 0.414997] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1198worker1 # [ 0.413310] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1199client # [ 0.424009] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1200worker2 # [ 0.416312] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1201client # [ 0.425018] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1202worker1 # [ 0.414241] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1203worker2 # [ 0.417174] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1204lb # [ 0.396243] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1205worker1 # [ 0.415471] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1206client # [ 0.426541] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1207lb # [ 0.397173] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1208worker2 # [ 0.418018] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1209worker1 # [ 0.417014] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1210lb # [ 0.398013] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1211client # [ 0.428014] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1212lb # [ 0.399211] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1213worker1 # [ 0.418288] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1214client # [ 0.428896] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1215worker2 # [ 0.419433] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1216worker1 # [ 0.419193] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1217client # [ 0.429791] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1218lb # [ 0.400794] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1219worker2 # [ 0.420476] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1220worker1 # [ 0.420073] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1221worker2 # [ 0.422048] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222client # [ 0.431395] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1223lb # [ 0.402240] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1224worker1 # [ 0.421491] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1225worker2 # [ 0.423212] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1226lb # [ 0.403194] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1227client # [ 0.433036] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1228worker2 # [ 0.424151] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1229worker1 # [ 0.422464] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1230lb # [ 0.403994] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1231client # [ 0.433996] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1232client # [ 0.434960] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1233worker2 # [ 0.424994] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1234worker1 # [ 0.424020] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1235lb # [ 0.405601] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1236client # [ 0.436434] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1237worker2 # [ 0.426360] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1238worker1 # [ 0.425316] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1239lb # [ 0.407242] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1240worker1 # [ 0.426274] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1241client # [ 0.437782] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1242worker2 # [ 0.427195] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1243lb # [ 0.408105] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1244client # [ 0.438217] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1245worker1 # [ 0.427869] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1246lb # [ 0.408990] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1247worker2 # [ 0.428809] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1248worker1 # [ 0.429475] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1249client # [ 0.440331] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1250lb # [ 0.410677] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1251worker2 # [ 0.430479] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1252worker2 # [ 0.431341] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1253client # [ 0.441198] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1254worker1 # [ 0.430443] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1255worker2 # [ 0.432206] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1256client # [ 0.442662] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1257worker1 # [ 0.432032] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1258lb # [ 0.413241] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1259worker2 # [ 0.433871] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1260lb # [ 0.414240] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1261worker1 # [ 0.433492] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1262worker1 # [ 0.434365] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1263lb # [ 0.415253] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1264lb # [ 0.416210] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1265worker1 # [ 0.435225] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1266worker2 # [ 0.435428] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1267worker2 # [ 0.436426] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1268worker1 # [ 0.436002] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1269lb # [ 0.417765] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1270worker2 # [ 0.437419] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1271lb # [ 0.419093] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1272worker2 # [ 0.439036] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1273lb # [ 0.420004] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1274worker1 # [ 0.437486] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1275worker2 # [ 0.440036] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1276worker1 # [ 0.438472] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1277lb # [ 0.421673] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1278worker2 # [ 0.440632] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1279worker1 # [ 0.439487] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1280client # [ 0.443263] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1281worker2 # [ 0.441977] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1282lb # [ 0.423137] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1283worker1 # [ 0.441046] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1284client # [ 0.445659] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1285lb # [ 0.424025] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1286worker2 # [ 0.442609] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1287worker1 # [ 0.442051] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1288client # [ 0.446277] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1289worker2 # [ 0.443997] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1290lb # [ 0.425729] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1291worker1 # [ 0.442668] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1292client # [ 0.447371] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1293worker1 # [ 0.444064] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1294lb # [ 0.427217] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1295client # [ 0.448302] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1296lb # [ 0.428210] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1297worker1 # [ 0.444660] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1298client # [ 0.450025] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1299worker1 # [ 0.446001] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1300client # [ 0.450909] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1301lb # [ 0.429764] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1302client # [ 0.452035] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1303lb # [ 0.430993] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1304lb # [ 0.432001] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1305client # [ 0.453622] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1306lb # [ 0.433686] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1307worker2 # [ 0.444641] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1308client # [ 0.457501] ACPI: PCI: Interrupt link LNKA configured for IRQ 101309lb # [ 0.435249] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1310client # [ 0.458103] ACPI: PCI: Interrupt link LNKB configured for IRQ 101311worker2 # [ 0.446987] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1312lb # [ 0.436214] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1313client # [ 0.459097] ACPI: PCI: Interrupt link LNKC configured for IRQ 111314lb # [ 0.437189] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1315worker2 # [ 0.447712] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1316client # [ 0.460095] ACPI: PCI: Interrupt link LNKD configured for IRQ 111317client # [ 0.461097] ACPI: PCI: Interrupt link LNKE configured for IRQ 101318worker1 # [ 0.446728] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1319worker2 # [ 0.448704] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1320lb # [ 0.438788] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1321client # [ 0.462094] ACPI: PCI: Interrupt link LNKF configured for IRQ 101322worker1 # [ 0.448996] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1323worker2 # [ 0.449797] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1324client # [ 0.463095] ACPI: PCI: Interrupt link LNKG configured for IRQ 111325lb # [ 0.440212] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1326worker1 # [ 0.449700] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1327client # [ 0.464094] ACPI: PCI: Interrupt link LNKH configured for IRQ 111328lb # [ 0.441117] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1329worker2 # [ 0.451432] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1330client # [ 0.465045] ACPI: PCI: Interrupt link GSIA configured for IRQ 161331lb # [ 0.441924] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1332worker2 # [ 0.452296] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1333worker1 # [ 0.450757] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1334client # [ 0.466013] ACPI: PCI: Interrupt link GSIB configured for IRQ 171335worker2 # [ 0.453454] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1336worker1 # [ 0.451716] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1337client # [ 0.467011] ACPI: PCI: Interrupt link GSIC configured for IRQ 181338lb # [ 0.442792] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1339client # [ 0.467942] ACPI: PCI: Interrupt link GSID configured for IRQ 191340worker2 # [ 0.455035] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1341lb # [ 0.443856] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1342worker1 # [ 0.453346] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1343client # [ 0.468774] ACPI: PCI: Interrupt link GSIE configured for IRQ 201344worker1 # [ 0.454093] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1345client # [ 0.469730] ACPI: PCI: Interrupt link GSIF configured for IRQ 211346lb # [ 0.444969] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1347client # [ 0.470008] ACPI: PCI: Interrupt link GSIG configured for IRQ 221348worker1 # [ 0.454782] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1349client # [ 0.470963] ACPI: PCI: Interrupt link GSIH configured for IRQ 231350lb # [ 0.446724] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1351worker2 # [ 0.458562] ACPI: PCI: Interrupt link LNKA configured for IRQ 101352worker1 # [ 0.456032] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1353client # [ 0.472734] iommu: Default domain type: Translated1354lb # [ 0.447456] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1355worker2 # [ 0.459530] ACPI: PCI: Interrupt link LNKB configured for IRQ 101356client # [ 0.473004] iommu: DMA domain TLB invalidation policy: lazy mode1357worker2 # [ 0.460523] ACPI: PCI: Interrupt link LNKC configured for IRQ 111358lb # [ 0.448772] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1359client # [ 0.474281] ACPI: bus type USB registered1360worker2 # [ 0.461520] ACPI: PCI: Interrupt link LNKD configured for IRQ 111361client # [ 0.475054] usbcore: registered new interface driver usbfs1362worker2 # [ 0.462517] ACPI: PCI: Interrupt link LNKE configured for IRQ 101363worker1 # [ 0.459514] ACPI: PCI: Interrupt link LNKA configured for IRQ 101364client # [ 0.475956] usbcore: registered new interface driver hub1365worker2 # [ 0.463515] ACPI: PCI: Interrupt link LNKF configured for IRQ 101366client # [ 0.476663] usbcore: registered new device driver usb1367worker1 # [ 0.460572] ACPI: PCI: Interrupt link LNKB configured for IRQ 101368worker2 # [ 0.464557] ACPI: PCI: Interrupt link LNKG configured for IRQ 111369worker1 # [ 0.461564] ACPI: PCI: Interrupt link LNKC configured for IRQ 111370client # [ 0.477931] NetLabel: Initializing1371worker2 # [ 0.465530] ACPI: PCI: Interrupt link LNKH configured for IRQ 111372worker1 # [ 0.462562] ACPI: PCI: Interrupt link LNKD configured for IRQ 111373client # [ 0.478431] NetLabel: domain hash size = 1281374worker2 # [ 0.466457] ACPI: PCI: Interrupt link GSIA configured for IRQ 161375client # [ 0.478996] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1376worker1 # [ 0.463575] ACPI: PCI: Interrupt link LNKE configured for IRQ 101377worker2 # [ 0.467427] ACPI: PCI: Interrupt link GSIB configured for IRQ 171378client # [ 0.479927] NetLabel: unlabeled traffic allowed by default1379worker1 # [ 0.464573] ACPI: PCI: Interrupt link LNKF configured for IRQ 101380client # [ 0.480670] PCI: Using ACPI for IRQ routing1381worker2 # [ 0.468436] ACPI: PCI: Interrupt link GSIC configured for IRQ 181382worker1 # [ 0.465564] ACPI: PCI: Interrupt link LNKG configured for IRQ 111383worker2 # [ 0.469437] ACPI: PCI: Interrupt link GSID configured for IRQ 191384worker1 # [ 0.466564] ACPI: PCI: Interrupt link LNKH configured for IRQ 111385worker2 # [ 0.470432] ACPI: PCI: Interrupt link GSIE configured for IRQ 201386worker1 # [ 0.467503] ACPI: PCI: Interrupt link GSIA configured for IRQ 161387worker2 # [ 0.471432] ACPI: PCI: Interrupt link GSIF configured for IRQ 211388worker1 # [ 0.468482] ACPI: PCI: Interrupt link GSIB configured for IRQ 171389worker2 # [ 0.472431] ACPI: PCI: Interrupt link GSIG configured for IRQ 221390worker1 # [ 0.469472] ACPI: PCI: Interrupt link GSIC configured for IRQ 181391lb # [ 0.449428] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1392worker2 # [ 0.473436] ACPI: PCI: Interrupt link GSIH configured for IRQ 231393worker1 # [ 0.470481] ACPI: PCI: Interrupt link GSID configured for IRQ 191394lb # [ 0.451746] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1395worker1 # [ 0.471477] ACPI: PCI: Interrupt link GSIE configured for IRQ 201396worker2 # [ 0.475507] iommu: Default domain type: Translated1397worker1 # [ 0.472477] ACPI: PCI: Interrupt link GSIF configured for IRQ 211398worker2 # [ 0.476335] iommu: DMA domain TLB invalidation policy: lazy mode1399lb # [ 0.452489] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1400worker1 # [ 0.473449] ACPI: PCI: Interrupt link GSIG configured for IRQ 221401worker2 # [ 0.477441] ACPI: bus type USB registered1402worker1 # [ 0.474230] ACPI: PCI: Interrupt link GSIH configured for IRQ 231403lb # [ 0.453489] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1404worker2 # [ 0.478243] usbcore: registered new interface driver usbfs1405worker2 # [ 0.479162] usbcore: registered new interface driver hub1406worker1 # [ 0.475508] iommu: Default domain type: Translated1407lb # [ 0.454494] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1408worker2 # [ 0.479431] usbcore: registered new device driver usb1409worker1 # [ 0.476333] iommu: DMA domain TLB invalidation policy: lazy mode1410lb # [ 0.456155] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1411worker1 # [ 0.477530] ACPI: bus type USB registered1412worker2 # [ 0.481162] NetLabel: Initializing1413worker2 # [ 0.481426] NetLabel: domain hash size = 1281414worker1 # [ 0.478295] usbcore: registered new interface driver usbfs1415lb # [ 0.456897] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1416worker2 # [ 0.482156] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1417worker1 # [ 0.479154] usbcore: registered new interface driver hub1418lb # [ 0.457528] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1419worker2 # [ 0.482461] NetLabel: unlabeled traffic allowed by default1420worker1 # [ 0.479487] usbcore: registered new device driver usb1421worker2 # [ 0.483389] PCI: Using ACPI for IRQ routing1422worker1 # [ 0.481168] NetLabel: Initializing1423lb # [ 0.459218] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1424worker1 # [ 0.481472] NetLabel: domain hash size = 1281425worker1 # [ 0.482237] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker1 # [ 0.482502] NetLabel: unlabeled traffic allowed by default1427worker1 # [ 0.483454] PCI: Using ACPI for IRQ routing1428lb # [ 0.463212] ACPI: PCI: Interrupt link LNKA configured for IRQ 101429lb # [ 0.464310] ACPI: PCI: Interrupt link LNKB configured for IRQ 101430lb # [ 0.465306] ACPI: PCI: Interrupt link LNKC configured for IRQ 111431lb # [ 0.466313] ACPI: PCI: Interrupt link LNKD configured for IRQ 111432lb # [ 0.467307] ACPI: PCI: Interrupt link LNKE configured for IRQ 101433lb # [ 0.468313] ACPI: PCI: Interrupt link LNKF configured for IRQ 101434lb # [ 0.469306] ACPI: PCI: Interrupt link LNKG configured for IRQ 111435lb # [ 0.470305] ACPI: PCI: Interrupt link LNKH configured for IRQ 111436lb # [ 0.471252] ACPI: PCI: Interrupt link GSIA configured for IRQ 161437lb # [ 0.472221] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438lb # [ 0.473224] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439lb # [ 0.474224] ACPI: PCI: Interrupt link GSID configured for IRQ 191440lb # [ 0.475229] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441lb # [ 0.476232] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442lb # [ 0.477218] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443lb # [ 0.478219] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444lb # [ 0.480350] iommu: Default domain type: Translated1445lb # [ 0.481212] iommu: DMA domain TLB invalidation policy: lazy mode1446lb # [ 0.482517] ACPI: bus type USB registered1447lb # [ 0.483276] usbcore: registered new interface driver usbfs1448client # [ 0.523569] pci 0000:00:01.0: vgaarb: setting as boot VGA device1449lb # [ 0.484224] usbcore: registered new interface driver hub1450client # [ 0.523993] pci 0000:00:01.0: vgaarb: bridge control possible1451lb # [ 0.485146] usbcore: registered new device driver usb1452client # [ 0.523993] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1453client # [ 0.524001] vgaarb: loaded1454lb # [ 0.486980] NetLabel: Initializing1455client # [ 0.524744] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01456lb # [ 0.487205] NetLabel: domain hash size = 1281457client # [ 0.524996] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1458lb # [ 0.487976] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1459lb # [ 0.488259] NetLabel: unlabeled traffic allowed by default1460lb # [ 0.489219] PCI: Using ACPI for IRQ routing1461client # [ 0.529081] clocksource: Switched to clocksource kvm-clock1462client # [ 0.532598] VFS: Disk quotas dquot_6.6.01463client # [ 0.533314] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464client # [ 0.534698] pnp: PnP ACPI init1465client # [ 0.535566] ACPI: IRQ 4 override to edge(!), high(!)1466client # [ 0.536557] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1467worker2 # [ 0.527950] pci 0000:00:01.0: vgaarb: setting as boot VGA device1468client # [ 0.538094] pnp: PnP ACPI: found 5 devices1469worker2 # [ 0.528416] pci 0000:00:01.0: vgaarb: bridge control possible1470worker2 # [ 0.528416] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471worker2 # [ 0.528425] vgaarb: loaded1472worker1 # [ 0.527554] pci 0000:00:01.0: vgaarb: setting as boot VGA device1473worker2 # [ 0.529221] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01474worker1 # [ 0.528461] pci 0000:00:01.0: vgaarb: bridge control possible1475worker2 # [ 0.530033] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1476worker1 # [ 0.528461] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1477worker1 # [ 0.528469] vgaarb: loaded1478worker1 # [ 0.529241] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01479worker1 # [ 0.529467] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1480client # [ 0.545855] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1481client # [ 0.547315] clocksource: Switched to clocksource acpi_pm1482worker2 # [ 0.534504] clocksource: Switched to clocksource kvm-clock1483client # [ 0.548287] NET: Registered PF_INET protocol family1484client # [ 0.549526] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1485worker1 # [ 0.532548] clocksource: Switched to clocksource kvm-clock1486worker2 # [ 0.538121] VFS: Disk quotas dquot_6.6.01487worker2 # [ 0.538883] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1488worker2 # [ 0.540335] pnp: PnP ACPI init1489worker1 # [ 0.536365] VFS: Disk quotas dquot_6.6.01490worker2 # [ 0.541129] ACPI: IRQ 4 override to edge(!), high(!)1491worker1 # [ 0.537129] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1492worker2 # [ 0.542111] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1493worker1 # [ 0.538517] pnp: PnP ACPI init1494worker2 # [ 0.543765] pnp: PnP ACPI: found 5 devices1495worker1 # [ 0.539343] ACPI: IRQ 4 override to edge(!), high(!)1496worker1 # [ 0.540316] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1497worker1 # [ 0.541945] pnp: PnP ACPI: found 5 devices1498worker2 # [ 0.551602] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1499worker2 # [ 0.553102] clocksource: Switched to clocksource acpi_pm1500client # [ 0.566054] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1501worker2 # [ 0.554059] NET: Registered PF_INET protocol family1502worker1 # [ 0.549593] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503client # [ 0.567608] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1504worker2 # [ 0.555222] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1505worker1 # [ 0.551091] clocksource: Switched to clocksource acpi_pm1506client # [ 0.568919] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1507worker1 # [ 0.552068] NET: Registered PF_INET protocol family1508client # [ 0.570289] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1509worker1 # [ 0.553311] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1510client # [ 0.571595] TCP: Hash tables configured (established 16384 bind 16384)1511client # [ 0.572751] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1512client # [ 0.574071] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1513client # [ 0.575183] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1514client # [ 0.576453] NET: Registered PF_UNIX/PF_LOCAL protocol family1515lb # [ 0.538657] pci 0000:00:01.0: vgaarb: setting as boot VGA device1516client # [ 0.577455] NET: Registered PF_XDP protocol family1517lb # [ 0.539200] pci 0000:00:01.0: vgaarb: bridge control possible1518client # [ 0.578306] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1519lb # [ 0.539200] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1520client # [ 0.579339] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1521lb # [ 0.539211] vgaarb: loaded1522client # [ 0.580377] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1523lb # [ 0.540038] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01524lb # [ 0.540813] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1525client # [ 0.581549] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1526client # [ 0.582668] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1527client # [ 0.583837] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1528client # [ 0.585612] ACPI: \_SB_.GSIA: Enabled at IRQ 161529worker2 # [ 0.572102] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1530lb # [ 0.544325] clocksource: Switched to clocksource kvm-clock1531worker2 # [ 0.573654] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1532lb # [ 0.546169] VFS: Disk quotas dquot_6.6.01533client # [ 0.587819] ACPI: \_SB_.GSIB: Enabled at IRQ 171534worker1 # [ 0.570037] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1535worker2 # [ 0.575013] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1536worker1 # [ 0.571561] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1537worker2 # [ 0.576365] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1538client # [ 0.589756] ACPI: \_SB_.GSIC: Enabled at IRQ 181539worker2 # [ 0.577600] TCP: Hash tables configured (established 16384 bind 16384)1540lb # [ 0.548960] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1541worker1 # [ 0.572854] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1542lb # [ 0.550358] pnp: PnP ACPI init1543client # [ 0.592002] ACPI: \_SB_.GSID: Enabled at IRQ 191544worker2 # [ 0.578680] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1545worker1 # [ 0.574151] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1546lb # [ 0.551180] ACPI: IRQ 4 override to edge(!), high(!)1547worker2 # [ 0.579896] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker1 # [ 0.575355] TCP: Hash tables configured (established 16384 bind 16384)1549client # [ 0.593687] PCI: CLS 0 bytes, default 641550lb # [ 0.552219] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1551worker2 # [ 0.580992] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1552worker1 # [ 0.576472] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1553worker2 # [ 0.582152] NET: Registered PF_UNIX/PF_LOCAL protocol family1554lb # [ 0.553763] pnp: PnP ACPI: found 5 devices1555client # [ 0.594588] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1556worker1 # [ 0.577717] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1557worker2 # [ 0.583058] NET: Registered PF_XDP protocol family1558client # [ 0.596314] Trying to unpack rootfs image as initramfs...1559worker1 # [ 0.578764] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1560worker2 # [ 0.583869] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1561worker1 # [ 0.579924] NET: Registered PF_UNIX/PF_LOCAL protocol family1562worker2 # [ 0.584891] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1563worker1 # [ 0.580895] NET: Registered PF_XDP protocol family1564worker2 # [ 0.585913] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565worker1 # [ 0.581753] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1566worker2 # [ 0.587036] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1567worker1 # [ 0.582778] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1568worker2 # [ 0.588180] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker1 # [ 0.583792] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1570worker2 # [ 0.589316] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1571worker1 # [ 0.584895] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1572lb # [ 0.561429] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1573worker2 # [ 0.591012] ACPI: \_SB_.GSIA: Enabled at IRQ 161574worker1 # [ 0.586007] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1575lb # [ 0.563114] clocksource: Switched to clocksource acpi_pm1576worker1 # [ 0.587114] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1577lb # [ 0.564262] NET: Registered PF_INET protocol family1578worker2 # [ 0.593296] ACPI: \_SB_.GSIB: Enabled at IRQ 171579worker1 # [ 0.588869] ACPI: \_SB_.GSIA: Enabled at IRQ 161580lb # [ 0.565568] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1581worker2 # [ 0.595541] ACPI: \_SB_.GSIC: Enabled at IRQ 181582worker1 # [ 0.591663] ACPI: \_SB_.GSIB: Enabled at IRQ 171583worker2 # [ 0.597556] ACPI: \_SB_.GSID: Enabled at IRQ 191584worker1 # [ 0.593944] ACPI: \_SB_.GSIC: Enabled at IRQ 181585worker2 # [ 0.599369] PCI: CLS 0 bytes, default 641586worker1 # [ 0.595982] ACPI: \_SB_.GSID: Enabled at IRQ 191587worker2 # [ 0.600339] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1588worker2 # [ 0.602133] Trying to unpack rootfs image as initramfs...1589worker1 # [ 0.597739] PCI: CLS 0 bytes, default 641590worker1 # [ 0.598676] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1591worker1 # [ 0.600553] Trying to unpack rootfs image as initramfs...1592lb # [ 0.582634] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1593lb # [ 0.584402] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1594lb # [ 0.585837] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1595lb # [ 0.587260] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1596lb # [ 0.588614] TCP: Hash tables configured (established 16384 bind 16384)1597lb # [ 0.589763] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1598lb # [ 0.591085] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1599lb # [ 0.592257] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1600lb # [ 0.593566] NET: Registered PF_UNIX/PF_LOCAL protocol family1601lb # [ 0.594593] NET: Registered PF_XDP protocol family1602lb # [ 0.595479] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1603lb # [ 0.596531] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1604lb # [ 0.597596] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1605client # [ 0.639818] Initialise system trusted keyrings1606lb # [ 0.598643] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1607client # [ 0.640952] workingset: timestamp_bits=40 max_order=19 bucket_order=01608lb # [ 0.599718] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1609lb # [ 0.600812] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1610lb # [ 0.602673] ACPI: \_SB_.GSIA: Enabled at IRQ 161611lb # [ 0.605042] ACPI: \_SB_.GSIB: Enabled at IRQ 171612lb # [ 0.607021] ACPI: \_SB_.GSIC: Enabled at IRQ 181613lb # [ 0.609450] ACPI: \_SB_.GSID: Enabled at IRQ 191614lb # [ 0.611178] PCI: CLS 0 bytes, default 641615lb # [ 0.612100] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1616lb # [ 0.613879] Trying to unpack rootfs image as initramfs...1617worker2 # [ 0.646024] Initialise system trusted keyrings1618worker1 # [ 0.642337] Initialise system trusted keyrings1619worker2 # [ 0.648943] workingset: timestamp_bits=40 max_order=19 bucket_order=01620worker1 # [ 0.645304] workingset: timestamp_bits=40 max_order=19 bucket_order=01621client # [ 0.664562] Key type asymmetric registered1622client # [ 0.665307] Asymmetric key parser 'x509' registered1623client # [ 0.666186] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1624client # [ 0.671580] io scheduler mq-deadline registered1625client # [ 0.672391] io scheduler kyber registered1626client # [ 0.674956] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1627client # [ 0.676300] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1628client # [ 0.682093] Linux agpgart interface v0.1031629client # [ 0.682902] ACPI: bus type drm_connector registered1630worker2 # [ 0.670347] Key type asymmetric registered1631worker2 # [ 0.672909] Asymmetric key parser 'x509' registered1632client # [ 0.685880] usbcore: registered new interface driver usbserial_generic1633worker1 # [ 0.669431] Key type asymmetric registered1634client # [ 0.686953] usbserial: USB Serial support registered for generic1635worker2 # [ 0.673754] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1636worker1 # [ 0.670185] Asymmetric key parser 'x509' registered1637worker1 # [ 0.671131] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638worker2 # [ 0.677044] io scheduler mq-deadline registered1639worker2 # [ 0.677838] io scheduler kyber registered1640client # [ 0.690456] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1641client # [ 0.690456] Please enable it if your BIOS has the CPPC option.1642worker1 # [ 0.674366] io scheduler mq-deadline registered1643worker1 # [ 0.675149] io scheduler kyber registered1644client # [ 0.692719] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1645worker2 # [ 0.680457] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1646worker2 # [ 0.681746] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1647worker1 # [ 0.677879] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1648client # [ 0.695569] drop_monitor: Initializing network drop monitor service1649client # [ 0.696697] NET: Registered PF_INET6 protocol family1650worker1 # [ 0.679152] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1651client # [ 0.699944] Segment Routing with IPv61652worker2 # [ 0.687740] Linux agpgart interface v0.1031653worker2 # [ 0.688621] ACPI: bus type drm_connector registered1654lb # [ 0.660751] Initialise system trusted keyrings1655worker1 # [ 0.684878] Linux agpgart interface v0.1031656client # [ 0.703463] In-situ OAM (IOAM) with IPv61657worker1 # [ 0.685778] ACPI: bus type drm_connector registered1658lb # [ 0.661780] workingset: timestamp_bits=40 max_order=19 bucket_order=01659client # [ 0.704490] IPI shorthand broadcast: enabled1660worker2 # [ 0.691337] usbcore: registered new interface driver usbserial_generic1661worker2 # [ 0.692416] usbserial: USB Serial support registered for generic1662worker1 # [ 0.688774] usbcore: registered new interface driver usbserial_generic1663worker1 # [ 0.689892] usbserial: USB Serial support registered for generic1664worker2 # [ 0.696904] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1665worker1 # [ 0.692270] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1666worker2 # [ 0.696904] Please enable it if your BIOS has the CPPC option.1667worker1 # [ 0.692270] Please enable it if your BIOS has the CPPC option.1668worker2 # [ 0.699138] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1669worker1 # [ 0.694540] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670client # [ 0.712695] sched_clock: Marking stable (579014297, 133417803)->(785022015, -72589915)1671worker2 # [ 0.702076] drop_monitor: Initializing network drop monitor service1672worker2 # [ 0.703327] NET: Registered PF_INET6 protocol family1673worker1 # [ 0.699393] drop_monitor: Initializing network drop monitor service1674client # [ 0.717618] registered taskstats version 11675worker1 # [ 0.700665] NET: Registered PF_INET6 protocol family1676client # [ 0.718619] Loading compiled-in X.509 certificates1677worker2 # [ 0.707368] Segment Routing with IPv61678worker1 # [ 0.703819] Segment Routing with IPv61679worker2 # [ 0.709913] In-situ OAM (IOAM) with IPv61680worker2 # [ 0.710988] IPI shorthand broadcast: enabled1681worker1 # [ 0.706281] In-situ OAM (IOAM) with IPv61682worker1 # [ 0.707413] IPI shorthand broadcast: enabled1683lb # [ 0.685467] Key type asymmetric registered1684lb # [ 0.686239] Asymmetric key parser 'x509' registered1685lb # [ 0.687158] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1686worker2 # [ 0.718450] sched_clock: Marking stable (584014961, 133873241)->(791009925, -73121723)1687lb # [ 0.692488] io scheduler mq-deadline registered1688worker1 # [ 0.715900] sched_clock: Marking stable (581014430, 134231429)->(790506217, -75260358)1689lb # [ 0.693307] io scheduler kyber registered1690client # [ 0.735092] Demotion targets for Node 0: null1691worker1 # [ 0.720346] registered taskstats version 11692worker2 # [ 0.725099] registered taskstats version 11693client # [ 0.738480] Key type .fscrypt registered1694lb # [ 0.696942] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1695worker1 # [ 0.721349] Loading compiled-in X.509 certificates1696worker2 # [ 0.726079] Loading compiled-in X.509 certificates1697client # [ 0.739126] Key type fscrypt-provisioning registered1698client # [ 0.740008] ima: No TPM chip found, activating TPM-bypass!1699lb # [ 0.698196] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1700client # [ 0.742444] ima: Allocated hash algorithm: sha11701client # [ 0.743200] ima: No architecture policies found1702lb # [ 0.702135] Linux agpgart interface v0.1031703lb # [ 0.702947] ACPI: bus type drm_connector registered1704client # [ 0.745444] PM: Magic number: 10:890:1631705client # [ 0.747052] RAS: Correctable Errors collector initialized.1706lb # [ 0.708109] usbcore: registered new interface driver usbserial_generic1707lb # [ 0.709220] usbserial: USB Serial support registered for generic1708lb # [ 0.710244] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1709lb # [ 0.710244] Please enable it if your BIOS has the CPPC option.1710lb # [ 0.712550] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1711client # [ 0.756109] clk: Disabling unused clocks1712worker2 # [ 0.744904] Demotion targets for Node 0: null1713worker1 # [ 0.740269] Demotion targets for Node 0: null1714worker2 # [ 0.745798] Key type .fscrypt registered1715worker1 # [ 0.742303] Key type .fscrypt registered1716lb # [ 0.718452] drop_monitor: Initializing network drop monitor service1717client # [ 0.760468] PM: genpd: Disabling unused power domains1718worker1 # [ 0.742987] Key type fscrypt-provisioning registered1719lb # [ 0.719603] NET: Registered PF_INET6 protocol family1720worker1 # [ 0.743915] ima: No TPM chip found, activating TPM-bypass!1721worker2 # [ 0.748893] Key type fscrypt-provisioning registered1722worker2 # [ 0.749826] ima: No TPM chip found, activating TPM-bypass!1723worker2 # [ 0.750788] ima: Allocated hash algorithm: sha11724lb # [ 0.722875] Segment Routing with IPv61725worker2 # [ 0.751648] ima: No architecture policies found1726worker1 # [ 0.748256] ima: Allocated hash algorithm: sha11727worker1 # [ 0.749150] ima: No architecture policies found1728worker2 # [ 0.753907] PM: Magic number: 10:890:1631729lb # [ 0.726338] In-situ OAM (IOAM) with IPv61730worker2 # [ 0.755556] RAS: Correctable Errors collector initialized.1731worker1 # [ 0.751260] PM: Magic number: 10:890:1631732lb # [ 0.727445] IPI shorthand broadcast: enabled1733worker1 # [ 0.752956] RAS: Correctable Errors collector initialized.1734worker2 # [ 0.764586] clk: Disabling unused clocks1735lb # [ 0.735584] sched_clock: Marking stable (603014019, 132278968)->(809438263, -74145276)1736worker1 # [ 0.761878] clk: Disabling unused clocks1737worker2 # [ 0.767922] PM: genpd: Disabling unused power domains1738lb # [ 0.740535] registered taskstats version 11739worker1 # [ 0.765266] PM: genpd: Disabling unused power domains1740lb # [ 0.741568] Loading compiled-in X.509 certificates1741lb # [ 0.757927] Demotion targets for Node 0: null1742lb # [ 0.761342] Key type .fscrypt registered1743lb # [ 0.762034] Key type fscrypt-provisioning registered1744lb # [ 0.763041] ima: No TPM chip found, activating TPM-bypass!1745lb # [ 0.765406] ima: Allocated hash algorithm: sha11746lb # [ 0.766217] ima: No architecture policies found1747lb # [ 0.770307] PM: Magic number: 10:890:1631748lb # [ 0.771994] RAS: Correctable Errors collector initialized.1749lb # [ 0.780107] clk: Disabling unused clocks1750lb # [ 0.784320] PM: genpd: Disabling unused power domains1751client # [ 0.925331] Freeing initrd memory: 29056K1752client # [ 0.928551] Freeing unused decrypted memory: 2028K1753client # [ 0.931226] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.932308] Write protecting the kernel read-only data: 32768k1755client # [ 0.934217] Freeing unused kernel image (text/rodata gap) memory: 1184K1756client # [ 0.935764] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker2 # [ 0.938044] Freeing initrd memory: 29060K1758worker2 # [ 0.941371] Freeing unused decrypted memory: 2028K1759worker2 # [ 0.944118] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.945224] Write protecting the kernel read-only data: 32768k1761worker2 # [ 0.947133] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker2 # [ 0.948674] Freeing unused kernel image (rodata/data gap) memory: 720K1763worker1 # [ 0.946993] Freeing initrd memory: 29056K1764worker1 # [ 0.950367] Freeing unused decrypted memory: 2028K1765worker1 # [ 0.953088] Freeing unused kernel image (initmem) memory: 3652K1766worker1 # [ 0.954189] Write protecting the kernel read-only data: 32768k1767worker1 # [ 0.956148] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker1 # [ 0.957697] Freeing unused kernel image (rodata/data gap) memory: 720K1769client # [ 0.986752] x86/mm: Checked W+X mappings: passed, no W+X pages found.1770client # [ 0.987878] Run /init as init process1771lb # [ 0.951896] Freeing initrd memory: 29064K1772lb # [ 0.955142] Freeing unused decrypted memory: 2028K1773client # [ 0.998592] systemd[1]: Inserted module 'autofs4'1774lb # [ 0.957898] Freeing unused kernel image (initmem) memory: 3652K1775lb # [ 0.959023] Write protecting the kernel read-only data: 32768k1776lb # [ 0.960961] Freeing unused kernel image (text/rodata gap) memory: 1184K1777lb # [ 0.962546] Freeing unused kernel image (rodata/data gap) memory: 720K1778worker2 # [ 0.999588] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker2 # [ 1.000934] Run /init as init process1780client # [ 1.018393] fuse: init (API version 7.45)1781worker2 # [ 1.012381] systemd[1]: Inserted module 'autofs4'1782client # [ 1.025880] ACPI: \_SB_.GSIG: Enabled at IRQ 221783worker1 # [ 1.008815] x86/mm: Checked W+X mappings: passed, no W+X pages found.1784worker1 # [ 1.010028] Run /init as init process1785client # [ 1.028543] ACPI: \_SB_.GSIH: Enabled at IRQ 231786client # [ 1.031569] ACPI: \_SB_.GSIE: Enabled at IRQ 201787client # [ 1.033511] ACPI: \_SB_.GSIF: Enabled at IRQ 211788worker1 # [ 1.021204] systemd[1]: Inserted module 'autofs4'1789client # [ 1.038944] virtiofs virtio5: discovered new tag: nix-store1790client # [ 1.040524] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1791worker2 # [ 1.032525] fuse: init (API version 7.45)1792client # [ 1.047785] virtiofs virtio6: discovered new tag: shared1793client # [ 1.049516] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1794client # [ 1.052531] virtiofs virtio7: discovered new tag: xchg1795worker2 # [ 1.040123] ACPI: \_SB_.GSIG: Enabled at IRQ 221796client # [ 1.053953] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1797lb # [ 1.013508] x86/mm: Checked W+X mappings: passed, no W+X pages found.1798worker2 # [ 1.042601] ACPI: \_SB_.GSIH: Enabled at IRQ 231799lb # [ 1.014649] Run /init as init process1800worker2 # [ 1.045517] ACPI: \_SB_.GSIE: Enabled at IRQ 201801worker1 # [ 1.041060] fuse: init (API version 7.45)1802worker2 # [ 1.047498] ACPI: \_SB_.GSIF: Enabled at IRQ 211803worker2 # [ 1.052668] virtiofs virtio5: discovered new tag: nix-store1804worker1 # [ 1.048670] ACPI: \_SB_.GSIG: Enabled at IRQ 221805lb # [ 1.025636] systemd[1]: Inserted module 'autofs4'1806worker2 # [ 1.054292] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1807worker1 # [ 1.051297] ACPI: \_SB_.GSIH: Enabled at IRQ 231808worker1 # [ 1.054026] ACPI: \_SB_.GSIE: Enabled at IRQ 201809worker1 # [ 1.056268] ACPI: \_SB_.GSIF: Enabled at IRQ 211810worker2 # [ 1.061179] virtiofs virtio6: discovered new tag: shared1811worker2 # [ 1.062840] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1812client # [ 1.078620] systemd[1]: Successfully made /usr/ read-only.1813worker2 # [ 1.066111] virtiofs virtio7: discovered new tag: xchg1814worker1 # [ 1.061569] virtiofs virtio5: discovered new tag: nix-store1815worker2 # [ 1.067573] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1816worker1 # [ 1.063114] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1817lb # [ 1.045403] fuse: init (API version 7.45)1818worker1 # [ 1.071401] virtiofs virtio6: discovered new tag: shared1819worker1 # [ 1.073219] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1820lb # [ 1.052902] ACPI: \_SB_.GSIG: Enabled at IRQ 221821worker1 # [ 1.077672] virtiofs virtio7: discovered new tag: xchg1822worker1 # [ 1.079334] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1823lb # [ 1.055762] ACPI: \_SB_.GSIH: Enabled at IRQ 231824lb # [ 1.058723] ACPI: \_SB_.GSIE: Enabled at IRQ 201825lb # [ 1.060711] ACPI: \_SB_.GSIF: Enabled at IRQ 211826worker2 # [ 1.093036] systemd[1]: Successfully made /usr/ read-only.1827lb # [ 1.065891] virtiofs virtio5: discovered new tag: nix-store1828lb # [ 1.067485] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1829lb # [ 1.075913] virtiofs virtio6: discovered new tag: shared1830worker1 # [ 1.101369] systemd[1]: Successfully made /usr/ read-only.1831lb # [ 1.077617] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1832lb # [ 1.082138] virtiofs virtio7: discovered new tag: xchg1833lb # [ 1.083810] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1834lb # [ 1.106270] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.414281] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.419110] systemd[1]: Detected virtualization kvm.1837client # [ 1.419957] systemd[1]: Detected architecture x86-64.1838client # [ 1.420811] systemd[1]: Running in initrd.1839client # [ 1.421946] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.423254] systemd[1]: Hostname set to <client>.1841worker2 # [ 1.429628] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker2 # [ 1.441738] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.443951] systemd[1]: Detected architecture x86-64.1844worker2 # [ 1.446060] systemd[1]: Running in initrd.1845worker1 # [ 1.437192] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1846worker1 # [ 1.443319] systemd[1]: Detected virtualization kvm.1847worker1 # [ 1.444365] systemd[1]: Detected architecture x86-64.1848worker1 # [ 1.445403] systemd[1]: Running in initrd.1849worker2 # [ 1.448593] systemd[1]: Initializing machine ID from random generator.1850worker1 # [ 1.446764] systemd[1]: Initializing machine ID from random generator.1851worker2 # [ 1.451601] systemd[1]: Hostname set to <worker2>.1852worker1 # [ 1.448225] systemd[1]: Hostname set to <worker1>.1853lb # [ 1.443236] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854lb # [ 1.455356] systemd[1]: Detected virtualization kvm.1855lb # [ 1.457464] systemd[1]: Detected architecture x86-64.1856lb # [ 1.459572] systemd[1]: Running in initrd.1857lb # [ 1.461940] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.464898] systemd[1]: Hostname set to <lb>.1859client # [ 1.627035] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker2 # [ 1.664621] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.680668] systemd[1]: Queued start job for default target Initrd Default Target.1862worker1 # [ 1.662728] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.685039] systemd[1]: Created slice Slice /system/modprobe.1864client # [ 1.686198] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865client # [ 1.687573] systemd[1]: Expecting device /dev/disk/by-label/nixos...1866client # [ 1.688648] systemd[1]: Reached target Path Units.1867client # [ 1.689530] systemd[1]: Reached target Slice Units.1868client # [ 1.690363] systemd[1]: Reached target Swaps.1869client # [ 1.691119] systemd[1]: Reached target Timer Units.1870client # [ 1.692125] systemd[1]: Listening on D-Bus System Message Bus Socket.1871client # [ 1.693284] systemd[1]: Listening on Journal Socket (/dev/log).1872client # [ 1.694465] systemd[1]: Listening on Journal Sockets.1873client # [ 1.695475] systemd[1]: Listening on udev Control Socket.1874client # [ 1.696485] systemd[1]: Listening on udev Kernel Socket.1875client # [ 1.697407] systemd[1]: Reached target Socket Units.1876client # [ 1.699192] systemd[1]: Starting Create List of Static Device Nodes...1877client # [ 1.703509] systemd[1]: Starting Load Kernel Module configfs...1878worker2 # [ 1.704984] systemd[1]: Queued start job for default target Initrd Default Target.1879worker2 # [ 1.709287] systemd[1]: Created slice Slice /system/modprobe.1880worker2 # [ 1.710458] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1881client # [ 1.713130] systemd[1]: Starting Journal Service...1882worker2 # [ 1.711952] systemd[1]: Expecting device /dev/disk/by-label/nixos...1883worker2 # [ 1.713022] systemd[1]: Reached target Path Units.1884lb # [ 1.683499] systemd[1]: bpf-restrict-fs: LSM BPF program attached1885worker2 # [ 1.713934] systemd[1]: Reached target Slice Units.1886worker2 # [ 1.714797] systemd[1]: Reached target Swaps.1887worker2 # [ 1.715629] systemd[1]: Reached target Timer Units.1888worker2 # [ 1.716634] systemd[1]: Listening on D-Bus System Message Bus Socket.1889worker2 # [ 1.717911] systemd[1]: Listening on Journal Socket (/dev/log).1890worker2 # [ 1.719095] systemd[1]: Listening on Journal Sockets.1891worker2 # [ 1.720138] systemd[1]: Listening on udev Control Socket.1892worker2 # [ 1.721174] systemd[1]: Listening on udev Kernel Socket.1893worker1 # [ 1.716174] systemd[1]: Queued start job for default target Initrd Default Target.1894worker2 # [ 1.722107] systemd[1]: Reached target Socket Units.1895worker2 # [ 1.723957] systemd[1]: Starting Create List of Static Device Nodes...1896worker1 # [ 1.720619] systemd[1]: Created slice Slice /system/modprobe.1897client # [ 1.739528] systemd[1]: Starting Load Kernel Modules...1898worker1 # [ 1.721806] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1899worker1 # [ 1.723152] systemd[1]: Expecting device /dev/disk/by-label/nixos...1900worker2 # [ 1.727740] systemd[1]: Starting Load Kernel Module configfs...1901worker1 # [ 1.724267] systemd[1]: Reached target Path Units.1902worker1 # [ 1.725170] systemd[1]: Reached target Slice Units.1903worker1 # [ 1.726081] systemd[1]: Reached target Swaps.1904client # [ 1.743538] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1905worker1 # [ 1.726870] systemd[1]: Reached target Timer Units.1906worker1 # [ 1.727835] systemd[1]: Listening on D-Bus System Message Bus Socket.1907worker1 # [ 1.729046] systemd[1]: Listening on Journal Socket (/dev/log).1908worker1 # [ 1.730267] systemd[1]: Listening on Journal Sockets.1909worker1 # [ 1.731289] systemd[1]: Listening on udev Control Socket.1910worker1 # [ 1.732338] systemd[1]: Listening on udev Kernel Socket.1911client # [ 1.750619] systemd[1]: Starting Coldplug All udev Devices...1912worker1 # [ 1.733364] systemd[1]: Reached target Socket Units.1913worker1 # [ 1.735210] systemd[1]: Starting Create List of Static Device Nodes...1914worker1 # [ 1.739121] systemd[1]: Starting Load Kernel Module configfs...1915client # [ 1.762315] systemd-journald[66]: Collecting audit messages is disabled.1916worker2 # [ 1.739988] systemd[1]: Starting Journal Service...1917client # [ 1.765544] systemd[1]: Finished Create List of Static Device Nodes.1918client # [ 1.771185] systemd[1]: modprobe@configfs.service: Deactivated successfully.1919client # [ 1.776832] systemd[1]: Finished Load Kernel Module configfs.1920worker1 # [ 1.749451] systemd[1]: Starting Journal Service...1921worker2 # [ 1.766079] systemd[1]: Starting Load Kernel Modules...1922lb # [ 1.737307] systemd[1]: Queued start job for default target Initrd Default Target.1923lb # [ 1.741664] systemd[1]: Created slice Slice /system/modprobe.1924worker2 # [ 1.769993] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1925lb # [ 1.742857] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1926client # [ 1.783832] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927lb # [ 1.744313] systemd[1]: Expecting device /dev/disk/by-label/nixos...1928lb # [ 1.745459] systemd[1]: Reached target Path Units.1929lb # [ 1.746385] systemd[1]: Reached target Slice Units.1930lb # [ 1.747317] systemd[1]: Reached target Swaps.1931lb # [ 1.748146] systemd[1]: Reached target Timer Units.1932worker2 # [ 1.777073] systemd[1]: Starting Coldplug All udev Devices...1933lb # [ 1.749157] systemd[1]: Listening on D-Bus System Message Bus Socket.1934lb # [ 1.750429] systemd[1]: Listening on Journal Socket (/dev/log).1935lb # [ 1.751649] systemd[1]: Listening on Journal Sockets.1936worker1 # [ 1.775881] systemd[1]: Starting Load Kernel Modules...1937lb # [ 1.752649] systemd[1]: Listening on udev Control Socket.1938lb # [ 1.753669] systemd[1]: Listening on udev Kernel Socket.1939client # [ 1.795583] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1940lb # [ 1.754622] systemd[1]: Reached target Socket Units.1941lb # [ 1.756358] systemd[1]: Starting Create List of Static Device Nodes...1942worker1 # [ 1.780387] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1943worker2 # [ 1.789504] systemd-journald[66]: Collecting audit messages is disabled.1944lb # [ 1.761207] systemd[1]: Starting Load Kernel Module configfs...1945client # [ 1.802890] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1946worker2 # [ 1.792226] systemd[1]: Finished Create List of Static Device Nodes.1947worker1 # [ 1.790479] systemd[1]: Starting Coldplug All udev Devices...1948worker2 # [ 1.798005] systemd[1]: modprobe@configfs.service: Deactivated successfully.1949client # [ 1.814627] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1950worker1 # [ 1.801014] systemd-journald[66]: Collecting audit messages is disabled.1951worker2 # [ 1.806183] systemd[1]: Finished Load Kernel Module configfs.1952worker1 # [ 1.802598] systemd[1]: Finished Create List of Static Device Nodes.1953worker2 # [ 1.810342] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1954lb # [ 1.771956] systemd[1]: Starting Journal Service...1955worker1 # [ 1.809582] systemd[1]: modprobe@configfs.service: Deactivated successfully.1956worker1 # [ 1.817644] systemd[1]: Finished Load Kernel Module configfs.1957worker2 # [ 1.823965] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1958client # [ 1.837608] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1959lb # [ 1.797999] systemd[1]: Starting Load Kernel Modules...1960worker1 # [ 1.821680] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1961worker2 # [ 1.829605] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1962client # [ 1.846648] systemd[1]: Starting Create Static Device Nodes in /dev...1963lb # [ 1.805428] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1964worker1 # [ 1.835480] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965worker2 # [ 1.841138] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966lb # [ 1.813955] systemd[1]: Starting Coldplug All udev Devices...1967client # [ 1.860058] systemd[1]: Finished Load Kernel Modules.1968worker1 # [ 1.847340] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969lb # [ 1.824346] systemd-journald[66]: Collecting audit messages is disabled.1970client # [ 1.869188] systemd[1]: Starting Apply Kernel Variables...1971lb # [ 1.827662] systemd[1]: Finished Create List of Static Device Nodes.1972worker1 # [ 1.855288] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1973lb # [ 1.833678] systemd[1]: modprobe@configfs.service: Deactivated successfully.1974worker2 # [ 1.867570] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1975lb # [ 1.840711] systemd[1]: Finished Load Kernel Module configfs.1976lb # [ 1.846751] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1977client # [ 1.755193] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.889766] systemd[1]: Finished Create Static Device Nodes in /dev.1978client # 1979client # [ 1.758185] systemd-modules-load[68]: Inserted module 'virtio_balloon'1980worker2 # [ 1.880234] systemd[1]: Starting Create Static Device Nodes in /dev...1981client # [ 1.761265] systemd-modules-load[68]: Inserted module 'virtio_gpu'1982client # [ 1.896377] systemd[1]: Started Journal Service.1983worker1 # [ 1.882897] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1984lb # [ 1.860381] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1985worker2 # [ 1.890468] systemd[1]: Finished Load Kernel Modules.1986client # [ 1.770900] systemd[1]: Reached target Preparation for Local File Systems.1987client # [ 1.775077] systemd[1]: Reached target Local File Systems.1988worker1 # [ 1.892467] systemd[1]: Starting Create Static Device Nodes in /dev...1989client # [ 1.777916] systemd[1]: Starting Create System Files and Directories...1990lb # [ 1.867269] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1991worker2 # [ 1.901988] systemd[1]: Starting Apply Kernel Variables...1992client # [ 1.783164] systemd[1]: Starting Rule-based Manager for Device Events and Files...1993worker1 # [ 1.902828] systemd[1]: Finished Load Kernel Modules.1994lb # [ 1.879329] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1995client # [ 1.790651] systemd[1]: Finished Apply Kernel Variables.1996worker1 # [ 1.912332] systemd[1]: Starting Apply Kernel Variables...1997worker2 # [ 1.781468] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.918422] systemd[1]: Started Journal Service.1998worker2 # 1999worker2 # [ 1.785490] systemd-modules-load[68]: Inserted module 'virtio_balloon'2000worker2 # [ 1.786613] systemd-modules-load[68]: Inserted module 'virtio_gpu'
loading 3,002 hidden lines…
5003client # copying 1 paths to 'lb:50051'...5004client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...5005worker2 # [ 47.745541] nix-grpc-daemon[783]: ts=2026-09-23T05:12:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825006worker2 # [ 47.753576] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5007worker2 # [ 47.845258] nix-daemon[873]: accepted connection from pid 969, user nix-grpc-daemon (trusted)5008worker2 # [ 47.862060] nix-daemon[873]: reaped child process 972, status = succeeded5009lb # [ 47.834803] niks3-server[1002]: 2026/09/23 05:12:20 INFO Received uploads request method=POST path=/api/pending_closures5010worker2 # [ 47.870271] nix-grpc-daemon[925]: time=2026-09-23T05:12:21.060Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5011worker2 # [ 47.871726] nix-grpc-daemon[925]: time=2026-09-23T05:12:21.061Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"5012lb # [ 47.869815] niks3-server[1002]: 2026/09/23 05:12:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5013lb # [ 47.876161] niks3-server[1002]: 2026/09/23 05:12:20 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst5014lb # [ 47.884531] niks3-server[1002]: 2026/09/23 05:12:20 INFO Signed narinfos id=11 count=15015worker2 # [ 47.916051] nix-grpc-daemon[925]: time=2026-09-23T05:12:21.106Z level=INFO msg="Uploading 1 narinfos"5016lb # [ 47.893071] niks3-server[1002]: 2026/09/23 05:12:20 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls5017lb # [ 47.904991] niks3-server[1002]: 2026/09/23 05:12:20 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5018lb # [ 47.914493] niks3-server[1002]: 2026/09/23 05:12:20 INFO Completed upload id=115019worker2 # [ 47.942058] nix-grpc-daemon[925]: time=2026-09-23T05:12:21.131Z level=INFO msg="Upload complete. (176ms)"5020lb # [ 47.917740] niks3-server[1002]: 2026/09/23 05:12:20 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo5021worker2 # [ 47.946790] nix-grpc-daemon[783]: ts=2026-09-23T05:12:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=6245022worker2 # [ 47.953295] nix-grpc-daemon[783]: ts=2026-09-23T05:12:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825023worker2 # [ 47.962130] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5024worker2 # [ 47.973267] nix-daemon[873]: reaped child process 967, status = succeeded5025worker2 # [ 47.977681] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5026client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...5027client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5028client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5029client # farm-b> LOG-farm-b-sig5030worker2 # [ 48.794201] nix-daemon[873]: reaped child process 976, status = succeeded5031worker2 # [ 48.865782] nix-daemon[873]: accepted connection from pid 992, user nix-grpc-daemon (trusted)5032worker2 # [ 48.878744] nix-daemon[873]: reaped child process 995, status = succeeded5033lb # [ 48.859503] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received uploads request method=POST path=/api/pending_closures5034worker2 # [ 48.896163] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.085Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5035worker2 # [ 48.898061] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.087Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"5036lb # [ 48.901561] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst5037lb # [ 48.910488] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5038lb # [ 48.916249] niks3-server[1002]: 2026/09/23 05:12:21 INFO Signed narinfos id=12 count=15039worker2 # [ 48.946799] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.136Z level=INFO msg="Uploading 1 narinfos"5040lb # [ 48.924946] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5041lb # [ 48.934628] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls5042lb # [ 48.941804] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5043lb # [ 48.945510] niks3-server[1002]: 2026/09/23 05:12:21 INFO Completed upload id=125044worker2 # [ 48.973060] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.162Z level=INFO msg="Upload complete. (189ms)"5045lb # [ 48.947993] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo5046worker2 # [ 48.983803] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=15047worker2 # [ 48.994141] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=05048worker2 # [ 48.998713] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=15049worker2 # [ 49.009550] nix-daemon[873]: reaped child process 974, status = succeeded5050worker2 # [ 49.038101] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825051worker2 # [ 49.052408] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825052worker2 # [ 49.066229] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.256Z level=INFO msg="All 2 paths already cached"5053client # copying 1 paths to 'lb:50051'...5054client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...5055worker2 # [ 49.075157] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825056worker2 # [ 49.083326] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5057worker2 # [ 49.179274] nix-daemon[873]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)5058worker2 # [ 49.192508] nix-daemon[873]: reaped child process 1004, status = succeeded5059lb # [ 49.172757] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received uploads request method=POST path=/api/pending_closures5060worker2 # [ 49.208700] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.398Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5061worker2 # [ 49.210754] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.400Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"5062lb # [ 49.209122] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5063lb # [ 49.221258] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5064worker2 # [ 49.254437] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.444Z level=INFO msg="Uploading 1 narinfos"5065lb # [ 49.226927] niks3-server[1002]: 2026/09/23 05:12:21 INFO Signed narinfos id=13 count=15066lb # [ 49.239060] niks3-server[1002]: 2026/09/23 05:12:21 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5067lb # [ 49.267573] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5068lb # [ 49.273183] niks3-server[1002]: 2026/09/23 05:12:21 INFO Completed upload id=135069worker2 # [ 49.300371] nix-grpc-daemon[925]: time=2026-09-23T05:12:22.490Z level=INFO msg="Upload complete. (204ms)"5070lb # [ 49.277681] niks3-server[1002]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5071worker2 # [ 49.305563] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=7525072worker2 # [ 49.311919] nix-grpc-daemon[783]: ts=2026-09-23T05:12:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825073worker2 # [ 49.325320] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5074worker2 # [ 49.333646] nix-daemon[873]: reaped child process 999, status = succeeded5075client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5076worker2 # [ 49.342594] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5077client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5078client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5079client # farm-top> LOG-farm-top-sig5080worker2 # [ 50.359856] nix-daemon[873]: reaped child process 1008, status = succeeded5081worker2 # [ 50.435023] nix-daemon[873]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)5082worker2 # [ 50.448253] nix-daemon[873]: reaped child process 1027, status = succeeded5083lb # [ 50.428068] niks3-server[1002]: 2026/09/23 05:12:22 INFO Received uploads request method=POST path=/api/pending_closures5084worker2 # [ 50.462192] nix-grpc-daemon[925]: time=2026-09-23T05:12:23.650Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5085worker2 # [ 50.463680] nix-grpc-daemon[925]: time=2026-09-23T05:12:23.650Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5086lb # [ 50.448722] niks3-server[1002]: 2026/09/23 05:12:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5087worker2 # [ 50.478690] nix-grpc-daemon[925]: time=2026-09-23T05:12:23.668Z level=INFO msg="Uploading 1 narinfos"5088lb # [ 50.451923] niks3-server[1002]: 2026/09/23 05:12:22 INFO Signed narinfos id=14 count=15089lb # [ 50.459223] niks3-server[1002]: 2026/09/23 05:12:22 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5090lb # [ 50.462071] niks3-server[1002]: 2026/09/23 05:12:22 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5091lb # [ 50.466401] niks3-server[1002]: 2026/09/23 05:12:22 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5092lb # [ 50.472941] niks3-server[1002]: 2026/09/23 05:12:22 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5093lb # [ 50.477894] niks3-server[1002]: 2026/09/23 05:12:22 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5094lb # [ 50.480382] niks3-server[1002]: 2026/09/23 05:12:22 INFO Completed upload id=145095worker2 # [ 50.507452] nix-grpc-daemon[925]: time=2026-09-23T05:12:23.697Z level=INFO msg="Upload complete. (153ms)"5096worker2 # [ 50.510050] nix-grpc-daemon[783]: ts=2026-09-23T05:12:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15097worker2 # [ 50.517861] nix-daemon[873]: reaped child process 1006, status = succeeded5098client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.90 seconds)5099client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5100client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.13 seconds)5101client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5102worker1 # [ 50.834888] nix-grpc-daemon[783]: ts=2026-09-23T05:12:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:388225103client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5104client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5105client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5106client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5107worker1 # [ 50.853370] nix-grpc-daemon[783]: ts=2026-09-23T05:12:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:388225108worker1 # [ 50.859906] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5109worker1 # [ 50.870647] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5110worker1 # [ 50.879679] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5111client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)5112client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5113worker2 # [ 51.034568] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:562485114worker2 # [ 51.039556] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:562485115client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.18 seconds)5116client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5117client # finding garbage collector roots...5118client # removing stale temporary roots file "/nix/var/nix/temproots/802"5119client # removing stale temporary roots file "/nix/var/nix/temproots/873"5120client # removing stale temporary roots file "/nix/var/nix/temproots/881"5121client # removing stale temporary roots file "/nix/var/nix/temproots/840"5122client # removing stale temporary roots file "/nix/var/nix/temproots/819"5123client # removing stale temporary roots file "/nix/var/nix/temproots/896"5124client # removing stale temporary roots file "/nix/var/nix/temproots/860"5125client # removing stale temporary roots file "/nix/var/nix/temproots/852"5126client # deleting specified paths...5127client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5128client # deleting unused links...5129client # note: hard linking is currently saving 0.0 KiB5130worker1 # [ 51.340585] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528225131worker1 # [ 51.343547] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225132worker1 # [ 51.348853] nix-grpc-daemon[863]: time=2026-09-23T05:12:24.393Z level=INFO msg="All 1 paths already cached"5133worker2 # [ 51.359243] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825134worker1 # [ 51.360459] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:528225135client # copying 3 paths...5136worker2 # [ 51.372865] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825137client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5138worker1 # [ 51.382239] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:528225139worker1 # [ 51.385213] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_out=2325140worker2 # [ 51.394638] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:534825141worker1 # [ 51.394304] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:528225142worker1 # [ 51.397027] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_out=1285143worker2 # [ 51.400350] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_out=1765144client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5145client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5146client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.39 seconds)5147(finished: subtest: outputs read back through any worker carry the cache signature, in 5.79 seconds)5148subtest: two clients wanting the same drv build it once5149client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5150client # [ 51.516744] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5151client # Running as unit: dedup1.service; invocation ID: e9a83eb352e14927b05a0f8dc186acde5152client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5153client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5154client # [ 51.700706] nix[949]: don't know how to build these paths:5155client # [ 51.701703] nix[949]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5156worker2 # [ 51.686268] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:534825157worker2 # [ 51.695867] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05158worker2 # [ 51.700152] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15159worker1 # [ 51.734701] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225160worker1 # [ 51.746767] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225161client # [ 51.773476] nix[949]: copying 1 paths to 'lb:50051'...5162client # [ 51.776958] nix[949]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5163worker1 # [ 51.758978] nix-grpc-daemon[783]: ts=2026-09-23T05:12:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225164worker1 # [ 51.770932] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5165worker1 # [ 51.881464] nix-daemon[850]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)5166worker1 # [ 51.895859] nix-daemon[850]: reaped child process 1016, status = succeeded5167lb # [ 51.881576] niks3-server[1002]: 2026/09/23 05:12:24 INFO Received uploads request method=POST path=/api/pending_closures5168worker1 # [ 51.912155] nix-grpc-daemon[863]: time=2026-09-23T05:12:24.956Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5169worker1 # [ 51.913633] nix-grpc-daemon[863]: time=2026-09-23T05:12:24.956Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5170lb # [ 51.912844] niks3-server[1002]: 2026/09/23 05:12:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5171worker1 # [ 51.944489] nix-grpc-daemon[863]: time=2026-09-23T05:12:24.988Z level=INFO msg="Uploading 1 narinfos"5172lb # [ 51.922227] niks3-server[1002]: 2026/09/23 05:12:24 INFO Signed narinfos id=15 count=15173lb # [ 51.935594] niks3-server[1002]: 2026/09/23 05:12:24 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5174lb # [ 51.940391] niks3-server[1002]: 2026/09/23 05:12:24 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5175lb # [ 51.950883] niks3-server[1002]: 2026/09/23 05:12:24 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5176lb # [ 51.956724] niks3-server[1002]: 2026/09/23 05:12:24 INFO Completed upload id=155177worker1 # [ 51.978879] nix-grpc-daemon[863]: time=2026-09-23T05:12:25.023Z level=INFO msg="Upload complete. (195ms)"5178lb # [ 51.959324] niks3-server[1002]: 2026/09/23 05:12:24 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5179worker1 # [ 51.984637] nix-grpc-daemon[783]: ts=2026-09-23T05:12:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=4325180worker1 # [ 51.993505] nix-grpc-daemon[783]: ts=2026-09-23T05:12:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225181worker1 # [ 52.005106] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5182worker1 # [ 52.013265] nix-daemon[850]: reaped child process 1011, status = succeeded5183worker1 # [ 52.018385] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5184client # [ 52.036973] nix[949]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5185client # [ 52.574902] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5186client # Running as unit: dedup2.service; invocation ID: ae920bf2437b44c4af20ef1be6eebde15187client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5188worker2 # [ 52.723955] nix-grpc-daemon[783]: ts=2026-09-23T05:12:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:534825189client # [ 52.744654] nix[965]: don't know how to build these paths:5190client # [ 52.745905] nix[965]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5191worker2 # [ 52.737799] nix-grpc-daemon[783]: ts=2026-09-23T05:12:25Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05192worker1 # [ 52.737950] nix-grpc-daemon[783]: ts=2026-09-23T05:12:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225193client # [ 52.761705] nix[965]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5194client: must succeed: sleep 25195lb # [ 55.697156] envoy[910]: conn peer=[2001:db8:1::1]:55114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4358 ms=47425196lb # [ 55.700977] envoy[910]: conn peer=[2001:db8:1::1]:55124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2890 ms=1195197lb # [ 55.705242] envoy[910]: conn peer=[2001:db8:1::1]:55158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=655198lb # [ 55.709204] envoy[910]: conn peer=[2001:db8:1::1]:55142 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=715199lb # [ 55.713223] envoy[910]: conn peer=[2001:db8:1::1]:55136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=775200client: (finished: must succeed: sleep 2, in 2.04 seconds)5201client: waiting for success: ! systemctl is-active dedup1 dedup25202worker1 # [ 56.063997] nix-daemon[850]: reaped child process 1020, status = succeeded5203worker1 # [ 56.143720] nix-daemon[850]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)5204worker1 # [ 56.158457] nix-daemon[850]: reaped child process 1053, status = succeeded5205lb # [ 56.141665] niks3-server[1002]: 2026/09/23 05:12:28 INFO Received uploads request method=POST path=/api/pending_closures5206worker1 # [ 56.173250] nix-grpc-daemon[863]: time=2026-09-23T05:12:29.217Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5207worker1 # [ 56.176063] nix-grpc-daemon[863]: time=2026-09-23T05:12:29.219Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5208lb # [ 56.192817] niks3-server[1002]: 2026/09/23 05:12:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5209lb # [ 56.200542] niks3-server[1002]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5210lb # [ 56.204792] niks3-server[1002]: 2026/09/23 05:12:28 INFO Signed narinfos id=16 count=15211worker1 # [ 56.230090] nix-grpc-daemon[863]: time=2026-09-23T05:12:29.274Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 56.208214] niks3-server[1002]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5213lb # [ 56.215660] niks3-server[1002]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5214lb # [ 56.226155] niks3-server[1002]: 2026/09/23 05:12:28 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5215lb # [ 56.229630] niks3-server[1002]: 2026/09/23 05:12:28 INFO Completed upload id=165216worker1 # [ 56.251862] nix-grpc-daemon[863]: time=2026-09-23T05:12:29.296Z level=INFO msg="Upload complete. (221ms)"5217lb # [ 56.232042] niks3-server[1002]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5218worker1 # [ 56.258696] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15219worker1 # [ 56.266321] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15220worker1 # [ 56.278661] nix-daemon[850]: reaped child process 1018, status = succeeded5221client # [ 56.323454] systemd[1]: dedup2.service: Deactivated successfully.5222client # [ 56.328200] systemd[1]: dedup2.service: Consumed 138ms CPU time over 3.752s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5223client # [ 56.334843] systemd[1]: dedup1.service: Deactivated successfully.5224client # [ 56.337214] systemd[1]: dedup1.service: Consumed 135ms CPU time over 4.817s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5225client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)5226client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5227client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5228worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5229worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5230worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5231worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)5232worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5233worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5234worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5235worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5236(finished: subtest: two clients wanting the same drv build it once, in 5.70 seconds)5237subtest: --timeout and --max-silent-time reach the builder5238worker1 # [ 57.328996] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528225239worker2 # [ 57.343081] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05240worker2 # [ 57.346756] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15241worker2 # [ 57.348945] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825242worker2 # [ 57.353954] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825243worker2 # [ 57.361836] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825244worker2 # [ 57.368393] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5245worker2 # [ 57.459943] nix-daemon[873]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)5246worker2 # [ 57.475805] nix-daemon[873]: reaped child process 1061, status = succeeded5247lb # [ 57.453867] niks3-server[1002]: 2026/09/23 05:12:29 INFO Received uploads request method=POST path=/api/pending_closures5248worker2 # [ 57.490047] nix-grpc-daemon[925]: time=2026-09-23T05:12:30.679Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5249worker2 # [ 57.492139] nix-grpc-daemon[925]: time=2026-09-23T05:12:30.681Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5250lb # [ 57.484717] niks3-server[1002]: 2026/09/23 05:12:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5251lb # [ 57.493939] niks3-server[1002]: 2026/09/23 05:12:29 INFO Signed narinfos id=17 count=15252worker2 # [ 57.524864] nix-grpc-daemon[925]: time=2026-09-23T05:12:30.714Z level=INFO msg="Uploading 1 narinfos"5253lb # [ 57.500360] niks3-server[1002]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5254lb # [ 57.508563] niks3-server[1002]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5255lb # [ 57.521888] niks3-server[1002]: 2026/09/23 05:12:29 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5256lb # [ 57.528878] niks3-server[1002]: 2026/09/23 05:12:29 INFO Completed upload id=175257lb # [ 57.531155] niks3-server[1002]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5258worker2 # [ 57.556929] nix-grpc-daemon[925]: time=2026-09-23T05:12:30.746Z level=INFO msg="Upload complete. (181ms)"5259worker2 # [ 57.561455] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=4245260worker2 # [ 57.567367] nix-grpc-daemon[783]: ts=2026-09-23T05:12:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825261worker2 # [ 57.576828] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5262worker2 # [ 57.586751] nix-daemon[873]: reaped child process 1056, status = succeeded5263worker2 # [ 57.591059] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5264worker2 # [ 63.806497] nix-grpc-daemon[783]: ts=2026-09-23T05:12:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05265worker2 # [ 63.831624] nix-daemon[873]: reaped child process 1063, status = succeeded5266worker2 # [ 63.833965] nix-daemon[873]: reaped child process 1065, status = succeeded5267worker2 # [ 63.969689] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:534825268worker2 # [ 63.980925] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05269worker2 # [ 63.985437] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15270worker1 # [ 63.980260] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225271worker1 # [ 63.992175] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225272worker1 # [ 64.002837] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225273worker1 # [ 64.014988] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5274worker1 # [ 64.123801] nix-daemon[850]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)5275worker1 # [ 64.138858] nix-daemon[850]: reaped child process 1069, status = succeeded5276lb # [ 64.123555] niks3-server[1002]: 2026/09/23 05:12:36 INFO Received uploads request method=POST path=/api/pending_closures5277worker1 # [ 64.154476] nix-grpc-daemon[863]: time=2026-09-23T05:12:37.198Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5278worker1 # [ 64.156097] nix-grpc-daemon[863]: time=2026-09-23T05:12:37.200Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5279lb # [ 64.166892] niks3-server[1002]: 2026/09/23 05:12:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5280lb # [ 64.173457] niks3-server[1002]: 2026/09/23 05:12:36 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5281worker1 # [ 64.202555] nix-grpc-daemon[863]: time=2026-09-23T05:12:37.246Z level=INFO msg="Uploading 1 narinfos"5282lb # [ 64.181132] niks3-server[1002]: 2026/09/23 05:12:36 INFO Signed narinfos id=18 count=15283lb # [ 64.193349] niks3-server[1002]: 2026/09/23 05:12:36 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5284lb # [ 64.221610] niks3-server[1002]: 2026/09/23 05:12:36 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5285lb # [ 64.229411] niks3-server[1002]: 2026/09/23 05:12:36 INFO Completed upload id=185286worker1 # [ 64.251470] nix-grpc-daemon[863]: time=2026-09-23T05:12:37.295Z level=INFO msg="Upload complete. (224ms)"5287lb # [ 64.233191] niks3-server[1002]: 2026/09/23 05:12:36 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5288worker1 # [ 64.256723] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=4245289worker1 # [ 64.262804] nix-grpc-daemon[783]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225290worker1 # [ 64.275160] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5291worker1 # [ 64.288224] nix-daemon[850]: reaped child process 1064, status = succeeded5292worker1 # [ 64.293750] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5293lb # [ 65.697629] envoy[910]: conn peer=[2001:db8:1::1]:55174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3037 tx=1536 ms=46335294lb # [ 65.701422] envoy[910]: conn peer=[2001:db8:1::1]:55184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=35665295lb # [ 65.705159] envoy[910]: conn peer=[2001:db8:1::1]:36340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=935 ms=64905296worker1 # [ 70.711952] nix-grpc-daemon[783]: ts=2026-09-23T05:12:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05297worker1 # [ 70.737606] nix-daemon[850]: reaped child process 1071, status = succeeded5298worker1 # [ 70.740258] nix-daemon[850]: reaped child process 1073, status = succeeded5299(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.63 seconds)5300subtest: the first client leaving does not fail the second5301client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5302client # [ 70.870526] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5303client # Running as unit: share1.service; invocation ID: 87442d6c679845819b5fe4733c0792095304client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5305client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5306worker1 # [ 70.993278] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528225307client # [ 71.018106] nix[1037]: don't know how to build these paths:5308client # [ 71.018986] nix[1037]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5309worker2 # [ 71.014382] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05310worker2 # [ 71.020231] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15311worker2 # [ 71.024681] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825312worker2 # [ 71.037513] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825313client # [ 71.061085] nix[1037]: copying 1 paths to 'lb:50051'...5314client # [ 71.061851] nix[1037]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5315worker2 # [ 71.048877] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825316worker2 # [ 71.060090] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5317worker2 # [ 71.159955] nix-daemon[873]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)5318worker2 # [ 71.173034] nix-daemon[873]: reaped child process 1095, status = succeeded5319lb # [ 71.153264] niks3-server[1002]: 2026/09/23 05:12:43 INFO Received uploads request method=POST path=/api/pending_closures5320worker2 # [ 71.189149] nix-grpc-daemon[925]: time=2026-09-23T05:12:44.378Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5321worker2 # [ 71.191146] nix-grpc-daemon[925]: time=2026-09-23T05:12:44.380Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5322lb # [ 71.186202] niks3-server[1002]: 2026/09/23 05:12:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5323worker2 # [ 71.220472] nix-grpc-daemon[925]: time=2026-09-23T05:12:44.410Z level=INFO msg="Uploading 1 narinfos"5324lb # [ 71.193089] niks3-server[1002]: 2026/09/23 05:12:43 INFO Signed narinfos id=19 count=15325lb # [ 71.208104] niks3-server[1002]: 2026/09/23 05:12:43 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5326lb # [ 71.214192] niks3-server[1002]: 2026/09/23 05:12:43 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5327lb # [ 71.226671] niks3-server[1002]: 2026/09/23 05:12:43 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5328lb # [ 71.231187] niks3-server[1002]: 2026/09/23 05:12:43 INFO Completed upload id=195329worker2 # [ 71.258568] nix-grpc-daemon[925]: time=2026-09-23T05:12:44.448Z level=INFO msg="Upload complete. (187ms)"5330lb # [ 71.233734] niks3-server[1002]: 2026/09/23 05:12:43 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5331worker2 # [ 71.263556] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=4325332worker2 # [ 71.270147] nix-grpc-daemon[783]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825333worker2 # [ 71.279759] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5334worker2 # [ 71.294456] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5335client # [ 71.307773] nix[1037]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5336worker2 # [ 71.296574] nix-daemon[873]: reaped child process 1090, status = succeeded5337client # [ 71.934558] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5338client # Running as unit: share2.service; invocation ID: 329cad54885e4732b3ae78b8d85db7655339client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5340client # [ 72.078570] nix[1053]: don't know how to build these paths:5341client # [ 72.079559] nix[1053]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5342worker1 # [ 72.058474] nix-grpc-daemon[783]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528225343worker2 # [ 72.073072] nix-grpc-daemon[783]: ts=2026-09-23T05:12:45Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05344client # [ 72.092534] nix[1053]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5345worker2 # [ 72.078836] nix-grpc-daemon[783]: ts=2026-09-23T05:12:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825346client: must succeed: sleep 2; systemctl kill -s INT share15347client # [ 75.196893] systemd[1]: share1.service: Sent signal SIGINT to main process 1037 (nix) on client request.5348client # [ 75.203564] nix[1037]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5349client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5350client: waiting for success: ! systemctl is-active share15351client # [ 75.210349] nix[1037]: error: interrupted by the user5352client # [ 75.251450] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5353client # [ 75.253121] systemd[1]: share1.service: Failed with result 'exit-code'.5354client # [ 75.255104] systemd[1]: share1.service: Consumed 116ms CPU time over 4.383s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5355lb # [ 75.697933] envoy[910]: conn peer=[2001:db8:1::1]:44742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=957 ms=67605356lb # [ 75.701856] envoy[910]: conn peer=[2001:db8:1::1]:44744 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=795 ms=42135357client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5358client: waiting for success: ! systemctl is-active share25359worker2 # [ 76.516522] nix-daemon[873]: reaped child process 1099, status = succeeded5360worker2 # [ 76.591816] nix-daemon[873]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)5361worker2 # [ 76.605174] nix-daemon[873]: reaped child process 1138, status = succeeded5362lb # [ 76.586427] niks3-server[1002]: 2026/09/23 05:12:48 INFO Received uploads request method=POST path=/api/pending_closures5363worker2 # [ 76.622452] nix-grpc-daemon[925]: time=2026-09-23T05:12:49.812Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5364worker2 # [ 76.625350] nix-grpc-daemon[925]: time=2026-09-23T05:12:49.814Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5365lb # [ 76.630476] niks3-server[1002]: 2026/09/23 05:12:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5366worker2 # [ 76.667349] nix-grpc-daemon[925]: time=2026-09-23T05:12:49.857Z level=INFO msg="Uploading 1 narinfos"5367lb # [ 76.640358] niks3-server[1002]: 2026/09/23 05:12:48 INFO Signed narinfos id=20 count=15368lb # [ 76.644762] niks3-server[1002]: 2026/09/23 05:12:48 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5369lb # [ 76.651829] niks3-server[1002]: 2026/09/23 05:12:48 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5370lb # [ 76.656098] niks3-server[1002]: 2026/09/23 05:12:48 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5371lb # [ 76.667190] niks3-server[1002]: 2026/09/23 05:12:48 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5372lb # [ 76.671473] niks3-server[1002]: 2026/09/23 05:12:48 INFO Completed upload id=205373worker2 # [ 76.698811] nix-grpc-daemon[925]: time=2026-09-23T05:12:49.888Z level=INFO msg="Upload complete. (192ms)"5374lb # [ 76.672724] niks3-server[1002]: 2026/09/23 05:12:48 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5375worker2 # [ 76.705573] nix-grpc-daemon[783]: ts=2026-09-23T05:12:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15376worker2 # [ 76.712653] nix-grpc-daemon[783]: ts=2026-09-23T05:12:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15377worker2 # [ 76.725069] nix-daemon[873]: reaped child process 1097, status = succeeded5378client # [ 76.754868] systemd[1]: share2.service: Deactivated successfully.5379client # [ 76.758753] systemd[1]: share2.service: Consumed 118ms CPU time over 4.823s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5380client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5381client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5382client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5383(finished: subtest: the first client leaving does not fail the second, in 6.76 seconds)5384subtest: build hook: nix-daemon with builders = grpc://lb5385client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5386client # [ 77.676223] systemd[1]: Started Nix Daemon.5387client # [ 77.761450] nix-daemon[1104]: accepted connection from pid 1101, user root (trusted)5388worker1 # [ 77.857573] nix-grpc-daemon[783]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:528225389worker1 # [ 77.860238] nix-grpc-daemon[783]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225390worker2 # [ 77.879772] nix-grpc-daemon[783]: ts=2026-09-23T05:12:51Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05391worker1 # [ 77.877079] nix-grpc-daemon[783]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225392worker2 # [ 77.883938] nix-grpc-daemon[783]: ts=2026-09-23T05:12:51Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15393worker1 # [ 77.889800] nix-grpc-daemon[783]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225394worker1 # [ 77.901701] nix-grpc-daemon[783]: ts=2026-09-23T05:12:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225395worker1 # [ 77.913298] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5396worker1 # [ 78.017209] nix-daemon[850]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)5397worker1 # [ 78.030953] nix-daemon[850]: reaped child process 1115, status = succeeded5398lb # [ 78.016537] niks3-server[1002]: 2026/09/23 05:12:50 INFO Received uploads request method=POST path=/api/pending_closures5399worker1 # [ 78.046417] nix-grpc-daemon[863]: time=2026-09-23T05:12:51.090Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5400worker1 # [ 78.047851] nix-grpc-daemon[863]: time=2026-09-23T05:12:51.090Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5401lb # [ 78.052716] niks3-server[1002]: 2026/09/23 05:12:50 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5402lb # [ 78.061706] niks3-server[1002]: 2026/09/23 05:12:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5403lb # [ 78.070160] niks3-server[1002]: 2026/09/23 05:12:50 INFO Signed narinfos id=21 count=15404worker1 # [ 78.095856] nix-grpc-daemon[863]: time=2026-09-23T05:12:51.139Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 78.076440] niks3-server[1002]: 2026/09/23 05:12:50 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5406lb # [ 78.087994] niks3-server[1002]: 2026/09/23 05:12:50 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5407lb # [ 78.090774] niks3-server[1002]: 2026/09/23 05:12:50 INFO Completed upload id=215408worker1 # [ 78.112976] nix-grpc-daemon[863]: time=2026-09-23T05:12:51.157Z level=INFO msg="Upload complete. (186ms)"5409lb # [ 78.093283] niks3-server[1002]: 2026/09/23 05:12:50 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5410worker1 # [ 78.117579] nix-grpc-daemon[783]: ts=2026-09-23T05:12:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=5125411worker1 # [ 78.123480] nix-grpc-daemon[783]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225412worker1 # [ 78.135129] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5413worker1 # [ 78.144128] nix-daemon[850]: reaped child process 1110, status = succeeded5414worker1 # [ 78.147115] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5415worker1 # [ 79.181672] nix-daemon[850]: reaped child process 1119, status = succeeded5416worker1 # [ 79.261939] nix-daemon[850]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)5417worker1 # [ 79.276443] nix-daemon[850]: reaped child process 1138, status = succeeded5418lb # [ 79.258891] niks3-server[1002]: 2026/09/23 05:12:51 INFO Received uploads request method=POST path=/api/pending_closures5419worker1 # [ 79.290174] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5420worker1 # [ 79.291907] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.336Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5421lb # [ 79.303105] niks3-server[1002]: 2026/09/23 05:12:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5422lb # [ 79.308658] niks3-server[1002]: 2026/09/23 05:12:51 INFO Signed narinfos id=22 count=15423worker1 # [ 79.333758] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.378Z level=INFO msg="Uploading 1 narinfos"5424lb # [ 79.316127] niks3-server[1002]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5425lb # [ 79.320596] niks3-server[1002]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5426lb # [ 79.329157] niks3-server[1002]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5427lb # [ 79.336993] niks3-server[1002]: 2026/09/23 05:12:51 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5428lb # [ 79.353966] niks3-server[1002]: 2026/09/23 05:12:51 INFO Completed upload id=225429worker1 # [ 79.376149] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.420Z level=INFO msg="Upload complete. (221ms)"5430lb # [ 79.356800] niks3-server[1002]: 2026/09/23 05:12:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5431worker1 # [ 79.384346] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15432worker1 # [ 79.390731] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:528225433worker1 # [ 79.402809] nix-daemon[850]: reaped child process 1117, status = succeeded5434worker1 # [ 79.413886] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:528225435worker1 # [ 79.418853] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_out=1285436worker1 # [ 79.626899] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:528225437worker2 # [ 79.637074] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825438worker2 # [ 79.659575] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05439worker1 # [ 79.657307] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225440worker2 # [ 79.664410] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15441worker1 # [ 79.668838] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225442worker1 # [ 79.684246] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.728Z level=INFO msg="All 1 paths already cached"5443worker1 # [ 79.692620] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225444worker1 # [ 79.701457] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5445worker1 # [ 79.809944] nix-daemon[850]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5446worker1 # [ 79.824254] nix-daemon[850]: reaped child process 1145, status = succeeded5447lb # [ 79.807409] niks3-server[1002]: 2026/09/23 05:12:52 INFO Received uploads request method=POST path=/api/pending_closures5448worker1 # [ 79.837942] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.882Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5449worker1 # [ 79.839427] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.883Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5450lb # [ 79.838262] niks3-server[1002]: 2026/09/23 05:12:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5451worker1 # [ 79.868071] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.912Z level=INFO msg="Uploading 1 narinfos"5452lb # [ 79.846249] niks3-server[1002]: 2026/09/23 05:12:52 INFO Signed narinfos id=23 count=15453lb # [ 79.855570] niks3-server[1002]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5454lb # [ 79.861385] niks3-server[1002]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5455lb # [ 79.874351] niks3-server[1002]: 2026/09/23 05:12:52 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5456lb # [ 79.880772] niks3-server[1002]: 2026/09/23 05:12:52 INFO Completed upload id=235457worker1 # [ 79.902858] nix-grpc-daemon[863]: time=2026-09-23T05:12:52.947Z level=INFO msg="Upload complete. (185ms)"5458lb # [ 79.884104] niks3-server[1002]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5459worker1 # [ 79.907668] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=6245460worker1 # [ 79.913322] nix-grpc-daemon[783]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225461worker1 # [ 79.925108] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5462worker1 # [ 79.934974] nix-daemon[850]: reaped child process 1140, status = succeeded5463worker1 # [ 79.937878] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5464worker1 # [ 80.774555] nix-daemon[850]: reaped child process 1149, status = succeeded5465worker1 # [ 80.859098] nix-daemon[850]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)5466worker1 # [ 80.872174] nix-daemon[850]: reaped child process 1168, status = succeeded5467lb # [ 80.856939] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received uploads request method=POST path=/api/pending_closures5468worker1 # [ 80.888685] nix-grpc-daemon[863]: time=2026-09-23T05:12:53.932Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5469worker1 # [ 80.892064] nix-grpc-daemon[863]: time=2026-09-23T05:12:53.934Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5470lb # [ 80.905215] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5471lb # [ 80.910134] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5472lb # [ 80.916130] niks3-server[1002]: 2026/09/23 05:12:53 INFO Signed narinfos id=24 count=15473worker1 # [ 80.942067] nix-grpc-daemon[863]: time=2026-09-23T05:12:53.986Z level=INFO msg="Uploading 1 narinfos"5474lb # [ 80.922342] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5475lb # [ 80.934824] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5476lb # [ 80.938658] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5477lb # [ 80.941447] niks3-server[1002]: 2026/09/23 05:12:53 INFO Completed upload id=245478worker1 # [ 80.963920] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.008Z level=INFO msg="Upload complete. (218ms)"5479lb # [ 80.943963] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5480worker1 # [ 80.970071] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15481worker2 # [ 80.980491] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825482worker1 # [ 80.984427] nix-daemon[850]: reaped child process 1147, status = succeeded5483worker2 # [ 80.990393] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5484worker2 # [ 81.001695] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5485worker2 # [ 81.026648] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:534825486worker2 # [ 81.031511] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_out=1765487worker2 # [ 81.239696] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:534825488worker2 # [ 81.246402] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825489worker2 # [ 81.252097] nix-grpc-daemon[925]: time=2026-09-23T05:12:54.440Z level=INFO msg="All 2 paths already cached"5490worker2 # [ 81.261571] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05491worker1 # [ 81.259425] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225492worker2 # [ 81.266214] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15493worker1 # [ 81.271396] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225494worker1 # [ 81.283762] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.328Z level=INFO msg="All 2 paths already cached"5495worker1 # [ 81.291140] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225496worker1 # [ 81.300166] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5497worker1 # [ 81.406861] nix-daemon[850]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5498worker1 # [ 81.421334] nix-daemon[850]: reaped child process 1175, status = succeeded5499lb # [ 81.406737] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received uploads request method=POST path=/api/pending_closures5500worker1 # [ 81.436746] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5501worker1 # [ 81.438738] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.482Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5502lb # [ 81.437652] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5503lb # [ 81.445827] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5504lb # [ 81.451588] niks3-server[1002]: 2026/09/23 05:12:53 INFO Signed narinfos id=25 count=15505worker1 # [ 81.476883] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.521Z level=INFO msg="Uploading 1 narinfos"5506lb # [ 81.457673] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5507lb # [ 81.472069] niks3-server[1002]: 2026/09/23 05:12:53 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5508lb # [ 81.479967] niks3-server[1002]: 2026/09/23 05:12:53 INFO Completed upload id=255509worker1 # [ 81.501502] nix-grpc-daemon[863]: time=2026-09-23T05:12:54.545Z level=INFO msg="Upload complete. (188ms)"5510lb # [ 81.483642] niks3-server[1002]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5511worker1 # [ 81.505861] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=7525512worker1 # [ 81.510725] nix-grpc-daemon[783]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225513worker1 # [ 81.520202] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5514worker1 # [ 81.528163] nix-daemon[850]: reaped child process 1170, status = succeeded5515worker1 # [ 81.533150] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5516worker1 # [ 82.376814] nix-daemon[850]: reaped child process 1179, status = succeeded5517worker1 # [ 82.446882] nix-daemon[850]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5518worker1 # [ 82.460519] nix-daemon[850]: reaped child process 1198, status = succeeded5519lb # [ 82.445517] niks3-server[1002]: 2026/09/23 05:12:54 INFO Received uploads request method=POST path=/api/pending_closures5520worker1 # [ 82.475605] nix-grpc-daemon[863]: time=2026-09-23T05:12:55.519Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5521worker1 # [ 82.477190] nix-grpc-daemon[863]: time=2026-09-23T05:12:55.521Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5522lb # [ 82.479690] niks3-server[1002]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5523lb # [ 82.493172] niks3-server[1002]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5524lb # [ 82.500914] niks3-server[1002]: 2026/09/23 05:12:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5525lb # [ 82.508128] niks3-server[1002]: 2026/09/23 05:12:54 INFO Signed narinfos id=26 count=15526worker1 # [ 82.530770] nix-grpc-daemon[863]: time=2026-09-23T05:12:55.574Z level=INFO msg="Uploading 1 narinfos"5527lb # [ 82.516264] niks3-server[1002]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5528lb # [ 82.523836] niks3-server[1002]: 2026/09/23 05:12:54 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5529lb # [ 82.527855] niks3-server[1002]: 2026/09/23 05:12:54 INFO Completed upload id=265530lb # [ 82.529697] niks3-server[1002]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5531worker1 # [ 82.550251] nix-grpc-daemon[863]: time=2026-09-23T05:12:55.594Z level=INFO msg="Upload complete. (210ms)"5532worker1 # [ 82.556751] nix-grpc-daemon[783]: ts=2026-09-23T05:12:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15533worker1 # [ 82.564388] nix-grpc-daemon[783]: ts=2026-09-23T05:12:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:528225534worker1 # [ 82.575725] nix-daemon[850]: reaped child process 1177, status = succeeded5535worker2 # [ 82.582176] nix-grpc-daemon[783]: ts=2026-09-23T05:12:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:534825536worker2 # [ 82.591465] nix-grpc-daemon[783]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5537worker2 # [ 82.603293] nix-grpc-daemon[783]: ts=2026-09-23T05:12:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_out=2325538client # [ 82.667119] nix-daemon[1104]: reaped child process 1106, status = succeeded5539client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.09 seconds)5540client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5541client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.01 seconds)5542client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5543client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5544client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5545client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5546(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.14 seconds)5547subtest: an input only one worker has still reaches the builder5548client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5549client # this derivation will be built:5550client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5551client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5552client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5553client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5554client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5555worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5556worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5557worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5558worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5559client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25560client # [ 83.314912] nix-daemon[1104]: accepted connection from pid 1187, user root (trusted)5561client # this derivation will be built:5562client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5563worker2 # [ 83.418156] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:534825564client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5565worker2 # [ 83.427428] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825566client # copying 1 paths...5567client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5568worker2 # [ 83.439655] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825569worker2 # [ 83.450911] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5570worker2 # [ 83.560901] nix-daemon[873]: accepted connection from pid 1154, user nix-grpc-daemon (trusted)5571worker2 # [ 83.575525] nix-daemon[873]: reaped child process 1157, status = succeeded5572lb # [ 83.557452] niks3-server[1002]: 2026/09/23 05:12:55 INFO Received uploads request method=POST path=/api/pending_closures5573worker2 # [ 83.594042] nix-grpc-daemon[925]: time=2026-09-23T05:12:56.783Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5574worker2 # [ 83.596098] nix-grpc-daemon[925]: time=2026-09-23T05:12:56.785Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5575lb # [ 83.593510] niks3-server[1002]: 2026/09/23 05:12:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5576worker2 # [ 83.627630] nix-grpc-daemon[925]: time=2026-09-23T05:12:56.817Z level=INFO msg="Uploading 1 narinfos"5577lb # [ 83.600521] niks3-server[1002]: 2026/09/23 05:12:55 INFO Signed narinfos id=27 count=15578lb # [ 83.607979] niks3-server[1002]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5579lb # [ 83.615598] niks3-server[1002]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5580lb # [ 83.630069] niks3-server[1002]: 2026/09/23 05:12:55 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5581lb # [ 83.635574] niks3-server[1002]: 2026/09/23 05:12:55 INFO Completed upload id=275582worker2 # [ 83.662738] nix-grpc-daemon[925]: time=2026-09-23T05:12:56.852Z level=INFO msg="Upload complete. (201ms)"5583lb # [ 83.642723] niks3-server[1002]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5584worker2 # [ 83.666965] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=1205585worker2 # [ 83.681678] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05586worker2 # [ 83.685314] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15587worker2 # [ 83.692482] nix-daemon[873]: reaped child process 1152, status = succeeded5588worker2 # [ 83.725533] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825589worker2 # [ 83.738586] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825590client # copying 2 paths to 'lb:50051'...5591client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5592worker2 # [ 83.754317] nix-grpc-daemon[783]: ts=2026-09-23T05:12:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825593client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5594worker2 # [ 83.762566] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5595worker2 # [ 83.864318] nix-daemon[873]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5596worker2 # [ 83.878351] nix-daemon[873]: reaped child process 1166, status = succeeded5597lb # [ 83.858893] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received uploads request method=POST path=/api/pending_closures5598lb # [ 83.957054] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received uploads request method=POST path=/api/pending_closures5599worker2 # [ 84.023063] nix-grpc-daemon[925]: time=2026-09-23T05:12:57.212Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5600worker2 # [ 84.029361] nix-grpc-daemon[925]: time=2026-09-23T05:12:57.216Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5601worker2 # [ 84.033343] nix-grpc-daemon[925]: time=2026-09-23T05:12:57.217Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5602lb # [ 84.036821] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5603lb # [ 84.039568] niks3-server[1002]: 2026/09/23 05:12:56 INFO Signed narinfos id=29 count=25604lb # [ 84.044156] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5605lb # [ 84.047537] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5606lb # [ 84.050035] niks3-server[1002]: 2026/09/23 05:12:56 INFO Signed narinfos id=28 count=15607lb # [ 84.053596] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5608worker2 # [ 84.080136] nix-grpc-daemon[925]: time=2026-09-23T05:12:57.269Z level=INFO msg="Uploading 3 narinfos"5609lb # [ 84.055631] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5610lb # [ 84.058629] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5611lb # [ 84.070173] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5612lb # [ 84.074133] niks3-server[1002]: 2026/09/23 05:12:56 INFO Completed upload id=285613lb # [ 84.076050] niks3-server[1002]: 2026/09/23 05:12:56 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5614lb # [ 84.079362] niks3-server[1002]: 2026/09/23 05:12:56 INFO Completed upload id=295615worker2 # [ 84.106507] nix-grpc-daemon[925]: time=2026-09-23T05:12:57.296Z level=INFO msg="Upload complete. (329ms)"5616lb # [ 84.081391] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5617lb # [ 84.084219] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5618lb # [ 84.085826] niks3-server[1002]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5619worker2 # [ 84.112302] nix-grpc-daemon[783]: ts=2026-09-23T05:12:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=2 nar_bytes_in=9765620worker2 # [ 84.118770] nix-grpc-daemon[783]: ts=2026-09-23T05:12:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825621worker2 # [ 84.132206] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5622worker2 # [ 84.142281] nix-daemon[873]: reaped child process 1161, status = succeeded5623client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5624worker2 # [ 84.150400] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5625client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5626client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5627worker2 # [ 85.572064] nix-daemon[873]: reaped child process 1170, status = succeeded5628worker2 # [ 85.649323] nix-daemon[873]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)5629worker2 # [ 85.663776] nix-daemon[873]: reaped child process 1189, status = succeeded5630lb # [ 85.643241] niks3-server[1002]: 2026/09/23 05:12:57 INFO Received uploads request method=POST path=/api/pending_closures5631worker2 # [ 85.680583] nix-grpc-daemon[925]: time=2026-09-23T05:12:58.870Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5632lb # [ 85.679624] niks3-server[1002]: 2026/09/23 05:12:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5633lb # [ 85.688544] niks3-server[1002]: 2026/09/23 05:12:57 INFO Signed narinfos id=30 count=15634worker2 # [ 85.718667] nix-grpc-daemon[925]: time=2026-09-23T05:12:58.908Z level=INFO msg="Uploading 1 narinfos"5635lb # [ 85.696080] niks3-server[1002]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5636lb # [ 85.700737] envoy[910]: conn peer=[2001:db8:1::1]:44756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=46595637lb # [ 85.706535] envoy[910]: conn peer=[2001:db8:1::1]:52986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3408 tx=1922 ms=16135638lb # [ 85.709529] envoy[910]: conn peer=[2001:db8:1::1]:52998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=635639lb # [ 85.712817] envoy[910]: conn peer=[2001:db8:1::1]:53006 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2164 ms=14525640lb # [ 85.716537] envoy[910]: conn peer=[2001:db8:1::1]:53012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=585641lb # [ 85.719268] envoy[910]: conn peer=[2001:db8:1::1]:53016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4323 tx=2367 ms=13735642lb # [ 85.722091] envoy[910]: conn peer=[2001:db8:1::1]:53030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=285643lb # [ 85.724872] niks3-server[1002]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5644lb # [ 85.730067] niks3-server[1002]: 2026/09/23 05:12:57 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5645lb # [ 85.733606] niks3-server[1002]: 2026/09/23 05:12:57 INFO Completed upload id=305646worker2 # [ 85.761197] nix-grpc-daemon[925]: time=2026-09-23T05:12:58.951Z level=INFO msg="Upload complete. (198ms)"5647lb # [ 85.738237] niks3-server[1002]: 2026/09/23 05:12:57 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5648client # copying 1 paths...5649worker2 # [ 85.769889] nix-grpc-daemon[783]: ts=2026-09-23T05:12:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15650worker2 # [ 85.776595] nix-grpc-daemon[783]: ts=2026-09-23T05:12:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825651client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5652worker2 # [ 85.788544] nix-daemon[873]: reaped child process 1168, status = succeeded5653worker1 # [ 85.791823] nix-grpc-daemon[783]: ts=2026-09-23T05:12:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:528225654worker1 # [ 85.802578] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5655worker1 # [ 85.815622] nix-grpc-daemon[783]: ts=2026-09-23T05:12:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_out=1205656client # [ 85.944502] nix-daemon[1104]: reaped child process 1190, status = succeeded5657client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.75 seconds)5658client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25659client # [ 86.043135] nix-daemon[1104]: accepted connection from pid 1209, user root (trusted)5660client # this derivation will be built:5661client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5662worker2 # [ 86.144858] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:534825663client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5664worker2 # [ 86.150624] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825665worker2 # [ 86.155961] nix-grpc-daemon[925]: time=2026-09-23T05:12:59.344Z level=INFO msg="All 1 paths already cached"5666worker2 # [ 86.165567] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05667worker1 # [ 86.163516] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225668worker2 # [ 86.170293] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15669worker1 # [ 86.175253] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528225670client # copying 1 paths to 'lb:50051'...5671client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5672worker1 # [ 86.191441] nix-grpc-daemon[783]: ts=2026-09-23T05:12:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528225673worker1 # [ 86.202323] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5674worker1 # [ 86.218126] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)5675worker1 # [ 86.240767] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 2/5)5676worker1 # [ 86.442194] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 375 ms (attempt 3/5)5677worker1 # [ 86.819970] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 4/5)5678worker1 # [ 86.827661] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5679worker1 # [ 86.830731] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5680worker1 # [ 86.836777] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5681worker1 # [ 86.947477] nix-daemon[850]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)5682worker1 # [ 86.962843] nix-daemon[850]: reaped child process 1216, status = succeeded5683lb # [ 86.946268] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received uploads request method=POST path=/api/pending_closures5684worker1 # [ 86.977242] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.021Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5685worker1 # [ 86.978738] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.023Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5686lb # [ 86.978150] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5687lb # [ 86.985368] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5688lb # [ 86.990380] niks3-server[1002]: 2026/09/23 05:12:59 INFO Signed narinfos id=31 count=15689worker1 # [ 87.016327] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.060Z level=INFO msg="Uploading 1 narinfos"5690lb # [ 87.003098] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5691lb # [ 87.012109] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5692lb # [ 87.019756] niks3-server[1002]: 2026/09/23 05:12:59 INFO Completed upload id=315693worker1 # [ 87.044373] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.088Z level=INFO msg="Upload complete. (188ms)"5694lb # [ 87.023248] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5695worker1 # [ 87.049166] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 paths=1 nar_bytes_in=5605696worker1 # [ 87.056394] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528225697worker1 # [ 87.067225] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5698worker1 # [ 87.078656] nix-daemon[850]: reaped child process 1206, status = succeeded5699worker1 # [ 87.082906] nix-daemon[850]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5700client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5701client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5702client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5703worker1 # [ 87.498518] nix-daemon[850]: reaped child process 1220, status = succeeded5704worker1 # [ 87.573578] nix-daemon[850]: accepted connection from pid 1236, user nix-grpc-daemon (trusted)5705worker1 # [ 87.586400] nix-daemon[850]: reaped child process 1239, status = succeeded5706lb # [ 87.571758] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received uploads request method=POST path=/api/pending_closures5707worker1 # [ 87.601730] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.646Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5708lb # [ 87.602583] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5709lb # [ 87.612128] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5710lb # [ 87.617446] niks3-server[1002]: 2026/09/23 05:12:59 INFO Signed narinfos id=32 count=15711worker1 # [ 87.642602] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.687Z level=INFO msg="Uploading 1 narinfos"5712lb # [ 87.624253] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5713lb # [ 87.635562] niks3-server[1002]: 2026/09/23 05:12:59 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5714lb # [ 87.644123] niks3-server[1002]: 2026/09/23 05:12:59 INFO Completed upload id=325715worker1 # [ 87.666133] nix-grpc-daemon[863]: time=2026-09-23T05:13:00.710Z level=INFO msg="Upload complete. (177ms)"5716lb # [ 87.646239] niks3-server[1002]: 2026/09/23 05:12:59 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5717client # copying 1 paths...5718worker1 # [ 87.671174] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52822 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15719worker2 # [ 87.683556] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825720worker1 # [ 87.683886] nix-daemon[850]: reaped child process 1218, status = succeeded5721worker2 # [ 87.693827] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5722client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5723worker2 # [ 87.720542] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:534825724worker2 # [ 87.725759] nix-grpc-daemon[783]: ts=2026-09-23T05:13:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_out=1205725client # [ 87.849382] nix-daemon[1104]: reaped child process 1212, status = succeeded5726client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 1.91 seconds)5727client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25728client # [ 87.950795] nix-daemon[1104]: accepted connection from pid 1231, user root (trusted)5729client # this derivation will be built:5730client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5731worker2 # [ 88.049204] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:534825732client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5733worker2 # [ 88.055342] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825734worker2 # [ 88.061189] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.249Z level=INFO msg="All 1 paths already cached"5735worker2 # [ 88.070153] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05736worker2 # [ 88.075702] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15737worker2 # [ 88.081522] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825738worker2 # [ 88.092947] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:534825739worker2 # [ 88.105303] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.295Z level=INFO msg="All 1 paths already cached"5740client # copying 1 paths to 'lb:50051'...5741client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5742worker2 # [ 88.112605] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:534825743worker2 # [ 88.123344] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5744worker2 # [ 88.217450] nix-daemon[873]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5745worker2 # [ 88.230279] nix-daemon[873]: reaped child process 1201, status = succeeded5746lb # [ 88.211557] niks3-server[1002]: 2026/09/23 05:13:00 INFO Received uploads request method=POST path=/api/pending_closures5747worker2 # [ 88.247405] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5748worker2 # [ 88.248924] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.439Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5749lb # [ 88.243523] niks3-server[1002]: 2026/09/23 05:13:00 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5750lb # [ 88.252543] niks3-server[1002]: 2026/09/23 05:13:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5751lb # [ 88.258375] niks3-server[1002]: 2026/09/23 05:13:00 INFO Signed narinfos id=33 count=15752worker2 # [ 88.288878] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.478Z level=INFO msg="Uploading 1 narinfos"5753lb # [ 88.269473] niks3-server[1002]: 2026/09/23 05:13:00 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5754lb # [ 88.277776] niks3-server[1002]: 2026/09/23 05:13:00 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5755lb # [ 88.283199] niks3-server[1002]: 2026/09/23 05:13:00 INFO Completed upload id=335756worker2 # [ 88.310230] nix-grpc-daemon[925]: time=2026-09-23T05:13:01.500Z level=INFO msg="Upload complete. (177ms)"5757worker2 # [ 88.314187] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_in=5605758lb # [ 88.289447] niks3-server[1002]: 2026/09/23 05:13:00 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5759worker2 # [ 88.320415] nix-grpc-daemon[783]: ts=2026-09-23T05:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:534825760worker2 # [ 88.332115] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5761worker2 # [ 88.338266] nix-daemon[873]: reaped child process 1196, status = succeeded5762worker2 # [ 88.343155] nix-daemon[873]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5763client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5764client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5765client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5766worker2 # [ 89.780679] nix-daemon[873]: reaped child process 1205, status = succeeded5767worker2 # [ 89.866462] nix-daemon[873]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5768worker2 # [ 89.879680] nix-daemon[873]: reaped child process 1224, status = succeeded5769lb # [ 89.859629] niks3-server[1002]: 2026/09/23 05:13:02 INFO Received uploads request method=POST path=/api/pending_closures5770worker2 # [ 89.896032] nix-grpc-daemon[925]: time=2026-09-23T05:13:03.085Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5771lb # [ 89.892142] niks3-server[1002]: 2026/09/23 05:13:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5772lb # [ 89.898500] niks3-server[1002]: 2026/09/23 05:13:02 INFO Signed narinfos id=34 count=15773worker2 # [ 89.928635] nix-grpc-daemon[925]: time=2026-09-23T05:13:03.118Z level=INFO msg="Uploading 1 narinfos"5774lb # [ 89.908257] niks3-server[1002]: 2026/09/23 05:13:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5775lb # [ 89.913745] niks3-server[1002]: 2026/09/23 05:13:02 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5776lb # [ 89.924128] niks3-server[1002]: 2026/09/23 05:13:02 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5777lb # [ 89.929397] niks3-server[1002]: 2026/09/23 05:13:02 INFO Completed upload id=345778worker2 # [ 89.956735] nix-grpc-daemon[925]: time=2026-09-23T05:13:03.146Z level=INFO msg="Upload complete. (205ms)"5779lb # [ 89.934281] niks3-server[1002]: 2026/09/23 05:13:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5780worker2 # [ 89.962541] nix-grpc-daemon[783]: ts=2026-09-23T05:13:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15781worker2 # [ 89.976782] nix-daemon[873]: reaped child process 1203, status = succeeded5782client # copying 1 paths...5783worker2 # [ 90.010572] nix-grpc-daemon[783]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:534825784client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5785worker2 # [ 90.032343] nix-grpc-daemon[783]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:534825786worker2 # [ 90.037143] nix-grpc-daemon[783]: ts=2026-09-23T05:13:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53482 duration_s=0 paths=1 nar_bytes_out=1205787client # [ 90.157304] nix-daemon[1104]: reaped child process 1234, status = succeeded5788client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds)5789(finished: subtest: an input only one worker has still reaches the builder, in 7.44 seconds)5790subtest: upload whose reference the worker lacks is completed from the cache5791client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5792client # this derivation will be built:5793client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5794client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5795client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5796client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5797client # this derivation will be built:5798client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5799client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5800client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5801client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5802client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5803worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5804worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5805worker2 # [ 90.889886] systemd[1]: Stopping Nix worker-protocol over gRPC...5806worker2 # [ 91.668971] nix-grpc-daemon[783]: ts=2026-09-23T05:13:04Z level=info event=signal_exit5807lb # [ 91.764785] niks3-server[1002]: 2026/09/23 05:13:04 INFO lead: released remote=192.168.1.4:367065808lb # [ 92.759640] niks3-server[1002]: 2026/09/23 05:13:05 INFO lead: acquired remote=192.168.1.3:364625809worker1 # [ 92.783111] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=scheduler_take_over5810worker1 # [ 92.790639] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=scheduler_restarting addr=lb:500515811worker2 # [ 92.795401] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=scheduler_restarting5812worker2 # [ 92.801860] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=scheduler_restarting addr=lb:500515813worker2 # [ 92.805414] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:534785814worker2 # [ 92.809847] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=worker_gone addr=192.168.1.4:500515815worker2 # [ 92.813126] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:534785816worker2 # [ 92.816708] nix-grpc-daemon[783]: ts=2026-09-23T05:13:05Z level=info event=worker_gone addr=192.168.1.3:500515817worker2 # [ 92.819997] nix-grpc-daemon[783]: E0000 00:00:1790140385.999482 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5818worker2 # [ 92.824765] nix-grpc-daemon[783]: E0000 00:00:1790140385.999762 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5819worker2 # [ 92.845172] nix-daemon[873]: reaped child process 875, status = succeeded5820worker1 # [ 92.991718] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:500515821worker1 # [ 92.995668] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 93.008838] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5823worker1 # [ 93.196506] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:500515824worker1 # [ 93.207628] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:557265825worker1 # [ 93.211663] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05826worker2 # [ 93.246574] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5827worker2 # [ 93.250148] systemd[1]: Stopped Nix worker-protocol over gRPC.5828worker2 # [ 93.254064] systemd[1]: nix-grpc-daemon.service: Consumed 2.017s CPU time over 1min 16.732s wall clock time, 64M memory peak, 98K incoming IP traffic, 264.6K outgoing IP traffic.5829worker2 # [ 93.266405] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5830worker2 # [ 93.270473] systemd[1]: Closed Nix worker-protocol over gRPC.5831worker2 # [ 93.274485] systemd[1]: nix-grpc-daemon.socket: Consumed 310us CPU time over 1min 24.672s wall clock time, 256K memory peak, 109.7K incoming IP traffic, 92.3K outgoing IP traffic.5832lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5833lb: must succeed: curl -sf localhost:9901/clusters5834lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5835lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5836client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25837worker1 # [ 93.462163] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528225838client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5839worker1 # [ 93.467529] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:528225840client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5841worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5842worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5843worker2 # [ 93.551853] systemd[1]: Listening on Nix worker-protocol over gRPC.5844worker1 # [ 93.591608] systemd[1]: Stopping Nix worker-protocol over gRPC...5845worker1 # [ 93.925367] nix-grpc-daemon[783]: ts=2026-09-23T05:13:06Z level=info event=signal_exit5846worker2 # [ 94.241557] systemd[1]: Starting scheduler-token.service...5847worker2 # [ 94.303795] systemd[1]: scheduler-token.service: Deactivated successfully.5848worker2 # [ 94.307811] systemd[1]: Finished scheduler-token.service.5849worker2 # [ 94.311421] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5850worker2 # [ 94.317532] systemd[1]: Starting Nix worker-protocol over gRPC...5851worker2 # [ 94.399375] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5852worker2 # [ 94.401497] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5853worker2 # [ 94.404770] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15854worker2 # [ 94.409056] nix-grpc-daemon[1248]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5855worker2 # [ 94.410516] nix-grpc-daemon[1248]: E0000 00:00:1790140387.599046 1248 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5856worker2 # [ 94.416064] nix-grpc-daemon[1248]: E0000 00:00:1790140387.604661 1248 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5857worker2 # [ 94.418279] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5858worker2 # [ 94.423252] systemd[1]: Started Nix worker-protocol over gRPC.5859worker2 # [ 94.428951] nix-grpc-daemon[1248]: E0000 00:00:1790140387.618329 1290 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5860worker2 # [ 94.442440] nix-grpc-daemon[1248]: E0000 00:00:1790140387.632448 1290 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5861worker2 # [ 94.447054] nix-grpc-daemon[1248]: E0000 00:00:1790140387.634681 1290 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5862worker2 # [ 94.456515] nix-grpc-daemon[1248]: E0000 00:00:1790140387.646528 1290 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5863worker2 # [ 94.465896] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:500515864worker2 # [ 94.467504] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5865worker2 # [ 94.668718] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:500515866worker2 # [ 94.672857] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5867lb # [ 94.765557] niks3-server[1002]: 2026/09/23 05:13:07 INFO lead: released remote=192.168.1.3:364625868worker2 # [ 94.873942] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:500515869worker2 # [ 94.877902] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5870worker2 # [ 95.079241] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:500515871worker2 # [ 95.083201] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5872worker2 # [ 95.284597] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:500515873worker2 # [ 95.288529] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5874worker2 # [ 95.490572] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:500515875worker2 # [ 95.494654] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5876worker2 # [ 95.695696] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:500515877worker2 # [ 95.699602] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5878lb # [ 95.700244] envoy[910]: conn peer=[2001:db8:1::1]:42716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4207 tx=2054 ms=24555879lb # [ 95.703919] envoy[910]: conn peer=[2001:db8:1::1]:42732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=775880lb # [ 95.707343] envoy[910]: conn peer=[2001:db8:1::1]:42738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3700 tx=2084 ms=16335881lb # [ 95.710891] envoy[910]: conn peer=[2001:db8:1::1]:42754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=585882lb # [ 95.714497] envoy[910]: conn peer=[2001:db8:1::1]:42774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575883lb # [ 95.718591] envoy[910]: conn peer=[2001:db8:1::1]:42768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2083 ms=20415884lb # [ 95.722137] envoy[910]: conn peer=[2001:db8:1::4]:58208 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=71993 tx=3771 ms=763035885lb # [ 95.725738] envoy[910]: conn peer=[2001:db8:1::1]:54230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155886worker1 # [ 95.792902] nix-grpc-daemon[783]: ts=2026-09-23T05:13:08Z level=info event=scheduler_restarting5887worker1 # [ 95.796746] nix-grpc-daemon[783]: ts=2026-09-23T05:13:08Z level=info event=scheduler_restarting addr=lb:500515888worker1 # [ 95.800469] nix-grpc-daemon[783]: ts=2026-09-23T05:13:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:557265889worker1 # [ 95.804243] nix-grpc-daemon[783]: ts=2026-09-23T05:13:08Z level=info event=worker_gone addr=192.168.1.3:500515890worker1 # [ 95.828331] nix-daemon[850]: reaped child process 852, status = succeeded5891worker2 # [ 95.900731] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500515892worker2 # [ 95.904711] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5893worker1 # [ 96.005651] nix-grpc-daemon[783]: ts=2026-09-23T05:13:09Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5894worker1 # [ 96.074522] nix-grpc-daemon[783]: E0000 00:00:1790140389.118594 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5895worker1 # [ 96.079645] nix-grpc-daemon[783]: E0000 00:00:1790140389.119184 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5896worker2 # [ 96.105700] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500515897worker2 # [ 96.109593] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5898worker2 # [ 96.310624] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500515899worker2 # [ 96.314572] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5900worker2 # [ 96.516383] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500515901worker2 # [ 96.520100] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5902worker2 # [ 96.718383] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500515903worker2 # [ 96.722481] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5904worker2 # [ 96.923794] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:500515905worker2 # [ 96.928097] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5906worker2 # [ 97.129247] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:500515907worker2 # [ 97.133208] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5908worker1 # [ 97.215926] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5909worker1 # [ 97.219870] systemd[1]: Stopped Nix worker-protocol over gRPC.5910worker1 # [ 97.223838] systemd[1]: nix-grpc-daemon.service: Consumed 2.431s CPU time over 1min 21.500s wall clock time, 65.9M memory peak, 105.6K incoming IP traffic, 197.3K outgoing IP traffic.5911worker1 # [ 97.236146] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5912worker1 # [ 97.240598] systemd[1]: Closed Nix worker-protocol over gRPC.5913worker1 # [ 97.243822] systemd[1]: nix-grpc-daemon.socket: Consumed 308us CPU time over 1min 29.551s wall clock time, 256K memory peak, 104.2K incoming IP traffic, 89.8K outgoing IP traffic.5914lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5915lb: must succeed: curl -sf localhost:9901/clusters5916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5917lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5918client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25919worker2 # [ 97.334288] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:500515920worker2 # [ 97.338177] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5921worker2 # [ 97.453261] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452005922worker2 # [ 97.465228] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5923worker2 # [ 97.481469] nix-grpc-daemon[1248]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5924worker2 # [ 97.534820] nix-grpc-daemon[1248]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 2/5)5925worker2 # [ 97.541371] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:500515926worker2 # [ 97.545109] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5927worker2 # [ 97.598441] nix-grpc-daemon[1248]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 3/5)5928worker2 # [ 97.631191] nix-grpc-daemon[1248]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 359 ms (attempt 4/5)5929worker2 # [ 97.746424] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:500515930worker2 # [ 97.750312] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker2 # [ 97.951379] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515932worker2 # [ 97.955353] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5933worker2 # [ 97.998268] nix-grpc-daemon[1248]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5934worker2 # [ 98.001875] nix-grpc-daemon[1248]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5935client # copying 2 paths...5936client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5937worker2 # [ 98.023630] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452005938client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5939worker2 # [ 98.033757] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)5940worker2 # [ 98.054583] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=niks3_push_started pid=13095941worker2 # [ 98.155886] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515942worker2 # [ 98.157855] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5943worker2 # [ 98.163887] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.353Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5944worker2 # [ 98.252462] nix-daemon[873]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5945worker2 # [ 98.264096] nix-daemon[873]: reaped child process 1325, status = succeeded5946lb # [ 98.249508] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received uploads request method=POST path=/api/pending_closures5947lb # [ 98.258788] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received uploads request method=POST path=/api/pending_closures5948worker2 # [ 98.293418] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.483Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5949worker2 # [ 98.294972] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.485Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5950worker2 # [ 98.296986] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.487Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5951lb # [ 98.315909] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5952lb # [ 98.320676] niks3-server[1002]: 2026/09/23 05:13:10 INFO Signed narinfos id=35 count=15953lb # [ 98.322246] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5954lb # [ 98.323836] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5955lb # [ 98.330642] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5956worker2 # [ 98.358645] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.547Z level=INFO msg="Uploading 3 narinfos"5957lb # [ 98.332194] niks3-server[1002]: 2026/09/23 05:13:10 INFO Signed narinfos id=36 count=25958worker2 # [ 98.360546] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515959worker2 # [ 98.362129] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5960lb # [ 98.347378] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5961lb # [ 98.350772] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5962lb # [ 98.362073] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5963lb # [ 98.370205] niks3-server[1002]: 2026/09/23 05:13:10 INFO Completed upload id=355964lb # [ 98.371681] niks3-server[1002]: 2026/09/23 05:13:10 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5965lb # [ 98.374082] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5966lb # [ 98.376579] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5967lb # [ 98.378561] niks3-server[1002]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5968lb # [ 98.382124] niks3-server[1002]: 2026/09/23 05:13:10 INFO Completed upload id=365969worker2 # [ 98.409611] nix-grpc-daemon[1309]: time=2026-09-23T05:13:11.599Z level=INFO msg="Upload complete. (242ms)"5970worker2 # [ 98.414959] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=0 paths=2 nar_bytes_in=3125971worker2 # [ 98.429403] nix-daemon[873]: reaped child process 1307, status = succeeded5972client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.13 seconds)5973worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5974worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5975worker1 # [ 98.510821] systemd[1]: Listening on Nix worker-protocol over gRPC.5976worker1 # [ 98.519674] systemd[1]: Starting Nix worker-protocol over gRPC...5977worker2 # [ 98.564092] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515978worker2 # [ 98.567789] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5979worker1 # [ 98.604708] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5980worker1 # [ 98.606792] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5981worker1 # [ 98.610511] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15982worker1 # [ 98.615060] nix-grpc-daemon[1262]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5983worker1 # [ 98.616395] nix-grpc-daemon[1262]: E0000 00:00:1790140391.658646 1262 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5984worker1 # [ 98.621730] nix-grpc-daemon[1262]: E0000 00:00:1790140391.663864 1262 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5985worker1 # [ 98.625884] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5986worker1 # [ 98.628400] systemd[1]: Started Nix worker-protocol over gRPC.5987worker1 # [ 98.630736] nix-grpc-daemon[1262]: E0000 00:00:1790140391.675218 1305 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5988lb # [ 98.610340] niks3-server[1002]: 2026/09/23 05:13:10 INFO lead: acquired remote=192.168.1.3:473505989worker1 # [ 98.636089] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=scheduler_take_over5990lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5991lb: must succeed: curl -sf localhost:9901/clusters5992worker1 # [ 98.647230] nix-grpc-daemon[1262]: E0000 00:00:1790140391.690469 1305 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5993worker1 # [ 98.650107] nix-grpc-daemon[1262]: E0000 00:00:1790140391.690628 1305 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5994worker1 # [ 98.660336] nix-grpc-daemon[1262]: E0000 00:00:1790140391.704757 1305 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5995worker1 # [ 98.674133] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515996worker1 # [ 98.675790] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5998worker2 # [ 98.769054] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500515999worker2 # [ 98.773139] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6000worker1 # [ 98.876827] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:500516001worker1 # [ 98.880815] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6002worker2 # [ 98.974467] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:500516003worker2 # [ 98.978443] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6004worker1 # [ 99.081605] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:500516005worker1 # [ 99.085859] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6006worker2 # [ 99.179723] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:500516007worker2 # [ 99.183730] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6008worker1 # [ 99.286780] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:500516009worker1 # [ 99.294149] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:557726010worker1 # [ 99.297758] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06011worker2 # [ 99.384969] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:500516012worker1 # [ 99.388180] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=oidc_jwks provider=mock keys=16013worker1 # [ 99.391875] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:557726014worker1 # [ 99.395815] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06015lb: must succeed: curl -sf localhost:9901/clusters6016lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6017lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6018(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.59 seconds)6019subtest: a niks3 restart does not cost the leader its role6020lb: must succeed: curl -sf localhost:9901/clusters6021lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6022lb: must succeed: curl -sf localhost:9901/clusters6023lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6024lb # [ 99.790465] niks3-server[1002]: 2026/09/23 05:13:12 INFO Shutdown signal received, draining in-flight requests timeout=10s6025lb # [ 99.793511] systemd[1]: Stopping niks3 server...6026lb # [ 99.797056] niks3-server[1002]: 2026/09/23 05:13:12 INFO lead: released remote=192.168.1.3:473506027worker1 # [ 99.819984] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:12Z level=info event=scheduler_lock_lost status=2006028worker2 # [ 99.825429] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:13Z level=info event=scheduler_lock_lost status=2006029lb # [ 99.810858] systemd[1]: niks3.service: Deactivated successfully.6030lb # [ 99.813190] systemd[1]: Stopped niks3 server.6031lb # [ 99.813879] systemd[1]: niks3.service: Consumed 770ms CPU time over 1min 17.263s wall clock time, 28.7M memory peak, 93.8K incoming IP traffic, 136.6K outgoing IP traffic.6032lb # [ 99.820691] systemd[1]: Starting niks3 server...6033lb # [ 99.871265] niks3-server[1146]: 2026/09/23 05:13:12 goose: no migrations to run. current version: 202609200000006034lb # [ 99.875810] niks3-server[1146]: 2026/09/23 05:13:12 OK 1_commit_pending_closure.sql (5.75ms)6035lb # [ 99.879348] niks3-server[1146]: 2026/09/23 05:13:12 OK 2_object_stats_trigger.sql (3.15ms)6036lb # [ 99.880442] niks3-server[1146]: 2026/09/23 05:13:12 goose: up to current file version: 26037lb # [ 99.882042] niks3-server[1146]: 2026/09/23 05:13:12 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key6038lb # [ 99.893266] niks3-server[1146]: 2026/09/23 05:13:12 INFO Read proxy enabled — serving cache objects from S36039lb # [ 99.894741] niks3-server[1146]: 2026/09/23 05:13:12 INFO Using socket-activated listener address=0.0.0.0:57516040lb # [ 99.896330] niks3-server[1146]: 2026/09/23 05:13:12 INFO systemd watchdog enabled interval=15s6041lb # [ 99.898059] systemd[1]: Started niks3 server.6042lb # [ 99.899510] niks3-server[1146]: 2026/09/23 05:13:12 INFO Starting HTTP server address=0.0.0.0:57516043lb: waiting for unit niks3.service6044lb: (finished: waiting for unit niks3.service, in 0.02 seconds)6045lb: must succeed: curl -sf localhost:9901/clusters6046lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6047worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6048worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6049lb: must succeed: curl -sf localhost:9901/clusters6050lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6051lb: must succeed: curl -sf localhost:9901/clusters6052lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6053worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield6054worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)6055client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&26056worker1 # [ 100.247625] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:557786057worker1 # [ 100.259469] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6058client # don't know how to build these paths:6059client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6060worker2 # [ 100.281146] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006061worker1 # [ 100.301622] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=06062worker1 # [ 100.306548] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=16063worker1 # [ 100.351340] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786064worker1 # [ 100.363451] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)6065worker1 # [ 100.393747] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 2/5)6066worker1 # [ 100.524083] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 3/5)6067worker1 # [ 100.561799] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 4/5)6068worker1 # [ 100.674610] nix-grpc-daemon[1262]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6069worker1 # [ 100.678358] nix-grpc-daemon[1262]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6070worker1 # [ 100.695454] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557786071client # copying 1 paths to 'lb:50051'...6072client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...6073worker1 # [ 100.709272] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557786074worker1 # [ 100.717686] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6075worker1 # [ 100.741444] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:13Z level=info event=niks3_push_started pid=13336076lb # [ 100.804586] niks3-server[1146]: 2026/09/23 05:13:13 INFO lead: acquired remote=192.168.1.3:473706077worker1 # [ 100.844724] nix-grpc-daemon[1333]: time=2026-09-23T05:13:13.888Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6078worker1 # [ 100.942096] nix-daemon[850]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)6079worker1 # [ 100.953892] nix-daemon[850]: reaped child process 1351, status = succeeded6080lb # [ 100.940446] niks3-server[1146]: 2026/09/23 05:13:13 INFO Received uploads request method=POST path=/api/pending_closures6081worker1 # [ 100.974559] nix-grpc-daemon[1333]: time=2026-09-23T05:13:14.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6082worker1 # [ 100.976193] nix-grpc-daemon[1333]: time=2026-09-23T05:13:14.020Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6083lb # [ 100.990820] niks3-server[1146]: 2026/09/23 05:13:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6084lb # [ 100.997884] niks3-server[1146]: 2026/09/23 05:13:13 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6085lb # [ 101.015039] niks3-server[1146]: 2026/09/23 05:13:13 INFO Signed narinfos id=37 count=16086worker1 # [ 101.038914] nix-grpc-daemon[1333]: time=2026-09-23T05:13:14.083Z level=INFO msg="Uploading 1 narinfos"6087lb # [ 101.020884] niks3-server[1146]: 2026/09/23 05:13:13 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6088lb # [ 101.026484] niks3-server[1146]: 2026/09/23 05:13:13 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6089lb # [ 101.034407] niks3-server[1146]: 2026/09/23 05:13:13 INFO Completed upload id=376090worker1 # [ 101.056884] nix-grpc-daemon[1333]: time=2026-09-23T05:13:14.101Z level=INFO msg="Upload complete. (208ms)"6091lb # [ 101.037643] niks3-server[1146]: 2026/09/23 05:13:13 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6092worker1 # [ 101.062644] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=0 paths=1 nar_bytes_in=5366093worker1 # [ 101.070258] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786094worker1 # [ 101.078724] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6095worker1 # [ 101.087864] nix-daemon[850]: reaped child process 1331, status = succeeded6096worker1 # [ 101.092040] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6097client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6098client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6099client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6100client # farm-a> LOG-farm-a-after-niks3-restart6101worker1 # [ 101.915610] nix-daemon[850]: reaped child process 1355, status = succeeded6102worker1 # [ 101.993362] nix-daemon[850]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)6103worker1 # [ 102.006728] nix-daemon[850]: reaped child process 1374, status = succeeded6104lb # [ 101.992975] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received uploads request method=POST path=/api/pending_closures6105worker1 # [ 102.027352] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.071Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6106worker1 # [ 102.029055] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.073Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6107lb # [ 102.041906] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6108lb # [ 102.049267] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6109worker1 # [ 102.081456] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.125Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 102.060313] niks3-server[1146]: 2026/09/23 05:13:14 INFO Signed narinfos id=38 count=16111lb # [ 102.064228] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6112lb # [ 102.070146] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6113lb # [ 102.078171] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6114lb # [ 102.083692] niks3-server[1146]: 2026/09/23 05:13:14 INFO Completed upload id=386115lb # [ 102.086248] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6116worker1 # [ 102.106173] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.150Z level=INFO msg="Upload complete. (207ms)"6117worker1 # [ 102.115698] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16118worker1 # [ 102.128464] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06119worker1 # [ 102.132457] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=16120worker1 # [ 102.142546] nix-daemon[850]: reaped child process 1353, status = succeeded6121worker1 # [ 102.171808] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786122worker1 # [ 102.184467] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557786123worker1 # [ 102.198207] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.242Z level=INFO msg="All 1 paths already cached"6124client # copying 1 paths to 'lb:50051'...6125client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6126worker1 # [ 102.206888] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557786127worker1 # [ 102.215385] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6128worker1 # [ 102.312813] nix-daemon[850]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)6129worker1 # [ 102.325952] nix-daemon[850]: reaped child process 1383, status = succeeded6130lb # [ 102.310614] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received uploads request method=POST path=/api/pending_closures6131worker1 # [ 102.343043] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.387Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6132worker1 # [ 102.344950] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.389Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6133lb # [ 102.344078] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6134lb # [ 102.351103] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6135lb # [ 102.355889] niks3-server[1146]: 2026/09/23 05:13:14 INFO Signed narinfos id=39 count=16136worker1 # [ 102.381182] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.425Z level=INFO msg="Uploading 1 narinfos"6137lb # [ 102.369521] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6138lb # [ 102.375628] niks3-server[1146]: 2026/09/23 05:13:14 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6139lb # [ 102.383575] niks3-server[1146]: 2026/09/23 05:13:14 INFO Completed upload id=396140worker1 # [ 102.405399] nix-grpc-daemon[1333]: time=2026-09-23T05:13:15.449Z level=INFO msg="Upload complete. (177ms)"6141lb # [ 102.385742] niks3-server[1146]: 2026/09/23 05:13:14 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6142worker1 # [ 102.409644] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=0 paths=1 nar_bytes_in=6566143worker1 # [ 102.414231] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786144worker1 # [ 102.423553] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6145worker1 # [ 102.433160] nix-daemon[850]: reaped child process 1378, status = succeeded6146worker1 # [ 102.436605] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6147client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6148client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6149client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6150client # farm-b> LOG-farm-b-after-niks3-restart6151worker1 # [ 103.877104] nix-daemon[850]: reaped child process 1387, status = succeeded6152worker1 # [ 103.957766] nix-daemon[850]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)6153worker1 # [ 103.971246] nix-daemon[850]: reaped child process 1406, status = succeeded6154lb # [ 103.957466] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received uploads request method=POST path=/api/pending_closures6155worker1 # [ 103.987202] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.031Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6156worker1 # [ 103.989119] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.033Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6157lb # [ 103.998555] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6158worker1 # [ 104.028424] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.072Z level=INFO msg="Uploading 1 narinfos"6159lb # [ 104.006584] niks3-server[1146]: 2026/09/23 05:13:16 INFO Signed narinfos id=40 count=16160lb # [ 104.014842] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6161lb # [ 104.019912] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6162lb # [ 104.027470] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6163lb # [ 104.038741] niks3-server[1146]: 2026/09/23 05:13:16 INFO Completed upload id=406164worker1 # [ 104.061262] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.105Z level=INFO msg="Upload complete. (215ms)"6165lb # [ 104.041385] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6166lb # [ 104.045381] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6167worker1 # [ 104.068524] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16168worker1 # [ 104.079632] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06169worker1 # [ 104.083762] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=16170worker1 # [ 104.094120] nix-daemon[850]: reaped child process 1385, status = succeeded6171worker1 # [ 104.124593] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786172worker1 # [ 104.136867] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557786173worker1 # [ 104.150553] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.194Z level=INFO msg="All 2 paths already cached"6174client # copying 1 paths to 'lb:50051'...6175client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6176worker1 # [ 104.159748] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557786177worker1 # [ 104.170139] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6178worker1 # [ 104.263557] nix-daemon[850]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)6179worker1 # [ 104.277472] nix-daemon[850]: reaped child process 1415, status = succeeded6180lb # [ 104.261897] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received uploads request method=POST path=/api/pending_closures6181worker1 # [ 104.293396] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.337Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6182worker1 # [ 104.294866] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.339Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6183lb # [ 104.297147] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6184worker1 # [ 104.325283] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.369Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 104.303114] niks3-server[1146]: 2026/09/23 05:13:16 INFO Signed narinfos id=41 count=16186lb # [ 104.311844] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6187lb # [ 104.316686] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6188lb # [ 104.324818] niks3-server[1146]: 2026/09/23 05:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6189lb # [ 104.329555] niks3-server[1146]: 2026/09/23 05:13:16 INFO Completed upload id=416190worker1 # [ 104.351445] nix-grpc-daemon[1333]: time=2026-09-23T05:13:17.395Z level=INFO msg="Upload complete. (171ms)"6191lb # [ 104.333820] niks3-server[1146]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6192worker1 # [ 104.354251] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=0 paths=1 nar_bytes_in=7846193worker1 # [ 104.359514] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557786194worker1 # [ 104.369252] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6195worker1 # [ 104.376193] nix-daemon[850]: reaped child process 1410, status = succeeded6196worker1 # [ 104.379278] nix-daemon[850]: accepted connection from pid 1262, user nix-grpc-daemon (trusted)6197client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6198client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6199client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6200client # farm-top> LOG-farm-top-after-niks3-restart6201worker1 # [ 105.617069] nix-daemon[850]: reaped child process 1419, status = succeeded6202worker1 # [ 105.700908] nix-daemon[850]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)6203worker1 # [ 105.712814] nix-daemon[850]: reaped child process 1438, status = succeeded6204lb # [ 105.698569] niks3-server[1146]: 2026/09/23 05:13:17 INFO Received uploads request method=POST path=/api/pending_closures6205lb # [ 105.705619] envoy[910]: conn peer=[2001:db8:1::3]:53754 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=20326 tx=4174 ms=800446206lb # [ 105.709408] envoy[910]: conn peer=[2001:db8:1::1]:54244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=282 ms=9766207worker1 # [ 105.737957] nix-grpc-daemon[1333]: time=2026-09-23T05:13:18.782Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6208worker1 # [ 105.739496] nix-grpc-daemon[1333]: time=2026-09-23T05:13:18.783Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6209lb # [ 105.748150] niks3-server[1146]: 2026/09/23 05:13:17 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6210lb # [ 105.754152] niks3-server[1146]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6211lb # [ 105.757538] niks3-server[1146]: 2026/09/23 05:13:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6212lb # [ 105.762672] niks3-server[1146]: 2026/09/23 05:13:18 INFO Signed narinfos id=42 count=16213worker1 # [ 105.787622] nix-grpc-daemon[1333]: time=2026-09-23T05:13:18.831Z level=INFO msg="Uploading 1 narinfos"6214lb # [ 105.769414] niks3-server[1146]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6215lb # [ 105.776311] niks3-server[1146]: 2026/09/23 05:13:18 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6216lb # [ 105.780725] niks3-server[1146]: 2026/09/23 05:13:18 INFO Completed upload id=426217lb # [ 105.782610] niks3-server[1146]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6218worker1 # [ 105.804454] nix-grpc-daemon[1333]: time=2026-09-23T05:13:18.848Z level=INFO msg="Upload complete. (215ms)"6219worker1 # [ 105.811858] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55778 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16220worker1 # [ 105.826442] nix-daemon[850]: reaped child process 1417, status = succeeded6221client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.75 seconds)6222client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6223client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)6224(finished: subtest: a niks3 restart does not cost the leader its role, in 6.22 seconds)6225subtest: scheduler down: the other node takes the lock and keeps it6226lb: must succeed: curl -sf localhost:9901/clusters6227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6228lb: must succeed: curl -sf localhost:9901/clusters6229lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6230lb: must succeed: curl -sf localhost:9901/clusters6231lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6232lb: must succeed: curl -sf localhost:9901/clusters6233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6234worker1 # [ 106.119244] systemd[1]: Stopping Nix worker-protocol over gRPC...6235worker1 # [ 106.622652] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:19Z level=info event=signal_exit6236lb # [ 106.808812] niks3-server[1146]: 2026/09/23 05:13:19 INFO lead: released remote=192.168.1.3:473706237worker2 # [ 107.837776] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_restarting addr=lb:500516238worker1 # [ 107.836112] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=scheduler_restarting6239worker1 # [ 107.839496] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=scheduler_restarting addr=lb:500516240worker1 # [ 107.843250] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:557726241worker1 # [ 107.846884] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=worker_gone addr=192.168.1.3:500516242worker1 # [ 107.850401] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:557726243worker1 # [ 107.853995] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:20Z level=info event=worker_gone addr=192.168.1.4:500516244worker1 # [ 107.870519] nix-daemon[850]: reaped child process 1323, status = succeeded6245worker2 # [ 108.038137] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_connected addr=lb:500516246worker2 # [ 108.042388] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6247worker1 # [ 108.044977] nix-grpc-daemon[1262]: ts=2026-09-23T05:13:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6248worker2 # [ 108.243607] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_connected addr=lb:500516249worker2 # [ 108.247578] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6250worker1 # [ 108.245684] nix-grpc-daemon[1262]: E0000 00:00:1790140401.289793 1308 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6251worker1 # [ 108.250695] nix-grpc-daemon[1262]: E0000 00:00:1790140401.290053 1308 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6252worker2 # [ 108.448730] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_connected addr=lb:500516253worker2 # [ 108.452652] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6254worker2 # [ 108.653663] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_connected addr=lb:500516255worker2 # [ 108.657653] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6256worker2 # [ 108.859693] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516257worker2 # [ 108.863609] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6258worker2 # [ 109.064744] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516259worker2 # [ 109.068827] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6260worker2 # [ 109.269963] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516261worker2 # [ 109.273903] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6262worker2 # [ 109.474830] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516263worker2 # [ 109.478726] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6264worker2 # [ 109.679764] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516265worker2 # [ 109.683770] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6266worker2 # [ 109.885772] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_connected addr=lb:500516267worker2 # [ 109.889738] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6268worker1 # [ 110.043182] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6269worker1 # [ 110.046269] systemd[1]: Stopped Nix worker-protocol over gRPC.6270worker1 # [ 110.049745] systemd[1]: nix-grpc-daemon.service: Consumed 810ms CPU time over 11.524s wall clock time, 37.4M memory peak, 36.2K incoming IP traffic, 60.7K outgoing IP traffic.6271worker1 # [ 110.060911] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6272worker1 # [ 110.065446] systemd[1]: Closed Nix worker-protocol over gRPC.6273worker1 # [ 110.069929] systemd[1]: nix-grpc-daemon.socket: Consumed 682us CPU time over 11.562s wall clock time, 256K memory peak, 33.1K incoming IP traffic, 24.1K outgoing IP traffic.6274worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6275worker2 # [ 110.091130] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_connected addr=lb:500516276worker2 # [ 110.095358] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6277worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)6278worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6279worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6280worker2 # [ 110.296408] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_connected addr=lb:500516281worker2 # [ 110.300296] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282worker2 # [ 110.501288] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_connected addr=lb:500516283worker2 # [ 110.505321] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6284worker2 # [ 110.706351] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 110.710275] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6286worker2 # [ 110.834423] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_take_over6287lb # [ 110.807257] niks3-server[1146]: 2026/09/23 05:13:23 INFO lead: acquired remote=192.168.1.4:547866288worker2 # [ 110.911903] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_connected addr=lb:500516289worker2 # [ 110.915802] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6290worker2 # [ 111.116683] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_connected addr=lb:500516291worker2 # [ 111.120632] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6292worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6293worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6294worker2 # [ 111.321621] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_connected addr=lb:500516295worker2 # [ 111.325636] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6296worker2 # [ 111.526882] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=scheduler_connected addr=lb:500516297worker2 # [ 111.545148] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=oidc_jwks provider=mock keys=16298worker2 # [ 111.548812] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355606299worker2 # [ 111.552745] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06300worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6301worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6302client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26303worker2 # [ 112.399650] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:452006304client # don't know how to build these paths:6305client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6306worker2 # [ 112.408225] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006307worker2 # [ 112.433111] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06308worker2 # [ 112.438503] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16309worker2 # [ 112.477088] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006310worker2 # [ 112.490168] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006311client # copying 1 paths to 'lb:50051'...6312client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6313worker2 # [ 112.503355] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452006314worker2 # [ 112.514863] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6315worker2 # [ 112.615823] nix-daemon[873]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)6316worker2 # [ 112.630248] nix-daemon[873]: reaped child process 1354, status = succeeded6317lb # [ 112.608887] niks3-server[1146]: 2026/09/23 05:13:24 INFO Received uploads request method=POST path=/api/pending_closures6318worker2 # [ 112.644200] nix-grpc-daemon[1309]: time=2026-09-23T05:13:25.834Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6319worker2 # [ 112.646431] nix-grpc-daemon[1309]: time=2026-09-23T05:13:25.835Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6320lb # [ 112.639868] niks3-server[1146]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6321lb # [ 112.646358] niks3-server[1146]: 2026/09/23 05:13:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6322lb # [ 112.651241] niks3-server[1146]: 2026/09/23 05:13:24 INFO Signed narinfos id=43 count=16323worker2 # [ 112.681494] nix-grpc-daemon[1309]: time=2026-09-23T05:13:25.871Z level=INFO msg="Uploading 1 narinfos"6324lb # [ 112.664604] niks3-server[1146]: 2026/09/23 05:13:24 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6325worker2 # [ 112.704408] nix-grpc-daemon[1309]: time=2026-09-23T05:13:25.894Z level=INFO msg="Upload complete. (178ms)"6326lb # [ 112.677976] niks3-server[1146]: 2026/09/23 05:13:24 INFO Completed upload id=436327worker2 # [ 112.707532] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=0 paths=1 nar_bytes_in=5206328lb # [ 112.684654] niks3-server[1146]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6329worker2 # [ 112.713216] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006330lb # [ 112.690827] niks3-server[1146]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6331worker2 # [ 112.721179] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6332worker2 # [ 112.726134] nix-daemon[873]: reaped child process 1349, status = succeeded6333worker2 # [ 112.730710] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6334client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6335client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6336client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6337client # farm-a> LOG-farm-a-on-standby6338worker2 # [ 114.566243] nix-daemon[873]: reaped child process 1358, status = succeeded6339worker2 # [ 114.654457] nix-daemon[873]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6340worker2 # [ 114.666132] nix-daemon[873]: reaped child process 1377, status = succeeded6341lb # [ 114.646545] niks3-server[1146]: 2026/09/23 05:13:26 INFO Received uploads request method=POST path=/api/pending_closures6342worker2 # [ 114.686648] nix-grpc-daemon[1309]: time=2026-09-23T05:13:27.876Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6343worker2 # [ 114.688289] nix-grpc-daemon[1309]: time=2026-09-23T05:13:27.878Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6344lb # [ 114.688118] niks3-server[1146]: 2026/09/23 05:13:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6345worker2 # [ 114.720902] nix-grpc-daemon[1309]: time=2026-09-23T05:13:27.910Z level=INFO msg="Uploading 1 narinfos"6346lb # [ 114.693503] niks3-server[1146]: 2026/09/23 05:13:26 INFO Signed narinfos id=44 count=16347lb # [ 114.704945] niks3-server[1146]: 2026/09/23 05:13:26 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6348lb # [ 114.707779] niks3-server[1146]: 2026/09/23 05:13:26 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6349lb # [ 114.712595] niks3-server[1146]: 2026/09/23 05:13:26 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6350lb # [ 114.717731] niks3-server[1146]: 2026/09/23 05:13:26 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6351lb # [ 114.721489] niks3-server[1146]: 2026/09/23 05:13:26 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6352lb # [ 114.724936] niks3-server[1146]: 2026/09/23 05:13:26 INFO Completed upload id=446353worker2 # [ 114.752384] nix-grpc-daemon[1309]: time=2026-09-23T05:13:27.942Z level=INFO msg="Upload complete. (211ms)"6354worker2 # [ 114.763917] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=2 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16355worker2 # [ 114.770697] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:27Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06356worker2 # [ 114.774184] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:27Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16357worker2 # [ 114.782773] nix-daemon[873]: reaped child process 1356, status = succeeded6358worker2 # [ 114.815121] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006359worker2 # [ 114.827806] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006360worker2 # [ 114.843446] nix-grpc-daemon[1309]: time=2026-09-23T05:13:28.033Z level=INFO msg="All 1 paths already cached"6361client # copying 1 paths to 'lb:50051'...6362client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6363worker2 # [ 114.852445] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452006364worker2 # [ 114.862169] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6365worker2 # [ 114.957444] nix-daemon[873]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)6366worker2 # [ 114.970083] nix-daemon[873]: reaped child process 1386, status = succeeded6367lb # [ 114.950175] niks3-server[1146]: 2026/09/23 05:13:27 INFO Received uploads request method=POST path=/api/pending_closures6368worker2 # [ 114.986900] nix-grpc-daemon[1309]: time=2026-09-23T05:13:28.176Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6369worker2 # [ 114.988426] nix-grpc-daemon[1309]: time=2026-09-23T05:13:28.178Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6370lb # [ 114.984631] niks3-server[1146]: 2026/09/23 05:13:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6371lb # [ 114.992151] niks3-server[1146]: 2026/09/23 05:13:27 INFO Signed narinfos id=45 count=16372worker2 # [ 115.021669] nix-grpc-daemon[1309]: time=2026-09-23T05:13:28.211Z level=INFO msg="Uploading 1 narinfos"6373lb # [ 114.997107] niks3-server[1146]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6374lb # [ 115.003578] niks3-server[1146]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6375lb # [ 115.011569] niks3-server[1146]: 2026/09/23 05:13:27 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6376lb # [ 115.015956] niks3-server[1146]: 2026/09/23 05:13:27 INFO Completed upload id=456377worker2 # [ 115.043206] nix-grpc-daemon[1309]: time=2026-09-23T05:13:28.233Z level=INFO msg="Upload complete. (169ms)"6378lb # [ 115.018615] niks3-server[1146]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6379worker2 # [ 115.047467] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=0 paths=1 nar_bytes_in=6406380worker2 # [ 115.051924] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006381worker2 # [ 115.063988] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6382worker2 # [ 115.066744] nix-daemon[873]: reaped child process 1381, status = succeeded6383worker2 # [ 115.070584] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6384client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6385client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6386lb # [ 115.703992] envoy[910]: conn peer=[2001:db8:1::1]:54246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9726 tx=4438 ms=55866387lb # [ 115.708097] envoy[910]: conn peer=[2001:db8:1::3]:50464 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1726 tx=592 ms=93706388client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6389client # farm-b> LOG-farm-b-on-standby6390worker2 # [ 116.107877] nix-daemon[873]: reaped child process 1390, status = succeeded6391worker2 # [ 116.191589] nix-daemon[873]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6392worker2 # [ 116.204545] nix-daemon[873]: reaped child process 1409, status = succeeded6393lb # [ 116.186691] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received uploads request method=POST path=/api/pending_closures6394worker2 # [ 116.221062] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.410Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6395worker2 # [ 116.222620] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.412Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6396lb # [ 116.229225] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6397lb # [ 116.239076] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6398worker2 # [ 116.272821] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.462Z level=INFO msg="Uploading 1 narinfos"6399lb # [ 116.245623] niks3-server[1146]: 2026/09/23 05:13:28 INFO Signed narinfos id=46 count=16400lb # [ 116.251112] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6401lb # [ 116.255567] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6402lb # [ 116.267150] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6403lb # [ 116.270197] niks3-server[1146]: 2026/09/23 05:13:28 INFO Completed upload id=466404worker2 # [ 116.297473] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.487Z level=INFO msg="Upload complete. (219ms)"6405lb # [ 116.273635] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6406worker2 # [ 116.305890] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16407worker2 # [ 116.316876] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06408worker2 # [ 116.320473] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16409worker2 # [ 116.329859] nix-daemon[873]: reaped child process 1388, status = succeeded6410worker2 # [ 116.361934] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006411worker2 # [ 116.374426] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006412worker2 # [ 116.387776] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.577Z level=INFO msg="All 2 paths already cached"6413client # copying 1 paths to 'lb:50051'...6414client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6415worker2 # [ 116.396770] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452006416worker2 # [ 116.407134] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6417worker2 # [ 116.502847] nix-daemon[873]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6418worker2 # [ 116.516667] nix-daemon[873]: reaped child process 1418, status = succeeded6419lb # [ 116.495126] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received uploads request method=POST path=/api/pending_closures6420worker2 # [ 116.531057] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.721Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6421worker2 # [ 116.532593] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.722Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6422lb # [ 116.526657] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6423lb # [ 116.537560] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6424worker2 # [ 116.571739] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.761Z level=INFO msg="Uploading 1 narinfos"6425lb # [ 116.544759] niks3-server[1146]: 2026/09/23 05:13:28 INFO Signed narinfos id=47 count=16426lb # [ 116.550111] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6427lb # [ 116.563864] niks3-server[1146]: 2026/09/23 05:13:28 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6428lb # [ 116.571251] niks3-server[1146]: 2026/09/23 05:13:28 INFO Completed upload id=476429lb # [ 116.572811] niks3-server[1146]: 2026/09/23 05:13:28 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6430worker2 # [ 116.598711] nix-grpc-daemon[1309]: time=2026-09-23T05:13:29.788Z level=INFO msg="Upload complete. (180ms)"6431worker2 # [ 116.603359] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=0 paths=1 nar_bytes_in=7686432worker2 # [ 116.609255] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006433worker2 # [ 116.620200] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6434worker2 # [ 116.629142] nix-daemon[873]: reaped child process 1413, status = succeeded6435worker2 # [ 116.634584] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6436client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6437client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6438client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6439client # farm-top> LOG-farm-top-on-standby6440worker2 # [ 117.675130] nix-daemon[873]: reaped child process 1422, status = succeeded6441worker2 # [ 117.758691] nix-daemon[873]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6442worker2 # [ 117.770814] nix-daemon[873]: reaped child process 1441, status = succeeded6443lb # [ 117.751332] niks3-server[1146]: 2026/09/23 05:13:29 INFO Received uploads request method=POST path=/api/pending_closures6444worker2 # [ 117.787335] nix-grpc-daemon[1309]: time=2026-09-23T05:13:30.977Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6445worker2 # [ 117.789036] nix-grpc-daemon[1309]: time=2026-09-23T05:13:30.978Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6446lb # [ 117.795589] niks3-server[1146]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6447lb # [ 117.800241] niks3-server[1146]: 2026/09/23 05:13:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6448worker2 # [ 117.835294] nix-grpc-daemon[1309]: time=2026-09-23T05:13:31.025Z level=INFO msg="Uploading 1 narinfos"6449lb # [ 117.808789] niks3-server[1146]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6450lb # [ 117.811833] niks3-server[1146]: 2026/09/23 05:13:30 INFO Signed narinfos id=48 count=16451lb # [ 117.819278] niks3-server[1146]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6452lb # [ 117.827190] niks3-server[1146]: 2026/09/23 05:13:30 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6453lb # [ 117.830870] niks3-server[1146]: 2026/09/23 05:13:30 INFO Completed upload id=486454worker2 # [ 117.858386] nix-grpc-daemon[1309]: time=2026-09-23T05:13:31.048Z level=INFO msg="Upload complete. (215ms)"6455lb # [ 117.833806] niks3-server[1146]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6456worker2 # [ 117.864757] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16457worker2 # [ 117.881056] nix-daemon[873]: reaped child process 1420, status = succeeded6458client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.69 seconds)6459client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6460client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6461worker1 # [ 118.102469] systemd[1]: Listening on Nix worker-protocol over gRPC.6462worker1 # [ 118.108726] systemd[1]: Starting Nix worker-protocol over gRPC...6463worker1 # [ 118.197976] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6464worker1 # [ 118.200162] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6465worker1 # [ 118.204165] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16466worker1 # [ 118.208077] nix-grpc-daemon[1456]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6467worker1 # [ 118.209418] nix-grpc-daemon[1456]: E0000 00:00:1790140411.252438 1456 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6468worker1 # [ 118.216074] nix-grpc-daemon[1456]: E0000 00:00:1790140411.257695 1456 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6469worker1 # [ 118.219603] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6470worker1 # [ 118.221258] systemd[1]: Started Nix worker-protocol over gRPC.6471worker1 # [ 118.226969] nix-grpc-daemon[1456]: E0000 00:00:1790140411.269594 1498 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6472lb: must succeed: curl -sf localhost:9901/clusters6473worker1 # [ 118.238641] nix-grpc-daemon[1456]: E0000 00:00:1790140411.283109 1498 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6474worker1 # [ 118.242166] nix-grpc-daemon[1456]: E0000 00:00:1790140411.285331 1498 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6475worker1 # [ 118.252893] nix-grpc-daemon[1456]: E0000 00:00:1790140411.297322 1498 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6476worker1 # [ 118.265094] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:31Z level=info event=scheduler_connected addr=lb:500516477worker2 # [ 118.272061] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:355606478worker2 # [ 118.275879] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06479lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6480worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6481worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6482lb: must succeed: curl -sf localhost:9901/clusters6483lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6484lb: must succeed: curl -sf localhost:9901/clusters6485lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6486worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6487worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6488(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.47 seconds)6489subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6490client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6491client # [ 118.500534] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6492client # Running as unit: rs.service; invocation ID: 034c1763ddbd46f680ef3c361eac005d6493client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6494client # [ 118.634714] nix[1358]: don't know how to build these paths:6495client # [ 118.635672] nix[1358]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6496worker2 # [ 118.619642] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:452006497worker2 # [ 118.630103] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06498worker2 # [ 118.635748] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16499worker2 # [ 118.674728] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006500worker2 # [ 118.686935] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:452006501client # [ 118.710193] nix[1358]: copying 1 paths to 'lb:50051'...6502worker2 # [ 118.698185] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:452006503client # [ 118.713148] nix[1358]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6504worker2 # [ 118.711112] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6505worker2 # [ 118.808166] nix-daemon[873]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)6506worker2 # [ 118.820731] nix-daemon[873]: reaped child process 1457, status = succeeded6507lb # [ 118.801852] niks3-server[1146]: 2026/09/23 05:13:31 INFO Received uploads request method=POST path=/api/pending_closures6508worker2 # [ 118.838120] nix-grpc-daemon[1309]: time=2026-09-23T05:13:32.027Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6509worker2 # [ 118.840073] nix-grpc-daemon[1309]: time=2026-09-23T05:13:32.029Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6510lb # [ 118.833871] niks3-server[1146]: 2026/09/23 05:13:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6511worker2 # [ 118.868964] nix-grpc-daemon[1309]: time=2026-09-23T05:13:32.058Z level=INFO msg="Uploading 1 narinfos"6512lb # [ 118.841458] niks3-server[1146]: 2026/09/23 05:13:31 INFO Signed narinfos id=49 count=16513lb # [ 118.849764] niks3-server[1146]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6514lb # [ 118.856231] niks3-server[1146]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6515lb # [ 118.864468] niks3-server[1146]: 2026/09/23 05:13:31 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6516lb # [ 118.873491] niks3-server[1146]: 2026/09/23 05:13:31 INFO Completed upload id=496517worker2 # [ 118.900773] nix-grpc-daemon[1309]: time=2026-09-23T05:13:32.090Z level=INFO msg="Upload complete. (178ms)"6518lb # [ 118.874764] niks3-server[1146]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6519worker2 # [ 118.905795] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=0 paths=1 nar_bytes_in=4166520worker2 # [ 118.911304] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:452006521worker2 # [ 118.919488] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6522worker2 # [ 118.927904] nix-daemon[873]: reaped child process 1452, status = succeeded6523worker2 # [ 118.932186] nix-daemon[873]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)6524client # [ 118.945295] nix[1358]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6525lb: must succeed: curl -sf localhost:9901/clusters6526lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6527lb: must succeed: curl -sf localhost:9901/clusters6528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6529lb: must succeed: curl -sf localhost:9901/clusters6530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6531lb: must succeed: curl -sf localhost:9901/clusters6532lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6533worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6534worker1 # [ 121.162477] systemd-journald[284]: Received client request to rotate journal, rotating.6535worker1 # [ 121.181351] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/b241ce36aa3840dbb28cbd4d785d478e.6536worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6537worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6538worker2 # [ 121.097340] systemd[1]: Stopping Nix worker-protocol over gRPC...6539worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6540client: must succeed: sleep 5.06541worker2 # [ 121.431216] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:34Z level=info event=draining6542lb # [ 125.704974] envoy[910]: conn peer=[2001:db8:1::1]:57504 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9599 tx=4345 ms=55246543client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6544worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6545worker2 # [ 126.353765] nix-daemon[873]: reaped child process 1461, status = succeeded6546worker2 # [ 126.428771] nix-daemon[873]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)6547worker2 # [ 126.442154] nix-daemon[873]: reaped child process 1506, status = succeeded6548lb # [ 126.422793] niks3-server[1146]: 2026/09/23 05:13:38 INFO Received uploads request method=POST path=/api/pending_closures6549worker2 # [ 126.459309] nix-grpc-daemon[1309]: time=2026-09-23T05:13:39.649Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6550worker2 # [ 126.462059] nix-grpc-daemon[1309]: time=2026-09-23T05:13:39.650Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6551worker2 # [ 126.495109] nix-grpc-daemon[1309]: time=2026-09-23T05:13:39.685Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 126.467757] niks3-server[1146]: 2026/09/23 05:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6553lb # [ 126.470813] niks3-server[1146]: 2026/09/23 05:13:38 INFO Signed narinfos id=50 count=16554lb # [ 126.476359] niks3-server[1146]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6555lb # [ 126.480557] niks3-server[1146]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6556lb # [ 126.488059] niks3-server[1146]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6557lb # [ 126.498442] niks3-server[1146]: 2026/09/23 05:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6558lb # [ 126.502214] niks3-server[1146]: 2026/09/23 05:13:38 INFO Completed upload id=506559worker2 # [ 126.529564] nix-grpc-daemon[1309]: time=2026-09-23T05:13:39.719Z level=INFO msg="Upload complete. (184ms)"6560lb # [ 126.504076] niks3-server[1146]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6561worker2 # [ 126.536686] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45200 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16562worker2 # [ 126.550766] nix-daemon[873]: reaped child process 1459, status = succeeded6563client # [ 126.576292] systemd[1]: rs.service: Deactivated successfully.6564client # [ 126.578277] systemd[1]: rs.service: Consumed 117ms CPU time over 8.077s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.6565worker2 # [ 127.432607] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:40Z level=info event=signal_exit6566lb # [ 127.812925] niks3-server[1146]: 2026/09/23 05:13:40 INFO lead: released remote=192.168.1.4:547866567lb # [ 128.205844] niks3-server[1146]: 2026/09/23 05:13:40 INFO lead: acquired remote=192.168.1.3:509266568worker1 # [ 128.228989] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:41Z level=info event=scheduler_take_over6569worker1 # [ 128.835353] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:41Z level=info event=scheduler_restarting addr=lb:500516570worker2 # [ 128.843997] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=scheduler_restarting6571worker2 # [ 128.847910] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=scheduler_restarting addr=lb:500516572worker2 # [ 128.851515] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:355606573worker2 # [ 128.854984] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=worker_gone addr=192.168.1.3:500516574worker2 # [ 128.858448] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:355606575worker2 # [ 128.862205] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=worker_gone addr=192.168.1.4:500516576worker2 # [ 128.878925] nix-daemon[873]: reaped child process 1301, status = succeeded6577worker1 # [ 129.036308] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516578worker1 # [ 129.040428] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6579worker2 # [ 129.054442] nix-grpc-daemon[1248]: ts=2026-09-23T05:13:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6580worker1 # [ 129.241696] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516581worker1 # [ 129.248382] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:330386582worker1 # [ 129.253369] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06583worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.17 seconds)6584client: waiting for success: ! systemctl is-active rs6585worker2 # [ 129.495925] nix-grpc-daemon[1248]: E0000 00:00:1790140422.685813 1294 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6586worker2 # [ 129.501439] nix-grpc-daemon[1248]: E0000 00:00:1790140422.691310 1294 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6587client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6588client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6589client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6590client: must fail: journalctl -u rs | grep -q 'reconnecting'6591client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6592lb: must succeed: curl -sf localhost:9901/clusters6593lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6594worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6595worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6596worker2 # [ 130.373492] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6597worker2 # [ 130.376517] systemd[1]: Stopped Nix worker-protocol over gRPC.6598worker2 # [ 130.379309] systemd[1]: nix-grpc-daemon.service: Consumed 1.241s CPU time over 36.055s wall clock time, 41.5M memory peak, 64.3K incoming IP traffic, 167.9K outgoing IP traffic.6599worker2 # [ 130.395173] systemd[1]: Starting scheduler-token.service...6600worker2 # [ 130.442662] systemd[1]: scheduler-token.service: Deactivated successfully.6601worker2 # [ 130.444270] systemd[1]: Finished scheduler-token.service.6602worker2 # [ 130.445415] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 50ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6603worker2 # [ 130.450099] systemd[1]: Starting Nix worker-protocol over gRPC...6604worker2 # [ 130.528309] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6605worker2 # [ 130.532648] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6606worker2 # [ 130.536186] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16607worker2 # [ 130.538167] nix-grpc-daemon[1516]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6608worker2 # [ 130.539522] nix-grpc-daemon[1516]: E0000 00:00:1790140423.725657 1516 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6609worker2 # [ 130.546327] nix-grpc-daemon[1516]: E0000 00:00:1790140423.732772 1516 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6610worker2 # [ 130.549550] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6611worker2 # [ 130.553048] systemd[1]: Started Nix worker-protocol over gRPC.6612worker2 # [ 130.553906] nix-grpc-daemon[1516]: E0000 00:00:1790140423.741859 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6613worker2 # [ 130.564604] nix-grpc-daemon[1516]: E0000 00:00:1790140423.754623 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6614worker2 # [ 130.569053] nix-grpc-daemon[1516]: E0000 00:00:1790140423.756930 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6615worker2 # [ 130.578863] nix-grpc-daemon[1516]: E0000 00:00:1790140423.768891 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6616worker2 # [ 130.589312] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500516617worker1 # [ 130.592841] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:43Z level=info event=oidc_jwks provider=mock keys=16618worker1 # [ 130.597157] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:330386619worker1 # [ 130.601089] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06620lb: must succeed: curl -sf localhost:9901/clusters6621lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6622worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6623worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6624(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.32 seconds)6625subtest: scheduler restart mid-build: build finishes, no double build6626client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6627client # [ 130.824789] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6628client # Running as unit: mid.service; invocation ID: 35a5fa302c8f4418a121dabd53d9fe6f6629client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6630worker1 # [ 130.943776] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330486631worker1 # [ 130.955189] nix-daemon[850]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)6632client # [ 130.986176] nix[1394]: don't know how to build these paths:6633client # [ 130.987094] nix[1394]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6634worker1 # [ 130.974758] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06635worker1 # [ 130.979198] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16636worker1 # [ 131.028710] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330486637worker1 # [ 131.039887] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)6638worker1 # [ 131.103443] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5)6639worker1 # [ 131.110531] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)6640worker1 # [ 131.258501] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 4/5)6641worker1 # [ 131.425275] nix-grpc-daemon[1456]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6642worker1 # [ 131.428333] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6643worker1 # [ 131.446101] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330486644client # [ 131.476197] nix[1394]: copying 1 paths to 'lb:50051'...6645worker1 # [ 131.458775] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330486646client # [ 131.478914] nix[1394]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6647worker1 # [ 131.471111] nix-daemon[850]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)6648worker1 # [ 131.490078] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=info event=niks3_push_started pid=15856649worker1 # [ 131.593578] nix-grpc-daemon[1585]: time=2026-09-23T05:13:44.637Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6650worker1 # [ 131.683094] nix-daemon[850]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)6651worker1 # [ 131.696156] nix-daemon[850]: reaped child process 1601, status = succeeded6652lb # [ 131.682296] niks3-server[1146]: 2026/09/23 05:13:43 INFO Received uploads request method=POST path=/api/pending_closures6653worker1 # [ 131.713221] nix-grpc-daemon[1585]: time=2026-09-23T05:13:44.757Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6654worker1 # [ 131.714779] nix-grpc-daemon[1585]: time=2026-09-23T05:13:44.759Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6655lb # [ 131.720422] niks3-server[1146]: 2026/09/23 05:13:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6656worker1 # [ 131.753781] nix-grpc-daemon[1585]: time=2026-09-23T05:13:44.797Z level=INFO msg="Uploading 1 narinfos"6657lb # [ 131.731253] niks3-server[1146]: 2026/09/23 05:13:43 INFO Signed narinfos id=51 count=16658lb # [ 131.736416] niks3-server[1146]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6659lb # [ 131.741787] niks3-server[1146]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6660lb # [ 131.753716] niks3-server[1146]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6661lb # [ 131.758722] niks3-server[1146]: 2026/09/23 05:13:44 INFO Completed upload id=516662worker1 # [ 131.782739] nix-grpc-daemon[1585]: time=2026-09-23T05:13:44.827Z level=INFO msg="Upload complete. (185ms)"6663lb # [ 131.762446] niks3-server[1146]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6664worker1 # [ 131.786267] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33048 duration_s=0 paths=1 nar_bytes_in=4246665worker1 # [ 131.792303] nix-grpc-daemon[1456]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330486666worker1 # [ 131.800483] nix-daemon[850]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)6667worker1 # [ 131.807144] nix-daemon[850]: reaped child process 1583, status = succeeded6668worker1 # [ 131.810868] nix-daemon[850]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)6669client # [ 131.829094] nix[1394]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6670lb: must succeed: curl -sf localhost:9901/clusters6671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6672lb: must succeed: curl -sf localhost:9901/clusters6673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6674worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6675worker1 # [ 133.302701] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1456 (nix-grpc-daemon) on client request.6676worker1 # [ 133.307335] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6677lb # [ 133.292422] niks3-server[1146]: 2026/09/23 05:13:45 INFO lead: released remote=192.168.1.3:509266678worker1 # [ 133.321598] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6679worker1 # [ 133.325382] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6680client # [ 133.341872] nix[1394]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6681worker1 # [ 133.326619] systemd[1]: nix-grpc-daemon.service: Consumed 374ms CPU time over 15.211s wall clock time, 35.6M memory peak, 15.4K incoming IP traffic, 27.4K outgoing IP traffic.6682worker2 # [ 133.328581] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:46Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6683client # [ 133.346542] nix[1394]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6684worker1 # [ 133.339030] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6685worker1 # [ 133.340590] systemd[1]: Starting Nix worker-protocol over gRPC...6686worker1 # [ 133.372622] nix-daemon[850]: reaped child process 1575, status = succeeded6687worker1 # [ 133.374458] nix-daemon[850]: reaped child process 1603, status = succeeded6688worker1 # [ 133.382220] nix-daemon[1605]: unexpected Nix daemon error: error: interrupted by the user6689worker1 # [ 133.392713] nix-daemon[850]: reaped child process 1605, status = failed with exit code 16690worker1 # [ 133.433672] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6691worker1 # [ 133.438085] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6692worker1 # [ 133.441452] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16693worker1 # [ 133.448601] nix-grpc-daemon[1642]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6694worker1 # [ 133.451577] nix-grpc-daemon[1642]: E0000 00:00:1790140426.485152 1642 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6695lb # [ 133.432163] niks3-server[1146]: 2026/09/23 05:13:45 INFO lead: acquired remote=192.168.1.3:506606696worker1 # [ 133.454751] nix-grpc-daemon[1642]: E0000 00:00:1790140426.489232 1642 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6697worker1 # [ 133.457576] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6698worker1 # [ 133.461036] systemd[1]: Started Nix worker-protocol over gRPC.6699worker1 # [ 133.461893] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=scheduler_take_over6700worker1 # [ 133.462990] nix-grpc-daemon[1642]: E0000 00:00:1790140426.503993 1684 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6701worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.20 seconds)6702client: must succeed: sleep 5.06703worker1 # [ 133.474548] nix-grpc-daemon[1642]: E0000 00:00:1790140426.518956 1684 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6704worker1 # [ 133.478150] nix-grpc-daemon[1642]: E0000 00:00:1790140426.521263 1684 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6705worker1 # [ 133.488730] nix-grpc-daemon[1642]: E0000 00:00:1790140426.533156 1684 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6706worker1 # [ 133.498392] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=scheduler_connected addr=lb:500516707worker1 # [ 133.504875] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:579226708worker1 # [ 133.506444] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06709worker2 # [ 133.529569] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:46Z level=info event=scheduler_connected addr=lb:500516710worker1 # [ 133.529068] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=oidc_jwks provider=mock keys=16711worker1 # [ 133.530708] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:579226712worker1 # [ 133.532355] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06713worker1 # [ 133.837714] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06714worker1 # [ 133.842812] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16715worker1 # [ 133.891147] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579266716worker1 # [ 133.901163] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6717worker1 # [ 133.915245] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6718client # [ 133.947139] nix[1394]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6719worker1 # [ 133.930099] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6720lb # [ 135.705418] envoy[910]: conn peer=[2001:db8:1::1]:41638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1523 ms=79336721lb # [ 135.709345] envoy[910]: conn peer=[2001:db8:1::4]:47952 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36793 tx=2105 ms=345826722lb # [ 135.713111] envoy[910]: conn peer=[2001:db8:1::3]:57988 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1010 tx=393 ms=150646723client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6724client: waiting for success: ! systemctl is-active mid6725worker1 # [ 138.755424] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:51Z level=info event=niks3_push_started pid=17266726worker1 # [ 138.784253] nix-daemon[850]: reaped child process 1701, status = succeeded6727worker1 # [ 138.862611] nix-grpc-daemon[1726]: time=2026-09-23T05:13:51.906Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6728worker1 # [ 138.952448] nix-daemon[850]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6729worker1 # [ 138.963804] nix-daemon[850]: reaped child process 1742, status = succeeded6730lb # [ 138.951972] niks3-server[1146]: 2026/09/23 05:13:51 INFO Received uploads request method=POST path=/api/pending_closures6731worker1 # [ 138.984675] nix-grpc-daemon[1726]: time=2026-09-23T05:13:52.029Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6732worker1 # [ 138.986317] nix-grpc-daemon[1726]: time=2026-09-23T05:13:52.030Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6733lb # [ 138.998567] niks3-server[1146]: 2026/09/23 05:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6734worker1 # [ 139.037212] nix-grpc-daemon[1726]: time=2026-09-23T05:13:52.081Z level=INFO msg="Uploading 1 narinfos"6735lb # [ 139.015216] niks3-server[1146]: 2026/09/23 05:13:51 INFO Signed narinfos id=52 count=16736lb # [ 139.025114] niks3-server[1146]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6737lb # [ 139.039056] niks3-server[1146]: 2026/09/23 05:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6738lb # [ 139.049253] niks3-server[1146]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6739lb # [ 139.053470] niks3-server[1146]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6740lb # [ 139.056406] niks3-server[1146]: 2026/09/23 05:13:51 INFO Completed upload id=526741worker1 # [ 139.078612] nix-grpc-daemon[1726]: time=2026-09-23T05:13:52.122Z level=INFO msg="Upload complete. (212ms)"6742lb # [ 139.058853] niks3-server[1146]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6743worker1 # [ 139.086156] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57926 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16744worker1 # [ 139.103163] nix-daemon[850]: reaped child process 1699, status = succeeded6745client # [ 139.143577] systemd[1]: mid.service: Deactivated successfully.6746client # [ 139.147311] systemd[1]: mid.service: Consumed 140ms CPU time over 8.322s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.6K outgoing IP traffic.6747client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)6748client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6749client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6750(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.04 seconds)6751subtest: interrupting the client stops the build on the worker6752client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6753client # [ 139.838762] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6754client # Running as unit: intr.service; invocation ID: 776fdbba88fb40d6b5f17bfe32dc99c76755client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6756client # [ 139.958990] nix[1430]: don't know how to build these paths:6757client # [ 139.960440] nix[1430]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6758worker1 # [ 139.939404] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579266759worker1 # [ 139.952694] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:52Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06760worker1 # [ 139.957078] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:52Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16761worker2 # [ 140.010839] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766762worker2 # [ 140.022494] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6763worker2 # [ 140.039231] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)6764worker2 # [ 140.095673] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5)6765worker2 # [ 140.297879] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 3/5)6766worker2 # [ 140.405764] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 4/5)6767worker2 # [ 140.430198] nix-grpc-daemon[1516]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6768worker2 # [ 140.433141] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6769worker2 # [ 140.450715] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602766770client # [ 140.475229] nix[1430]: copying 1 paths to 'lb:50051'...6771client # [ 140.479141] nix[1430]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6772worker2 # [ 140.465980] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602766773worker2 # [ 140.474547] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6774worker2 # [ 140.498430] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:53Z level=info event=niks3_push_started pid=16056775worker2 # [ 140.603692] nix-grpc-daemon[1605]: time=2026-09-23T05:13:53.793Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6776worker2 # [ 140.697395] nix-daemon[873]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6777worker2 # [ 140.710201] nix-daemon[873]: reaped child process 1621, status = succeeded6778lb # [ 140.691663] niks3-server[1146]: 2026/09/23 05:13:52 INFO Received uploads request method=POST path=/api/pending_closures6779worker2 # [ 140.729765] nix-grpc-daemon[1605]: time=2026-09-23T05:13:53.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6780worker2 # [ 140.731262] nix-grpc-daemon[1605]: time=2026-09-23T05:13:53.921Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6781lb # [ 140.737942] niks3-server[1146]: 2026/09/23 05:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6782worker2 # [ 140.770234] nix-grpc-daemon[1605]: time=2026-09-23T05:13:53.960Z level=INFO msg="Uploading 1 narinfos"6783lb # [ 140.743181] niks3-server[1146]: 2026/09/23 05:13:52 INFO Signed narinfos id=53 count=16784lb # [ 140.748667] niks3-server[1146]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6785lb # [ 140.755323] niks3-server[1146]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6786lb # [ 140.766268] niks3-server[1146]: 2026/09/23 05:13:53 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6787lb # [ 140.774668] niks3-server[1146]: 2026/09/23 05:13:53 INFO Completed upload id=536788worker2 # [ 140.802134] nix-grpc-daemon[1605]: time=2026-09-23T05:13:53.992Z level=INFO msg="Upload complete. (195ms)"6789lb # [ 140.777032] niks3-server[1146]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6790worker2 # [ 140.806276] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=4246791worker2 # [ 140.811989] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766792worker2 # [ 140.823147] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6793worker2 # [ 140.830131] nix-daemon[873]: reaped child process 1603, status = succeeded6794worker2 # [ 140.832896] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6795client # [ 140.846571] nix[1430]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6796client: must succeed: systemctl kill -s INT intr6797client # [ 142.169185] systemd[1]: intr.service: Sent signal SIGINT to main process 1430 (nix) on client request.6798worker2 # [ 142.162246] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:55Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6799client # [ 142.176270] nix[1430]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6800client # [ 142.182131] nix[1430]: error: interrupted by the user6801client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6802client # [ 142.209773] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6803client # [ 142.212667] systemd[1]: intr.service: Failed with result 'exit-code'.6804client # [ 142.216120] systemd[1]: intr.service: Consumed 116ms CPU time over 2.373s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6805worker2 # [ 142.261357] nix-daemon[873]: reaped child process 1623, status = succeeded6806worker2 # [ 142.262604] nix-daemon[1625]: unexpected Nix daemon error: error: interrupted by the user6807worker2 # [ 142.267419] nix-daemon[873]: reaped child process 1625, status = failed with exit code 16808(finished: subtest: interrupting the client stops the build on the worker, in 2.46 seconds)6809subtest: a deploy mid-build drains: the build finishes, the new generation takes over6810client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6811client # [ 142.324950] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6812client # Running as unit: sw.service; invocation ID: f524d68738104cf59e9090da6b151b1b6813client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6814client # [ 142.452838] nix[1451]: don't know how to build these paths:6815client # [ 142.453795] nix[1451]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6816worker1 # [ 142.433697] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579266817worker1 # [ 142.444480] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06818worker1 # [ 142.450166] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16819worker1 # [ 142.488973] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579266820worker1 # [ 142.500556] nix-grpc-daemon[1642]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5)6821worker1 # [ 142.518807] nix-grpc-daemon[1642]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)6822worker1 # [ 142.666640] nix-grpc-daemon[1642]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 3/5)6823worker1 # [ 142.844760] nix-grpc-daemon[1642]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 4/5)6824worker1 # [ 142.892245] nix-grpc-daemon[1642]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6825worker1 # [ 142.895748] nix-grpc-daemon[1642]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6826worker1 # [ 142.914519] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579266827client # [ 142.942960] nix[1451]: copying 1 paths to 'lb:50051'...6828worker1 # [ 142.926830] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579266829client # [ 142.946686] nix[1451]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6830worker1 # [ 142.937815] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6831worker1 # [ 143.043854] nix-daemon[850]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6832worker1 # [ 143.056508] nix-daemon[850]: reaped child process 1777, status = succeeded6833lb # [ 143.043253] niks3-server[1146]: 2026/09/23 05:13:55 INFO Received uploads request method=POST path=/api/pending_closures6834worker1 # [ 143.073841] nix-grpc-daemon[1726]: time=2026-09-23T05:13:56.118Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6835worker1 # [ 143.076698] nix-grpc-daemon[1726]: time=2026-09-23T05:13:56.120Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6836lb # [ 143.072648] niks3-server[1146]: 2026/09/23 05:13:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6837lb # [ 143.080452] niks3-server[1146]: 2026/09/23 05:13:55 INFO Signed narinfos id=54 count=16838worker1 # [ 143.107222] nix-grpc-daemon[1726]: time=2026-09-23T05:13:56.151Z level=INFO msg="Uploading 1 narinfos"6839lb # [ 143.090683] niks3-server[1146]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6840lb # [ 143.103092] niks3-server[1146]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6841lb # [ 143.108136] niks3-server[1146]: 2026/09/23 05:13:55 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6842lb # [ 143.116148] niks3-server[1146]: 2026/09/23 05:13:55 INFO Completed upload id=546843worker1 # [ 143.140566] nix-grpc-daemon[1726]: time=2026-09-23T05:13:56.184Z level=INFO msg="Upload complete. (188ms)"6844lb # [ 143.121540] niks3-server[1146]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6845worker1 # [ 143.144939] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57926 duration_s=0 paths=1 nar_bytes_in=4166846worker1 # [ 143.150901] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579266847worker1 # [ 143.160119] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6848worker1 # [ 143.170732] nix-daemon[850]: reaped child process 1772, status = succeeded6849worker1 # [ 143.174346] nix-daemon[850]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)6850client # [ 143.192102] nix[1451]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6851worker1: must succeed: systemctl reload nix-grpc-daemon.service6852worker1 # [ 144.739719] systemd[1]: Reloading Nix worker-protocol over gRPC...6853worker1 # [ 144.766535] systemd[1]: Reloaded Nix worker-protocol over gRPC.6854worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6855worker1: must succeed: pgrep -f 'read -t [9]0 x'6856worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6857client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26858worker2 # [ 144.954873] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602766859client # don't know how to build these paths:6860client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6861worker2 # [ 144.963862] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602766862worker1 # [ 144.980858] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:58Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06863worker1 # [ 144.984229] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:58Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16864worker2 # [ 145.035949] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766865worker2 # [ 145.048887] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602766866client # copying 1 paths to 'lb:50051'...6867worker2 # [ 145.062631] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602766868client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6869worker2 # [ 145.082888] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6870worker2 # [ 145.195592] nix-daemon[873]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6871worker2 # [ 145.209759] nix-daemon[873]: reaped child process 1675, status = succeeded6872lb # [ 145.190214] niks3-server[1146]: 2026/09/23 05:13:57 INFO Received uploads request method=POST path=/api/pending_closures6873worker2 # [ 145.224800] nix-grpc-daemon[1605]: time=2026-09-23T05:13:58.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6874worker2 # [ 145.226296] nix-grpc-daemon[1605]: time=2026-09-23T05:13:58.416Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6875lb # [ 145.221083] niks3-server[1146]: 2026/09/23 05:13:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6876worker2 # [ 145.255904] nix-grpc-daemon[1605]: time=2026-09-23T05:13:58.445Z level=INFO msg="Uploading 1 narinfos"6877lb # [ 145.228642] niks3-server[1146]: 2026/09/23 05:13:57 INFO Signed narinfos id=55 count=16878lb # [ 145.236067] niks3-server[1146]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6879lb # [ 145.246892] niks3-server[1146]: 2026/09/23 05:13:57 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6880worker2 # [ 145.281747] nix-grpc-daemon[1605]: time=2026-09-23T05:13:58.471Z level=INFO msg="Upload complete. (181ms)"6881lb # [ 145.255065] niks3-server[1146]: 2026/09/23 05:13:57 INFO Completed upload id=556882worker2 # [ 145.284348] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=5286883lb # [ 145.259145] niks3-server[1146]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6884worker2 # [ 145.290813] nix-grpc-daemon[1516]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766885lb # [ 145.267364] niks3-server[1146]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6886worker2 # [ 145.300170] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6887worker2 # [ 145.305104] nix-daemon[873]: reaped child process 1670, status = succeeded6888worker2 # [ 145.309880] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6889client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6890client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6891worker1 # [ 145.455429] nix-grpc-daemon[1642]: ts=2026-09-23T05:13:58Z level=info event=draining6892lb # [ 145.706636] envoy[910]: conn peer=[2001:db8:1::1]:54804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1915 ms=81656893lb # [ 145.710660] envoy[910]: conn peer=[2001:db8:1::1]:44830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=805 ms=22406894client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6895client # farm-a> LOG-farm-a-during-drain6896worker2 # [ 146.949728] nix-daemon[873]: reaped child process 1679, status = succeeded6897worker2 # [ 147.031248] nix-daemon[873]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6898worker2 # [ 147.044631] nix-daemon[873]: reaped child process 1698, status = succeeded6899lb # [ 147.025982] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received uploads request method=POST path=/api/pending_closures6900worker2 # [ 147.063373] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.253Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6901worker2 # [ 147.064953] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.255Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6902lb # [ 147.074897] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6903lb # [ 147.087117] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6904lb # [ 147.094059] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6905lb # [ 147.097496] niks3-server[1146]: 2026/09/23 05:13:59 INFO Signed narinfos id=56 count=16906worker2 # [ 147.128293] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.318Z level=INFO msg="Uploading 1 narinfos"6907lb # [ 147.104245] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6908lb # [ 147.112089] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6909lb # [ 147.115053] niks3-server[1146]: 2026/09/23 05:13:59 INFO Completed upload id=566910worker2 # [ 147.142459] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.332Z level=INFO msg="Upload complete. (224ms)"6911lb # [ 147.117407] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6912worker1 # [ 147.148468] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:00Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06913worker2 # [ 147.149848] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16914worker1 # [ 147.150334] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:00Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16915worker2 # [ 147.164322] nix-daemon[873]: reaped child process 1677, status = succeeded6916worker2 # [ 147.198692] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766917worker2 # [ 147.210712] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602766918worker2 # [ 147.224479] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.414Z level=INFO msg="All 1 paths already cached"6919client # copying 1 paths to 'lb:50051'...6920worker2 # [ 147.232858] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602766921client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6922worker2 # [ 147.252183] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6923worker2 # [ 147.355383] nix-daemon[873]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6924worker2 # [ 147.368459] nix-daemon[873]: reaped child process 1705, status = succeeded6925lb # [ 147.349707] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received uploads request method=POST path=/api/pending_closures6926worker2 # [ 147.385530] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.575Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6927worker2 # [ 147.387154] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.577Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6928lb # [ 147.386199] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6929lb # [ 147.393455] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6930lb # [ 147.398904] niks3-server[1146]: 2026/09/23 05:13:59 INFO Signed narinfos id=57 count=16931worker2 # [ 147.428940] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.618Z level=INFO msg="Uploading 1 narinfos"6932lb # [ 147.405500] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6933lb # [ 147.415651] niks3-server[1146]: 2026/09/23 05:13:59 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6934lb # [ 147.422481] niks3-server[1146]: 2026/09/23 05:13:59 INFO Completed upload id=576935lb # [ 147.424379] niks3-server[1146]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6936worker2 # [ 147.450225] nix-grpc-daemon[1605]: time=2026-09-23T05:14:00.639Z level=INFO msg="Upload complete. (184ms)"6937worker2 # [ 147.455170] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=6406938worker2 # [ 147.461797] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766939worker2 # [ 147.473234] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6940worker2 # [ 147.481182] nix-daemon[873]: reaped child process 1700, status = succeeded6941worker2 # [ 147.485296] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6942client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6943client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6944client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6945client # farm-b> LOG-farm-b-during-drain6946worker2 # [ 148.300585] nix-daemon[873]: reaped child process 1709, status = succeeded6947worker2 # [ 148.376558] nix-daemon[873]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6948worker2 # [ 148.389729] nix-daemon[873]: reaped child process 1728, status = succeeded6949lb # [ 148.371132] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received uploads request method=POST path=/api/pending_closures6950worker2 # [ 148.410419] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6951worker2 # [ 148.412030] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.601Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6952lb # [ 148.406151] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6953lb # [ 148.409051] niks3-server[1146]: 2026/09/23 05:14:00 INFO Signed narinfos id=58 count=16954worker2 # [ 148.436357] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.626Z level=INFO msg="Uploading 1 narinfos"6955lb # [ 148.413045] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6956lb # [ 148.415687] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6957lb # [ 148.420810] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6958lb # [ 148.426333] niks3-server[1146]: 2026/09/23 05:14:00 INFO Completed upload id=586959worker2 # [ 148.453560] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.643Z level=INFO msg="Upload complete. (162ms)"6960lb # [ 148.428190] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6961worker2 # [ 148.456430] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16962lb # [ 148.431653] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6963worker2 # [ 148.463561] nix-daemon[873]: reaped child process 1707, status = succeeded6964worker1 # [ 148.460192] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:01Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06965worker1 # [ 148.463237] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:01Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16966worker2 # [ 148.512143] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766967worker2 # [ 148.525153] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602766968worker2 # [ 148.538249] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.728Z level=INFO msg="All 2 paths already cached"6969client # copying 1 paths to 'lb:50051'...6970worker2 # [ 148.547538] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602766971client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6972worker2 # [ 148.567130] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6973worker2 # [ 148.664737] nix-daemon[873]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6974worker2 # [ 148.677794] nix-daemon[873]: reaped child process 1735, status = succeeded6975lb # [ 148.658323] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received uploads request method=POST path=/api/pending_closures6976worker2 # [ 148.693327] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.883Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6977worker2 # [ 148.694821] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.884Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6978lb # [ 148.689386] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6979lb # [ 148.697561] niks3-server[1146]: 2026/09/23 05:14:00 INFO Signed narinfos id=59 count=16980worker2 # [ 148.727685] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.917Z level=INFO msg="Uploading 1 narinfos"6981lb # [ 148.704109] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6982lb # [ 148.708741] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6983lb # [ 148.721158] niks3-server[1146]: 2026/09/23 05:14:00 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6984lb # [ 148.725476] niks3-server[1146]: 2026/09/23 05:14:00 INFO Completed upload id=596985worker2 # [ 148.755545] nix-grpc-daemon[1605]: time=2026-09-23T05:14:01.945Z level=INFO msg="Upload complete. (176ms)"6986lb # [ 148.732374] niks3-server[1146]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6987worker2 # [ 148.759714] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=7686988worker2 # [ 148.766373] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602766989worker2 # [ 148.778545] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6990worker2 # [ 148.781052] nix-daemon[873]: reaped child process 1730, status = succeeded6991worker2 # [ 148.786557] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6992client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6993client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6994client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6995client # farm-top> LOG-farm-top-during-drain6996worker2 # [ 150.001951] nix-daemon[873]: reaped child process 1739, status = succeeded6997worker2 # [ 150.077933] nix-daemon[873]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6998worker2 # [ 150.091221] nix-daemon[873]: reaped child process 1758, status = succeeded6999lb # [ 150.071743] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received uploads request method=POST path=/api/pending_closures7000worker2 # [ 150.107385] nix-grpc-daemon[1605]: time=2026-09-23T05:14:03.297Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7001worker2 # [ 150.110234] nix-grpc-daemon[1605]: time=2026-09-23T05:14:03.298Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"7002lb # [ 150.114435] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7003lb # [ 150.119652] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst7004lb # [ 150.126291] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7005lb # [ 150.131111] niks3-server[1146]: 2026/09/23 05:14:02 INFO Signed narinfos id=60 count=17006worker2 # [ 150.161387] nix-grpc-daemon[1605]: time=2026-09-23T05:14:03.351Z level=INFO msg="Uploading 1 narinfos"7007lb # [ 150.134913] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls7008lb # [ 150.145333] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7009lb # [ 150.149046] niks3-server[1146]: 2026/09/23 05:14:02 INFO Completed upload id=607010worker2 # [ 150.176416] nix-grpc-daemon[1605]: time=2026-09-23T05:14:03.366Z level=INFO msg="Upload complete. (182ms)"7011lb # [ 150.151299] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo7012worker2 # [ 150.182706] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=17013worker2 # [ 150.195714] nix-daemon[873]: reaped child process 1737, status = succeeded7014client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.41 seconds)7015client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath7016client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)7017worker1: must succeed: pkill -f 'read -t [9]0 x'7018worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)7019client: waiting for success: ! systemctl is-active sw7020worker1 # [ 150.394643] nix-daemon[850]: reaped child process 1781, status = succeeded7021worker1 # [ 150.470329] nix-daemon[850]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)7022worker1 # [ 150.482774] nix-daemon[850]: reaped child process 1831, status = succeeded7023lb # [ 150.469130] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received uploads request method=POST path=/api/pending_closures7024worker1 # [ 150.499174] nix-grpc-daemon[1726]: time=2026-09-23T05:14:03.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7025worker1 # [ 150.500782] nix-grpc-daemon[1726]: time=2026-09-23T05:14:03.545Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"7026lb # [ 150.506662] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7027lb # [ 150.511248] niks3-server[1146]: 2026/09/23 05:14:02 INFO Signed narinfos id=61 count=17028worker1 # [ 150.536793] nix-grpc-daemon[1726]: time=2026-09-23T05:14:03.581Z level=INFO msg="Uploading 1 narinfos"7029lb # [ 150.523062] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv7030lb # [ 150.528652] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls7031lb # [ 150.538119] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst7032lb # [ 150.545113] niks3-server[1146]: 2026/09/23 05:14:02 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7033lb # [ 150.550375] niks3-server[1146]: 2026/09/23 05:14:02 INFO Completed upload id=617034worker1 # [ 150.574321] nix-grpc-daemon[1726]: time=2026-09-23T05:14:03.618Z level=INFO msg="Upload complete. (189ms)"7035lb # [ 150.555728] niks3-server[1146]: 2026/09/23 05:14:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo7036worker1 # [ 150.582946] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57926 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=17037worker1 # [ 150.596342] nix-daemon[850]: reaped child process 1779, status = succeeded7038client # [ 150.631533] systemd[1]: sw.service: Deactivated successfully.7039client # [ 150.635545] systemd[1]: sw.service: Consumed 129ms CPU time over 8.309s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.7040client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)7041client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }7042client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)7043worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service7044worker1 # [ 151.459536] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:04Z level=info event=signal_exit7045lb # [ 152.440667] niks3-server[1146]: 2026/09/23 05:14:04 INFO lead: released remote=192.168.1.3:506607046worker2 # [ 153.468762] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:06Z level=info event=scheduler_restarting addr=lb:500517047worker1 # [ 153.461821] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=scheduler_restarting7048worker1 # [ 153.471311] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:579227049worker1 # [ 153.475159] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=worker_gone addr=192.168.1.4:500517050worker1 # [ 153.478921] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=scheduler_restarting addr=lb:500517051worker1 # [ 153.482178] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:579227052worker1 # [ 153.485610] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=worker_gone addr=192.168.1.3:500517053worker1 # [ 153.501709] nix-daemon[850]: reaped child process 1697, status = succeeded7054worker2 # [ 153.669847] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:06Z level=info event=scheduler_connected addr=lb:500517055worker2 # [ 153.673837] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7056worker1 # [ 153.679748] nix-grpc-daemon[1642]: ts=2026-09-23T05:14:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7057worker2 # [ 153.875057] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_connected addr=lb:500517058worker2 # [ 153.878932] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7059worker2 # [ 154.079811] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_connected addr=lb:500517060worker2 # [ 154.083690] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7061worker1 # [ 154.145660] nix-grpc-daemon[1642]: E0000 00:00:1790140447.189963 1688 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true7062worker1 # [ 154.150910] nix-grpc-daemon[1642]: E0000 00:00:1790140447.195300 1688 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7063worker2 # [ 154.284821] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_connected addr=lb:500517064worker2 # [ 154.288720] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7065worker2 # [ 154.490514] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_connected addr=lb:500517066worker2 # [ 154.494288] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7067worker2 # [ 154.695518] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_connected addr=lb:500517068worker2 # [ 154.699460] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7069worker2 # [ 154.900632] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517070worker2 # [ 154.904628] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7071worker2 # [ 155.105661] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517072worker2 # [ 155.109661] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7073worker2 # [ 155.310725] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517074worker2 # [ 155.314650] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7075worker1 # [ 155.488335] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7076worker1 # [ 155.492376] systemd[1]: nix-grpc-daemon.service: Consumed 583ms CPU time over 22.170s wall clock time, 36M memory peak, 25K incoming IP traffic, 32.6K outgoing IP traffic.7077worker1 # [ 155.508378] systemd[1]: Starting Nix worker-protocol over gRPC...7078worker2 # [ 155.516566] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517079worker2 # [ 155.520362] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7080worker2 # [ 155.553363] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_take_over7081lb # [ 155.526634] niks3-server[1146]: 2026/09/23 05:14:07 INFO lead: acquired remote=192.168.1.4:490767082worker1 # [ 155.601203] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7083worker1 # [ 155.603194] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7084worker1 # [ 155.606458] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17085worker1 # [ 155.610753] nix-grpc-daemon[1845]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7086worker1 # [ 155.612131] nix-grpc-daemon[1845]: E0000 00:00:1790140448.655018 1845 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7087worker1 # [ 155.617609] nix-grpc-daemon[1845]: E0000 00:00:1790140448.660329 1845 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7088worker1 # [ 155.623163] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7089worker1 # [ 155.626064] systemd[1]: Started Nix worker-protocol over gRPC.7090worker1 # [ 155.631096] nix-grpc-daemon[1845]: E0000 00:00:1790140448.672001 1887 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7091worker1 # [ 155.642117] nix-grpc-daemon[1845]: E0000 00:00:1790140448.685828 1887 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7092worker1 # [ 155.644249] nix-grpc-daemon[1845]: E0000 00:00:1790140448.685956 1887 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7093worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.19 seconds)7094lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7095lb: must succeed: curl -sf localhost:9901/clusters7096worker1 # [ 155.656388] nix-grpc-daemon[1845]: E0000 00:00:1790140448.700803 1887 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7097worker1 # [ 155.668088] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517098worker1 # [ 155.669729] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7099lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7100worker2 # [ 155.721400] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517101worker2 # [ 155.725261] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7102lb # [ 155.706459] envoy[910]: conn peer=[2001:db8:1::1]:44622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4350 ms=52457103lb # [ 155.710174] envoy[910]: conn peer=[2001:db8:1::1]:44846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1539 ms=81647104lb # [ 155.713705] envoy[910]: conn peer=[2001:db8:1::3]:50910 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=975 tx=361 ms=201767105worker1 # [ 155.870776] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=scheduler_connected addr=lb:500517106worker1 # [ 155.874686] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7107worker2 # [ 155.926264] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517108worker2 # [ 155.930269] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7109worker1 # [ 156.075496] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517110worker1 # [ 156.079532] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7111worker2 # [ 156.131482] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517112worker2 # [ 156.135349] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7113worker1 # [ 156.280613] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517114worker1 # [ 156.284595] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7115worker2 # [ 156.336438] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517116worker2 # [ 156.340300] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7117worker1 # [ 156.486483] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517118worker2 # [ 156.498951] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:484067119worker2 # [ 156.503735] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07120worker2 # [ 156.543410] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=scheduler_connected addr=lb:500517121worker2 # [ 156.550134] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=oidc_jwks provider=mock keys=17122worker2 # [ 156.553963] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:484067123worker2 # [ 156.558563] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07124lb: must succeed: curl -sf localhost:9901/clusters7125lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7126lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.07 seconds)7127lb: must succeed: curl -sf localhost:9901/clusters7128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7129worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7130worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7131(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.53 seconds)7132subtest: queue gauges show a burst while every build blocks7133lb: must succeed: curl -sf localhost:9901/clusters7134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7135lb: must succeed: curl -sf localhost:9901/clusters7136lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7137worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7138worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7139client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'7140client # [ 156.938577] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".7141client # Running as unit: burst.service; invocation ID: 2d91f39f784b4c699d312835155856ed7142client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)7143lb: must succeed: curl -sf localhost:9901/clusters7144lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7145lb: must succeed: curl -sf localhost:9901/clusters7146lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7147worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7148worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7149worker1 # [ 157.061646] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:369487150worker1 # [ 157.072960] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7151client # [ 157.103334] nix[1507]: don't know how to build these paths:7152client # [ 157.104213] nix[1507]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7153client # [ 157.105176] nix[1507]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7154client # [ 157.106093] nix[1507]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7155client # [ 157.107722] nix[1507]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7156worker2 # [ 157.101995] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=07157worker2 # [ 157.106643] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.3:50051 clients=17158worker2 # [ 157.110982] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=07159worker2 # [ 157.114948] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.4:50051 clients=17160worker2 # [ 157.119394] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07161worker2 # [ 157.123424] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07162worker2 # [ 157.145800] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767163worker1 # [ 157.140813] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487164worker2 # [ 157.157114] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767165worker1 # [ 157.151813] nix-grpc-daemon[1845]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)7166client # [ 157.180205] nix[1507]: copying 1 paths to 'lb:50051'...7167worker2 # [ 157.168209] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767168client # [ 157.183145] nix[1507]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7169worker2 # [ 157.178815] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7170worker1 # [ 157.215255] nix-grpc-daemon[1845]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)7171worker1 # [ 157.232710] nix-grpc-daemon[1845]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 265 ms (attempt 3/5)7172worker2 # [ 157.278117] nix-daemon[873]: accepted connection from pid 1773, user nix-grpc-daemon (trusted)7173worker2 # [ 157.290499] nix-daemon[873]: reaped child process 1776, status = succeeded7174lb # [ 157.272777] niks3-server[1146]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures7175worker2 # [ 157.309690] nix-grpc-daemon[1605]: time=2026-09-23T05:14:10.499Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7176worker2 # [ 157.311647] nix-grpc-daemon[1605]: time=2026-09-23T05:14:10.501Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7177lb # [ 157.313914] niks3-server[1146]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7178lb # [ 157.320247] niks3-server[1146]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7179lb # [ 157.341382] niks3-server[1146]: 2026/09/23 05:14:09 INFO Signed narinfos id=62 count=17180worker2 # [ 157.368470] nix-grpc-daemon[1605]: time=2026-09-23T05:14:10.558Z level=INFO msg="Uploading 1 narinfos"7181lb # [ 157.344155] niks3-server[1146]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7182lb # [ 157.349349] niks3-server[1146]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7183lb # [ 157.354080] niks3-server[1146]: 2026/09/23 05:14:09 INFO Completed upload id=627184worker2 # [ 157.381224] nix-grpc-daemon[1605]: time=2026-09-23T05:14:10.571Z level=INFO msg="Upload complete. (189ms)"7185lb # [ 157.359405] niks3-server[1146]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7186worker2 # [ 157.386080] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=4167187worker2 # [ 157.392278] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767188worker2 # [ 157.404108] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7189worker2 # [ 157.412287] nix-daemon[873]: reaped child process 1771, status = succeeded7190worker2 # [ 157.420210] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7191client # [ 157.433079] nix[1507]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7192worker1 # [ 157.505184] nix-grpc-daemon[1845]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 571 ms (attempt 4/5)7193lb: must succeed: curl -sf localhost:9901/clusters7194worker1 # [ 158.079848] nix-grpc-daemon[1845]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7196lb: must succeed: curl -sf localhost:9901/clusters7197worker1 # [ 158.083532] nix-grpc-daemon[1845]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7198worker1 # [ 158.097515] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487199client # [ 158.126904] nix[1507]: copying 1 paths to 'lb:50051'...7200lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7201worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7202worker1 # [ 158.108251] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487203client # [ 158.128896] nix[1507]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7204worker1 # [ 158.121181] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7205worker1 # [ 158.141993] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:11Z level=info event=niks3_push_started pid=19067206worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7207worker1 # [ 158.259531] nix-grpc-daemon[1906]: time=2026-09-23T05:14:11.303Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7208worker1 # [ 158.349464] nix-daemon[850]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)7209worker1 # [ 158.363219] nix-daemon[850]: reaped child process 1926, status = succeeded7210lb # [ 158.348240] niks3-server[1146]: 2026/09/23 05:14:10 INFO Received uploads request method=POST path=/api/pending_closures7211worker1 # [ 158.381708] nix-grpc-daemon[1906]: time=2026-09-23T05:14:11.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7212worker1 # [ 158.383249] nix-grpc-daemon[1906]: time=2026-09-23T05:14:11.427Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7213lb # [ 158.388401] niks3-server[1146]: 2026/09/23 05:14:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7214worker1 # [ 158.416484] nix-grpc-daemon[1906]: time=2026-09-23T05:14:11.460Z level=INFO msg="Uploading 1 narinfos"7215lb # [ 158.394458] niks3-server[1146]: 2026/09/23 05:14:10 INFO Signed narinfos id=63 count=17216lb # [ 158.399383] niks3-server[1146]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7217lb # [ 158.412690] niks3-server[1146]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7218lb # [ 158.414356] niks3-server[1146]: 2026/09/23 05:14:10 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7219lb # [ 158.422143] niks3-server[1146]: 2026/09/23 05:14:10 INFO Completed upload id=637220worker1 # [ 158.444316] nix-grpc-daemon[1906]: time=2026-09-23T05:14:11.488Z level=INFO msg="Upload complete. (181ms)"7221lb # [ 158.424061] niks3-server[1146]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7222worker1 # [ 158.447420] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=4167223worker1 # [ 158.452205] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487224worker1 # [ 158.460250] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7225worker1 # [ 158.465281] nix-daemon[850]: reaped child process 1904, status = succeeded7226worker1 # [ 158.470191] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7227client # [ 158.488411] nix[1507]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7228worker2 # [ 159.435686] nix-daemon[873]: reaped child process 1780, status = succeeded7229worker2 # [ 159.513903] nix-daemon[873]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)7230worker2 # [ 159.525663] nix-daemon[873]: reaped child process 1811, status = succeeded7231lb # [ 159.506504] niks3-server[1146]: 2026/09/23 05:14:11 INFO Received uploads request method=POST path=/api/pending_closures7232worker2 # [ 159.542459] nix-grpc-daemon[1605]: time=2026-09-23T05:14:12.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7233worker2 # [ 159.544070] nix-grpc-daemon[1605]: time=2026-09-23T05:14:12.733Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7234lb # [ 159.551179] niks3-server[1146]: 2026/09/23 05:14:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7235lb # [ 159.558071] niks3-server[1146]: 2026/09/23 05:14:11 INFO Signed narinfos id=64 count=17236worker2 # [ 159.589403] nix-grpc-daemon[1605]: time=2026-09-23T05:14:12.779Z level=INFO msg="Uploading 1 narinfos"7237lb # [ 159.566983] niks3-server[1146]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7238lb # [ 159.571512] niks3-server[1146]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7239lb # [ 159.575440] niks3-server[1146]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7240lb # [ 159.585062] niks3-server[1146]: 2026/09/23 05:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7241lb # [ 159.588740] niks3-server[1146]: 2026/09/23 05:14:11 INFO Completed upload id=647242worker2 # [ 159.615932] nix-grpc-daemon[1605]: time=2026-09-23T05:14:12.805Z level=INFO msg="Upload complete. (188ms)"7243lb # [ 159.591161] niks3-server[1146]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7244worker2 # [ 159.622184] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17245worker2 # [ 159.632164] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:12Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=17246worker2 # [ 159.642489] nix-daemon[873]: reaped child process 1778, status = succeeded7247worker2 # [ 159.675802] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767248worker2 # [ 159.687928] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767249client # [ 159.712937] nix[1507]: copying 1 paths to 'lb:50051'...7250worker2 # [ 159.699722] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767251client # [ 159.715263] nix[1507]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7252worker2 # [ 159.712159] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7253worker2 # [ 159.810114] nix-daemon[873]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)7254worker2 # [ 159.823173] nix-daemon[873]: reaped child process 1818, status = succeeded7255lb # [ 159.803383] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received uploads request method=POST path=/api/pending_closures7256worker2 # [ 159.839054] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.029Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7257worker2 # [ 159.840583] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.030Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7258lb # [ 159.835524] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7259worker2 # [ 159.867339] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.057Z level=INFO msg="Uploading 1 narinfos"7260lb # [ 159.840202] niks3-server[1146]: 2026/09/23 05:14:12 INFO Signed narinfos id=65 count=17261lb # [ 159.849438] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7262lb # [ 159.857096] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7263lb # [ 159.867978] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7264lb # [ 159.875200] niks3-server[1146]: 2026/09/23 05:14:12 INFO Completed upload id=657265worker2 # [ 159.902395] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.092Z level=INFO msg="Upload complete. (178ms)"7266lb # [ 159.880553] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo7267worker2 # [ 159.906653] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=4167268worker2 # [ 159.912564] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767269worker2 # [ 159.923493] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7270worker2 # [ 159.933300] nix-daemon[873]: reaped child process 1813, status = succeeded7271worker2 # [ 159.938409] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7272client # [ 159.950709] nix[1507]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7273worker2 # [ 160.551844] nix-daemon[873]: reaped child process 1822, status = succeeded7274worker2 # [ 160.628086] nix-daemon[873]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)7275worker2 # [ 160.639931] nix-daemon[873]: reaped child process 1846, status = succeeded7276lb # [ 160.620553] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received uploads request method=POST path=/api/pending_closures7277worker2 # [ 160.656498] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.846Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7278worker2 # [ 160.659059] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.848Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7279lb # [ 160.665693] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7280worker2 # [ 160.698746] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.888Z level=INFO msg="Uploading 1 narinfos"7281lb # [ 160.671381] niks3-server[1146]: 2026/09/23 05:14:12 INFO Signed narinfos id=66 count=17282lb # [ 160.680396] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7283lb # [ 160.686373] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7284worker1 # [ 160.711165] nix-daemon[850]: reaped child process 1930, status = succeeded7285lb # [ 160.695432] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7286lb # [ 160.701203] niks3-server[1146]: 2026/09/23 05:14:12 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7287lb # [ 160.705985] niks3-server[1146]: 2026/09/23 05:14:12 INFO Completed upload id=667288worker2 # [ 160.733533] nix-grpc-daemon[1605]: time=2026-09-23T05:14:13.923Z level=INFO msg="Upload complete. (190ms)"7289lb # [ 160.708357] niks3-server[1146]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7290worker2 # [ 160.739742] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17291worker2 # [ 160.747356] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.4:50051 clients=17292worker2 # [ 160.758075] nix-daemon[873]: reaped child process 1820, status = succeeded7293worker2 # [ 160.783072] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767294worker2 # [ 160.795208] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767295worker1 # [ 160.794474] nix-daemon[850]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)7296client # [ 160.818811] nix[1507]: copying 1 paths to 'lb:50051'...7297worker2 # [ 160.806320] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767298client # [ 160.821530] nix[1507]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7299worker1 # [ 160.808484] nix-daemon[850]: reaped child process 1958, status = succeeded7300worker2 # [ 160.818169] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7301lb # [ 160.792802] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures7302worker1 # [ 160.823263] nix-grpc-daemon[1906]: time=2026-09-23T05:14:13.867Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7303worker1 # [ 160.824859] nix-grpc-daemon[1906]: time=2026-09-23T05:14:13.869Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7304lb # [ 160.830461] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7305lb # [ 160.832732] niks3-server[1146]: 2026/09/23 05:14:13 INFO Signed narinfos id=67 count=17306worker1 # [ 160.856365] nix-grpc-daemon[1906]: time=2026-09-23T05:14:13.900Z level=INFO msg="Uploading 1 narinfos"7307lb # [ 160.837360] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7308lb # [ 160.839380] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7309lb # [ 160.846156] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7310lb # [ 160.849192] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7311lb # [ 160.853850] niks3-server[1146]: 2026/09/23 05:14:13 INFO Completed upload id=677312worker1 # [ 160.877495] nix-grpc-daemon[1906]: time=2026-09-23T05:14:13.921Z level=INFO msg="Upload complete. (198ms)"7313lb # [ 160.855831] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7314worker1 # [ 160.881237] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17315worker1 # [ 160.888238] nix-daemon[850]: reaped child process 1928, status = succeeded7316worker2 # [ 160.913459] nix-daemon[873]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)7317worker2 # [ 160.926128] nix-daemon[873]: reaped child process 1853, status = succeeded7318lb # [ 160.907293] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures7319worker2 # [ 160.942218] nix-grpc-daemon[1605]: time=2026-09-23T05:14:14.132Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7320worker2 # [ 160.943844] nix-grpc-daemon[1605]: time=2026-09-23T05:14:14.133Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7321lb # [ 160.936607] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7322lb # [ 160.939239] niks3-server[1146]: 2026/09/23 05:14:13 INFO Signed narinfos id=68 count=17323worker2 # [ 160.966661] nix-grpc-daemon[1605]: time=2026-09-23T05:14:14.156Z level=INFO msg="Uploading 1 narinfos"7324lb # [ 160.943881] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7325lb # [ 160.946882] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7326lb # [ 160.949868] niks3-server[1146]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7327lb # [ 160.953685] niks3-server[1146]: 2026/09/23 05:14:13 INFO Completed upload id=687328worker2 # [ 160.980968] nix-grpc-daemon[1605]: time=2026-09-23T05:14:14.171Z level=INFO msg="Upload complete. (152ms)"7329lb # [ 160.955929] niks3-server[1146]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7330worker2 # [ 160.982955] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=4167331worker2 # [ 160.987672] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767332worker2 # [ 160.993308] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7333worker2 # [ 160.995532] nix-daemon[873]: reaped child process 1848, status = succeeded7334worker2 # [ 161.000602] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7335client # [ 161.013689] nix[1507]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7336worker2 # [ 162.815622] nix-daemon[873]: reaped child process 1857, status = succeeded7337worker2 # [ 162.893088] nix-daemon[873]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)7338worker2 # [ 162.904462] nix-daemon[873]: reaped child process 1885, status = succeeded7339lb # [ 162.886436] niks3-server[1146]: 2026/09/23 05:14:15 INFO Received uploads request method=POST path=/api/pending_closures7340worker2 # [ 162.922221] nix-grpc-daemon[1605]: time=2026-09-23T05:14:16.112Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7341worker2 # [ 162.923824] nix-grpc-daemon[1605]: time=2026-09-23T05:14:16.113Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7342lb # [ 162.926181] niks3-server[1146]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7343lb # [ 162.936182] niks3-server[1146]: 2026/09/23 05:14:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7344lb # [ 162.944477] niks3-server[1146]: 2026/09/23 05:14:15 INFO Signed narinfos id=69 count=17345worker2 # [ 162.974795] nix-grpc-daemon[1605]: time=2026-09-23T05:14:16.164Z level=INFO msg="Uploading 1 narinfos"7346lb # [ 162.949422] niks3-server[1146]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7347lb # [ 162.961611] niks3-server[1146]: 2026/09/23 05:14:15 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7348lb # [ 162.966459] niks3-server[1146]: 2026/09/23 05:14:15 INFO Completed upload id=697349lb # [ 162.968633] niks3-server[1146]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7350worker2 # [ 162.995073] nix-grpc-daemon[1605]: time=2026-09-23T05:14:16.185Z level=INFO msg="Upload complete. (187ms)"7351lb # [ 162.971040] niks3-server[1146]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7352worker2 # [ 163.002551] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17353worker2 # [ 163.015270] nix-daemon[873]: reaped child process 1855, status = succeeded7354client # [ 163.052177] systemd[1]: burst.service: Deactivated successfully.7355client # [ 163.056146] systemd[1]: burst.service: Consumed 166ms CPU time over 6.117s wall clock time, 55.9M memory peak, 14.3K incoming IP traffic, 19.2K outgoing IP traffic.7356client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7357client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.05 seconds)7358lb: must succeed: curl -sf localhost:9901/clusters7359lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7360lb: must succeed: curl -sf localhost:9901/clusters7361lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7362worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7363worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7364(finished: subtest: queue gauges show a burst while every build blocks, in 7.35 seconds)7365subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7366client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17367worker1 # [ 164.281241] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:369487368worker2 # [ 164.295479] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767369worker2 # [ 164.337173] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:17Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07370worker2 # [ 164.341307] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:17Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17371worker2 # [ 164.345538] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:17Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07372worker2 # [ 164.349750] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:17Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07373worker1 # [ 164.378073] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487374worker1 # [ 164.391600] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487375worker1 # [ 164.403087] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487376worker1 # [ 164.414506] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7377worker1 # [ 164.517222] nix-daemon[850]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)7378worker1 # [ 164.530164] nix-daemon[850]: reaped child process 1977, status = succeeded7379lb # [ 164.517611] niks3-server[1146]: 2026/09/23 05:14:16 INFO Received uploads request method=POST path=/api/pending_closures7380worker1 # [ 164.547401] nix-grpc-daemon[1906]: time=2026-09-23T05:14:17.591Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7381worker1 # [ 164.549459] nix-grpc-daemon[1906]: time=2026-09-23T05:14:17.593Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7382lb # [ 164.548929] niks3-server[1146]: 2026/09/23 05:14:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7383lb # [ 164.553431] niks3-server[1146]: 2026/09/23 05:14:16 INFO Signed narinfos id=70 count=17384worker1 # [ 164.578147] nix-grpc-daemon[1906]: time=2026-09-23T05:14:17.622Z level=INFO msg="Uploading 1 narinfos"7385lb # [ 164.560145] niks3-server[1146]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7386lb # [ 164.564604] niks3-server[1146]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7387lb # [ 164.574729] niks3-server[1146]: 2026/09/23 05:14:16 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7388lb # [ 164.578099] niks3-server[1146]: 2026/09/23 05:14:16 INFO Completed upload id=707389worker1 # [ 164.600271] nix-grpc-daemon[1906]: time=2026-09-23T05:14:17.644Z level=INFO msg="Upload complete. (174ms)"7390lb # [ 164.580087] niks3-server[1146]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7391worker1 # [ 164.604309] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=5047392worker1 # [ 164.609048] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487393worker1 # [ 164.618470] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7394worker1 # [ 164.623120] nix-daemon[850]: reaped child process 1972, status = succeeded7395worker1 # [ 164.628261] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7396worker1 # [ 165.264823] nix-daemon[850]: reaped child process 1981, status = succeeded7397worker1 # [ 165.341848] nix-daemon[850]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)7398worker1 # [ 165.354353] nix-daemon[850]: reaped child process 2000, status = succeeded7399lb # [ 165.339466] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received uploads request method=POST path=/api/pending_closures7400worker1 # [ 165.370455] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7401worker1 # [ 165.371971] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.414Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7402lb # [ 165.377164] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7403lb # [ 165.385142] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7404lb # [ 165.391147] niks3-server[1146]: 2026/09/23 05:14:17 INFO Signed narinfos id=71 count=17405worker1 # [ 165.416779] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.460Z level=INFO msg="Uploading 1 narinfos"7406lb # [ 165.399135] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7407lb # [ 165.406406] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7408lb # [ 165.414993] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7409lb # [ 165.418183] niks3-server[1146]: 2026/09/23 05:14:17 INFO Completed upload id=717410worker1 # [ 165.440387] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.484Z level=INFO msg="Upload complete. (205ms)"7411lb # [ 165.420874] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7412worker1 # [ 165.446476] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17413worker2 # [ 165.452931] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:18Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17414worker1 # [ 165.454975] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487415worker2 # [ 165.462715] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:18Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07416worker1 # [ 165.465828] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487417worker1 # [ 165.474849] nix-daemon[850]: reaped child process 1979, status = succeeded7418worker1 # [ 165.478413] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487419worker1 # [ 165.482753] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7420worker1 # [ 165.565762] nix-daemon[850]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)7421worker1 # [ 165.579241] nix-daemon[850]: reaped child process 2007, status = succeeded7422lb # [ 165.563373] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received uploads request method=POST path=/api/pending_closures7423worker1 # [ 165.594348] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.638Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7424worker1 # [ 165.596354] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.640Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7425lb # [ 165.592363] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7426lb # [ 165.603576] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7427lb # [ 165.609776] niks3-server[1146]: 2026/09/23 05:14:17 INFO Signed narinfos id=72 count=17428worker1 # [ 165.634700] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.678Z level=INFO msg="Uploading 1 narinfos"7429lb # [ 165.615713] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7430lb # [ 165.627355] niks3-server[1146]: 2026/09/23 05:14:17 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7431lb # [ 165.631455] niks3-server[1146]: 2026/09/23 05:14:17 INFO Completed upload id=727432worker1 # [ 165.653669] nix-grpc-daemon[1906]: time=2026-09-23T05:14:18.698Z level=INFO msg="Upload complete. (166ms)"7433lb # [ 165.633812] niks3-server[1146]: 2026/09/23 05:14:17 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7434worker1 # [ 165.656900] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=5047435worker1 # [ 165.661726] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487436worker1 # [ 165.670786] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7437worker1 # [ 165.679215] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7438worker1 # [ 165.681488] nix-daemon[850]: reaped child process 2002, status = succeeded7439lb # [ 165.707842] envoy[910]: conn peer=[2001:db8:1::1]:32808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9690 tx=5227 ms=59537440worker1 # [ 166.495346] nix-daemon[850]: reaped child process 2011, status = succeeded7441worker1 # [ 166.569872] nix-daemon[850]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)7442worker1 # [ 166.583908] nix-daemon[850]: reaped child process 2030, status = succeeded7443lb # [ 166.567085] niks3-server[1146]: 2026/09/23 05:14:18 INFO Received uploads request method=POST path=/api/pending_closures7444worker1 # [ 166.599506] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7445worker1 # [ 166.601500] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.645Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7446lb # [ 166.608174] niks3-server[1146]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7447lb # [ 166.614950] niks3-server[1146]: 2026/09/23 05:14:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7448worker1 # [ 166.647894] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.692Z level=INFO msg="Uploading 1 narinfos"7449lb # [ 166.625722] niks3-server[1146]: 2026/09/23 05:14:18 INFO Signed narinfos id=73 count=17450lb # [ 166.629840] niks3-server[1146]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7451lb # [ 166.634229] niks3-server[1146]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7452lb # [ 166.647082] niks3-server[1146]: 2026/09/23 05:14:18 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7453lb # [ 166.651206] niks3-server[1146]: 2026/09/23 05:14:18 INFO Completed upload id=737454worker1 # [ 166.673465] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.717Z level=INFO msg="Upload complete. (186ms)"7455lb # [ 166.652931] niks3-server[1146]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7456worker1 # [ 166.679800] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17457worker2 # [ 166.688124] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:19Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17458worker2 # [ 166.695886] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:19Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07459worker1 # [ 166.694867] nix-daemon[850]: reaped child process 2009, status = succeeded7460worker1 # [ 166.726816] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487461worker1 # [ 166.739266] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487462worker1 # [ 166.751721] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487463worker1 # [ 166.761173] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7464worker1 # [ 166.857961] nix-daemon[850]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7465worker1 # [ 166.871170] nix-daemon[850]: reaped child process 2037, status = succeeded7466lb # [ 166.856110] niks3-server[1146]: 2026/09/23 05:14:19 INFO Received uploads request method=POST path=/api/pending_closures7467worker1 # [ 166.886100] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7468worker1 # [ 166.887618] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.932Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7469lb # [ 166.888845] niks3-server[1146]: 2026/09/23 05:14:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7470worker1 # [ 166.920142] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.964Z level=INFO msg="Uploading 1 narinfos"7471lb # [ 166.897546] niks3-server[1146]: 2026/09/23 05:14:19 INFO Signed narinfos id=74 count=17472lb # [ 166.903346] niks3-server[1146]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7473lb # [ 166.910230] niks3-server[1146]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7474lb # [ 166.921558] niks3-server[1146]: 2026/09/23 05:14:19 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7475lb # [ 166.928198] niks3-server[1146]: 2026/09/23 05:14:19 INFO Completed upload id=747476worker1 # [ 166.950499] nix-grpc-daemon[1906]: time=2026-09-23T05:14:19.994Z level=INFO msg="Upload complete. (177ms)"7477lb # [ 166.932433] niks3-server[1146]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7478worker1 # [ 166.954773] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=5047479worker1 # [ 166.961263] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487480worker1 # [ 166.973112] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7481worker1 # [ 166.981298] nix-daemon[850]: reaped child process 2032, status = succeeded7482worker1 # [ 166.985776] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7483worker1 # [ 167.810114] nix-daemon[850]: reaped child process 2041, status = succeeded7484worker1 # [ 167.880080] nix-daemon[850]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)7485worker1 # [ 167.893380] nix-daemon[850]: reaped child process 2060, status = succeeded7486lb # [ 167.877744] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received uploads request method=POST path=/api/pending_closures7487worker1 # [ 167.909293] nix-grpc-daemon[1906]: time=2026-09-23T05:14:20.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7488worker1 # [ 167.910703] nix-grpc-daemon[1906]: time=2026-09-23T05:14:20.952Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7489lb # [ 167.918968] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7490lb # [ 167.925459] niks3-server[1146]: 2026/09/23 05:14:20 INFO Signed narinfos id=75 count=17491worker1 # [ 167.950620] nix-grpc-daemon[1906]: time=2026-09-23T05:14:20.994Z level=INFO msg="Uploading 1 narinfos"7492lb # [ 167.935872] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7493lb # [ 167.941804] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7494lb # [ 167.947179] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7495lb # [ 167.956744] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7496lb # [ 167.960223] niks3-server[1146]: 2026/09/23 05:14:20 INFO Completed upload id=757497worker1 # [ 167.982561] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.026Z level=INFO msg="Upload complete. (190ms)"7498lb # [ 167.962948] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7499worker1 # [ 167.988670] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17500worker2 # [ 167.995513] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:21Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17501worker1 # [ 167.997306] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487502worker2 # [ 168.004817] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:21Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07503worker1 # [ 168.009530] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487504worker1 # [ 168.017156] nix-daemon[850]: reaped child process 2039, status = succeeded7505worker1 # [ 168.018920] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.063Z level=INFO msg="All 1 paths already cached"7506worker1 # [ 168.022924] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487507worker1 # [ 168.027856] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7508worker1 # [ 168.109180] nix-daemon[850]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)7509worker1 # [ 168.121979] nix-daemon[850]: reaped child process 2067, status = succeeded7510lb # [ 168.106907] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received uploads request method=POST path=/api/pending_closures7511worker1 # [ 168.136803] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7512worker1 # [ 168.138440] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.182Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7513lb # [ 168.140238] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7514worker1 # [ 168.169483] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.213Z level=INFO msg="Uploading 1 narinfos"7515lb # [ 168.147299] niks3-server[1146]: 2026/09/23 05:14:20 INFO Signed narinfos id=76 count=17516lb # [ 168.154629] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7517lb # [ 168.158850] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7518lb # [ 168.172115] niks3-server[1146]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7519worker1 # [ 168.198880] nix-grpc-daemon[1906]: time=2026-09-23T05:14:21.243Z level=INFO msg="Upload complete. (166ms)"7520lb # [ 168.177274] niks3-server[1146]: 2026/09/23 05:14:20 INFO Completed upload id=767521worker1 # [ 168.201402] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=6247522lb # [ 168.183055] niks3-server[1146]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7523worker1 # [ 168.206604] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487524worker1 # [ 168.218138] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7525worker1 # [ 168.222352] nix-daemon[850]: reaped child process 2062, status = succeeded7526worker1 # [ 168.226981] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7527worker1 # [ 169.042172] nix-daemon[850]: reaped child process 2071, status = succeeded7528worker1 # [ 169.115298] nix-daemon[850]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)7529worker1 # [ 169.128495] nix-daemon[850]: reaped child process 2090, status = succeeded7530lb # [ 169.113886] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received uploads request method=POST path=/api/pending_closures7531worker1 # [ 169.144983] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.189Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7532worker1 # [ 169.148060] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.190Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7533lb # [ 169.160134] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7534lb # [ 169.166369] niks3-server[1146]: 2026/09/23 05:14:21 INFO Signed narinfos id=77 count=17535worker1 # [ 169.191724] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.236Z level=INFO msg="Uploading 1 narinfos"7536lb # [ 169.171759] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7537lb # [ 169.176351] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7538lb # [ 169.180908] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7539lb # [ 169.190780] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7540lb # [ 169.194533] niks3-server[1146]: 2026/09/23 05:14:21 INFO Completed upload id=777541worker1 # [ 169.216812] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.261Z level=INFO msg="Upload complete. (183ms)"7542lb # [ 169.196298] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7543worker1 # [ 169.222499] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17544worker2 # [ 169.229466] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:22Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17545worker1 # [ 169.230974] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487546worker2 # [ 169.239365] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:22Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07547worker1 # [ 169.241477] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487548worker1 # [ 169.249650] nix-daemon[850]: reaped child process 2069, status = succeeded7549worker1 # [ 169.251748] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.296Z level=INFO msg="All 1 paths already cached"7550worker1 # [ 169.255958] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487551worker1 # [ 169.260867] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7552worker1 # [ 169.345240] nix-daemon[850]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7553worker1 # [ 169.358319] nix-daemon[850]: reaped child process 2097, status = succeeded7554lb # [ 169.343061] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received uploads request method=POST path=/api/pending_closures7555worker1 # [ 169.373804] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.418Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7556worker1 # [ 169.375746] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.419Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7557lb # [ 169.373294] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7558lb # [ 169.383542] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7559worker1 # [ 169.411119] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.455Z level=INFO msg="Uploading 1 narinfos"7560lb # [ 169.389320] niks3-server[1146]: 2026/09/23 05:14:21 INFO Signed narinfos id=78 count=17561lb # [ 169.395889] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7562lb # [ 169.405115] niks3-server[1146]: 2026/09/23 05:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7563lb # [ 169.411206] niks3-server[1146]: 2026/09/23 05:14:21 INFO Completed upload id=787564worker1 # [ 169.433387] nix-grpc-daemon[1906]: time=2026-09-23T05:14:22.477Z level=INFO msg="Upload complete. (167ms)"7565lb # [ 169.415728] niks3-server[1146]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7566worker1 # [ 169.437333] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=6247567worker1 # [ 169.442686] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487568worker1 # [ 169.455256] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7569worker1 # [ 169.458329] nix-daemon[850]: reaped child process 2092, status = succeeded7570worker1 # [ 169.462377] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7571worker1 # [ 170.699879] nix-daemon[850]: reaped child process 2101, status = succeeded7572worker1 # [ 170.783094] nix-daemon[850]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)7573worker1 # [ 170.795374] nix-daemon[850]: reaped child process 2120, status = succeeded7574lb # [ 170.780446] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received uploads request method=POST path=/api/pending_closures7575worker1 # [ 170.811093] nix-grpc-daemon[1906]: time=2026-09-23T05:14:23.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7576worker1 # [ 170.813758] nix-grpc-daemon[1906]: time=2026-09-23T05:14:23.857Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7577lb # [ 170.818932] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7578lb # [ 170.826244] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7579worker1 # [ 170.856131] nix-grpc-daemon[1906]: time=2026-09-23T05:14:23.900Z level=INFO msg="Uploading 1 narinfos"7580lb # [ 170.833711] niks3-server[1146]: 2026/09/23 05:14:23 INFO Signed narinfos id=79 count=17581lb # [ 170.838804] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7582lb # [ 170.846444] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7583lb # [ 170.852150] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7584lb # [ 170.857462] niks3-server[1146]: 2026/09/23 05:14:23 INFO Completed upload id=797585worker1 # [ 170.879764] nix-grpc-daemon[1906]: time=2026-09-23T05:14:23.924Z level=INFO msg="Upload complete. (208ms)"7586lb # [ 170.859377] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7587worker1 # [ 170.885873] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17588worker2 # [ 170.894523] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:24Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17589worker2 # [ 170.902540] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:24Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07590worker1 # [ 170.900819] nix-daemon[850]: reaped child process 2099, status = succeeded7591worker1 # [ 170.933215] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487592worker1 # [ 170.945709] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487593worker1 # [ 170.959100] nix-grpc-daemon[1906]: time=2026-09-23T05:14:24.003Z level=INFO msg="All 1 paths already cached"7594worker1 # [ 170.966643] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487595worker1 # [ 170.974696] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7596worker1 # [ 171.069601] nix-daemon[850]: accepted connection from pid 2124, user nix-grpc-daemon (trusted)7597worker1 # [ 171.083251] nix-daemon[850]: reaped child process 2127, status = succeeded7598lb # [ 171.067348] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received uploads request method=POST path=/api/pending_closures7599worker1 # [ 171.097202] nix-grpc-daemon[1906]: time=2026-09-23T05:14:24.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7600worker1 # [ 171.098653] nix-grpc-daemon[1906]: time=2026-09-23T05:14:24.143Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7601lb # [ 171.097387] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7602lb # [ 171.106635] niks3-server[1146]: 2026/09/23 05:14:23 INFO Signed narinfos id=80 count=17603worker1 # [ 171.131763] nix-grpc-daemon[1906]: time=2026-09-23T05:14:24.176Z level=INFO msg="Uploading 1 narinfos"7604lb # [ 171.110450] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7605lb # [ 171.117813] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7606lb # [ 171.129781] niks3-server[1146]: 2026/09/23 05:14:23 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7607worker1 # [ 171.158148] nix-grpc-daemon[1906]: time=2026-09-23T05:14:24.202Z level=INFO msg="Upload complete. (171ms)"7608lb # [ 171.135980] niks3-server[1146]: 2026/09/23 05:14:23 INFO Completed upload id=807609worker1 # [ 171.160707] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=6247610lb # [ 171.143229] niks3-server[1146]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7611worker1 # [ 171.166059] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487612worker1 # [ 171.176113] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7613worker1 # [ 171.183179] nix-daemon[850]: reaped child process 2122, status = succeeded7614worker1 # [ 171.186274] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7615worker1 # [ 172.227243] nix-daemon[850]: reaped child process 2131, status = succeeded7616worker1 # [ 172.307729] nix-daemon[850]: accepted connection from pid 2147, user nix-grpc-daemon (trusted)7617worker1 # [ 172.319242] nix-daemon[850]: reaped child process 2150, status = succeeded7618lb # [ 172.306510] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received uploads request method=POST path=/api/pending_closures7619worker1 # [ 172.338374] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.382Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7620worker1 # [ 172.339978] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.384Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7621lb # [ 172.353267] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7622worker1 # [ 172.381572] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.425Z level=INFO msg="Uploading 1 narinfos"7623lb # [ 172.359157] niks3-server[1146]: 2026/09/23 05:14:24 INFO Signed narinfos id=81 count=17624lb # [ 172.368115] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7625lb # [ 172.376112] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7626lb # [ 172.380742] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7627lb # [ 172.390127] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7628lb # [ 172.394490] niks3-server[1146]: 2026/09/23 05:14:24 INFO Completed upload id=817629worker1 # [ 172.416902] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.461Z level=INFO msg="Upload complete. (222ms)"7630lb # [ 172.398672] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7631worker1 # [ 172.422350] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17632worker2 # [ 172.429709] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:25Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17633worker1 # [ 172.436787] nix-daemon[850]: reaped child process 2129, status = succeeded7634worker2 # [ 172.440310] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:25Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07635worker1 # [ 172.474806] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487636worker1 # [ 172.487953] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487637worker1 # [ 172.501622] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.545Z level=INFO msg="All 2 paths already cached"7638worker1 # [ 172.509774] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487639worker1 # [ 172.520183] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7640worker1 # [ 172.616389] nix-daemon[850]: accepted connection from pid 2154, user nix-grpc-daemon (trusted)7641worker1 # [ 172.631429] nix-daemon[850]: reaped child process 2157, status = succeeded7642lb # [ 172.614682] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received uploads request method=POST path=/api/pending_closures7643worker1 # [ 172.645693] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.690Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7644worker1 # [ 172.647243] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.691Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7645lb # [ 172.649454] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7646lb # [ 172.658560] niks3-server[1146]: 2026/09/23 05:14:24 INFO Signed narinfos id=82 count=17647worker1 # [ 172.683626] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.728Z level=INFO msg="Uploading 1 narinfos"7648lb # [ 172.662264] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7649lb # [ 172.667694] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7650lb # [ 172.680542] niks3-server[1146]: 2026/09/23 05:14:24 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7651lb # [ 172.686628] niks3-server[1146]: 2026/09/23 05:14:24 INFO Completed upload id=827652worker1 # [ 172.708749] nix-grpc-daemon[1906]: time=2026-09-23T05:14:25.753Z level=INFO msg="Upload complete. (175ms)"7653worker1 # [ 172.712901] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=7527654lb # [ 172.692823] niks3-server[1146]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7655worker1 # [ 172.719683] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487656worker1 # [ 172.729306] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7657worker1 # [ 172.737761] nix-daemon[850]: reaped child process 2152, status = succeeded7658worker1 # [ 172.742442] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7659worker1 # [ 173.979390] nix-daemon[850]: reaped child process 2161, status = succeeded7660worker1 # [ 174.062475] nix-daemon[850]: accepted connection from pid 2177, user nix-grpc-daemon (trusted)7661worker1 # [ 174.076043] nix-daemon[850]: reaped child process 2180, status = succeeded7662lb # [ 174.060786] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received uploads request method=POST path=/api/pending_closures7663worker1 # [ 174.091336] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.135Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7664worker1 # [ 174.092833] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.137Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7665lb # [ 174.100187] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7666worker1 # [ 174.131964] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.176Z level=INFO msg="Uploading 1 narinfos"7667lb # [ 174.110149] niks3-server[1146]: 2026/09/23 05:14:26 INFO Signed narinfos id=83 count=17668lb # [ 174.116565] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7669lb # [ 174.122922] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7670lb # [ 174.128552] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7671lb # [ 174.137524] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7672lb # [ 174.143153] niks3-server[1146]: 2026/09/23 05:14:26 INFO Completed upload id=837673worker1 # [ 174.165563] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.209Z level=INFO msg="Upload complete. (213ms)"7674lb # [ 174.145445] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7675worker1 # [ 174.171771] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17676worker2 # [ 174.178850] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:27Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17677worker1 # [ 174.187437] nix-daemon[850]: reaped child process 2159, status = succeeded7678worker1 # [ 174.216892] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487679worker1 # [ 174.229266] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487680worker1 # [ 174.242283] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.286Z level=INFO msg="All 2 paths already cached"7681worker1 # [ 174.251222] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487682worker1 # [ 174.258744] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7683worker1 # [ 174.355123] nix-daemon[850]: accepted connection from pid 2184, user nix-grpc-daemon (trusted)7684worker1 # [ 174.367894] nix-daemon[850]: reaped child process 2187, status = succeeded7685lb # [ 174.353270] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received uploads request method=POST path=/api/pending_closures7686worker1 # [ 174.383511] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.427Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7687worker1 # [ 174.385041] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.429Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7688lb # [ 174.387156] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7689lb # [ 174.394724] niks3-server[1146]: 2026/09/23 05:14:26 INFO Signed narinfos id=84 count=17690worker1 # [ 174.420796] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.465Z level=INFO msg="Uploading 1 narinfos"7691lb # [ 174.401140] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7692lb # [ 174.406454] niks3-server[1146]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7693lb # [ 174.418800] niks3-server[1146]: 2026/09/23 05:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7694lb # [ 174.422635] niks3-server[1146]: 2026/09/23 05:14:26 INFO Completed upload id=847695worker1 # [ 174.446946] nix-grpc-daemon[1906]: time=2026-09-23T05:14:27.491Z level=INFO msg="Upload complete. (175ms)"7696worker1 # [ 174.451354] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=7527697worker1 # [ 174.456508] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487698worker1 # [ 174.468102] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7699worker1 # [ 174.476534] nix-daemon[850]: reaped child process 2182, status = succeeded7700worker1 # [ 174.479943] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7701worker1 # [ 175.517995] nix-daemon[850]: reaped child process 2191, status = succeeded7702worker1 # [ 175.597363] nix-daemon[850]: accepted connection from pid 2207, user nix-grpc-daemon (trusted)7703worker1 # [ 175.609210] nix-daemon[850]: reaped child process 2210, status = succeeded7704lb # [ 175.594944] niks3-server[1146]: 2026/09/23 05:14:27 INFO Received uploads request method=POST path=/api/pending_closures7705worker1 # [ 175.626432] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7706worker1 # [ 175.628398] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.672Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7707lb # [ 175.635882] niks3-server[1146]: 2026/09/23 05:14:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7708worker1 # [ 175.663041] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.707Z level=INFO msg="Uploading 1 narinfos"7709lb # [ 175.640619] niks3-server[1146]: 2026/09/23 05:14:27 INFO Signed narinfos id=85 count=17710lb # [ 175.648737] niks3-server[1146]: 2026/09/23 05:14:27 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7711lb # [ 175.653185] niks3-server[1146]: 2026/09/23 05:14:27 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7712lb # [ 175.660333] niks3-server[1146]: 2026/09/23 05:14:27 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7713lb # [ 175.664859] niks3-server[1146]: 2026/09/23 05:14:27 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7714lb # [ 175.667629] niks3-server[1146]: 2026/09/23 05:14:27 INFO Completed upload id=857715worker1 # [ 175.689912] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.734Z level=INFO msg="Upload complete. (202ms)"7716lb # [ 175.670787] niks3-server[1146]: 2026/09/23 05:14:27 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7717worker1 # [ 175.694516] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17718worker2 # [ 175.702502] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:28Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17719worker1 # [ 175.708296] nix-daemon[850]: reaped child process 2189, status = succeeded7720worker1 # [ 175.740706] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487721worker1 # [ 175.752709] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487722worker1 # [ 175.764376] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.808Z level=INFO msg="All 2 paths already cached"7723worker1 # [ 175.772551] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:369487724worker1 # [ 175.783159] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7725worker1 # [ 175.876848] nix-daemon[850]: accepted connection from pid 2214, user nix-grpc-daemon (trusted)7726worker1 # [ 175.890107] nix-daemon[850]: reaped child process 2217, status = succeeded7727lb # [ 175.874697] niks3-server[1146]: 2026/09/23 05:14:28 INFO Received uploads request method=POST path=/api/pending_closures7728worker1 # [ 175.905280] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7729worker1 # [ 175.906843] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.951Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7730lb # [ 175.902601] niks3-server[1146]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7731lb # [ 175.913150] niks3-server[1146]: 2026/09/23 05:14:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7732lb # [ 175.918787] niks3-server[1146]: 2026/09/23 05:14:28 INFO Signed narinfos id=86 count=17733worker1 # [ 175.943588] nix-grpc-daemon[1906]: time=2026-09-23T05:14:28.988Z level=INFO msg="Uploading 1 narinfos"7734lb # [ 175.930570] niks3-server[1146]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7735lb # [ 175.937227] niks3-server[1146]: 2026/09/23 05:14:28 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7736lb # [ 175.941258] niks3-server[1146]: 2026/09/23 05:14:28 INFO Completed upload id=867737worker1 # [ 175.964543] nix-grpc-daemon[1906]: time=2026-09-23T05:14:29.009Z level=INFO msg="Upload complete. (171ms)"7738lb # [ 175.944393] niks3-server[1146]: 2026/09/23 05:14:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7739worker1 # [ 175.967979] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=0 paths=1 nar_bytes_in=7527740worker1 # [ 175.972953] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:369487741worker1 # [ 175.982478] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7742worker1 # [ 175.990577] nix-daemon[850]: reaped child process 2212, status = succeeded7743worker1 # [ 175.994857] nix-daemon[850]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7744worker1 # [ 176.834940] nix-daemon[850]: reaped child process 2221, status = succeeded7745worker1 # [ 176.914676] nix-daemon[850]: accepted connection from pid 2237, user nix-grpc-daemon (trusted)7746worker1 # [ 176.927564] nix-daemon[850]: reaped child process 2240, status = succeeded7747lb # [ 176.913796] niks3-server[1146]: 2026/09/23 05:14:29 INFO Received uploads request method=POST path=/api/pending_closures7748worker1 # [ 176.945113] nix-grpc-daemon[1906]: time=2026-09-23T05:14:29.989Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7749worker1 # [ 176.946772] nix-grpc-daemon[1906]: time=2026-09-23T05:14:29.991Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7750lb # [ 176.956119] niks3-server[1146]: 2026/09/23 05:14:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7751worker1 # [ 176.986175] nix-grpc-daemon[1906]: time=2026-09-23T05:14:30.030Z level=INFO msg="Uploading 1 narinfos"7752lb # [ 176.963714] niks3-server[1146]: 2026/09/23 05:14:29 INFO Signed narinfos id=87 count=17753lb # [ 176.968262] niks3-server[1146]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7754lb # [ 176.973279] niks3-server[1146]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7755lb # [ 176.978879] niks3-server[1146]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7756lb # [ 176.986166] niks3-server[1146]: 2026/09/23 05:14:29 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7757lb # [ 176.989277] niks3-server[1146]: 2026/09/23 05:14:29 INFO Completed upload id=877758worker1 # [ 177.012805] nix-grpc-daemon[1906]: time=2026-09-23T05:14:30.057Z level=INFO msg="Upload complete. (210ms)"7759lb # [ 176.992604] niks3-server[1146]: 2026/09/23 05:14:29 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7760worker1 # [ 177.018869] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36948 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17761worker1 # [ 177.033221] nix-daemon[850]: reaped child process 2219, status = succeeded7762client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 12.93 seconds)7763worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7764worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7765client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7766client # [ 177.166521] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7767client # Running as unit: gpu.service; invocation ID: 10a2252b04c34af69b7a57333e218aff7768client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7769lb: must succeed: curl -sf localhost:9901/clusters7770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7771lb: must succeed: curl -sf localhost:9901/clusters7772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7773worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7774worker1 # [ 177.273530] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:369487775client # [ 177.298263] nix[1581]: don't know how to build these paths:7776client # [ 177.299176] nix[1581]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7777worker1 # [ 177.280694] nix-grpc-daemon[1845]: ts=2026-09-23T05:14:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:369487778worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7779client # [ 177.319100] nix[1581]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7780worker2 # [ 177.305760] nix-grpc-daemon[1516]: ts=2026-09-23T05:14:30Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07781lb: must succeed: curl -sf localhost:9901/clusters7782lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7783lb: must succeed: curl -sf localhost:9901/clusters7784lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7785worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7786worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7787client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7788lb # [ 185.707879] envoy[910]: conn peer=[2001:db8:1::1]:44766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27137 tx=12315 ms=127567789client # [ 207.319792] nix[1581]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7790client # [ 207.326132] nix[1581]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7791client # [ 207.342599] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7792client # [ 207.344053] systemd[1]: gpu.service: Failed with result 'exit-code'.7793client # [ 207.346056] systemd[1]: gpu.service: Consumed 125ms CPU time over 30.179s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7794client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.43 seconds)7795client: must succeed: journalctl -u gpu -o cat7796client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7797lb: must succeed: curl -sf localhost:9901/clusters7798lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7799lb: must succeed: curl -sf localhost:9901/clusters7800lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7801worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7802worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7803(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 43.85 seconds)7804subtest: low disk drains a worker and builds go to the other7805worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7806worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)7807worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7808worker1 # [ 209.635907] nix-grpc-daemon[1845]: ts=2026-09-23T05:15:02Z level=info event=unhealthy reason=min_free7809worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)7810client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27811worker1 # [ 210.204198] nix-grpc-daemon[1845]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:369487812client # don't know how to build these paths:7813client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7814worker2 # [ 210.219885] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767815worker2 # [ 210.242736] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07816worker2 # [ 210.247880] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17817worker2 # [ 210.287136] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767818worker2 # [ 210.299922] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767819client # copying 1 paths to 'lb:50051'...7820client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7821worker2 # [ 210.312690] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767822worker2 # [ 210.321825] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7823worker2 # [ 210.426437] nix-daemon[873]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)7824worker2 # [ 210.439858] nix-daemon[873]: reaped child process 1926, status = succeeded7825lb # [ 210.421178] niks3-server[1146]: 2026/09/23 05:15:02 INFO Received uploads request method=POST path=/api/pending_closures7826worker2 # [ 210.458337] nix-grpc-daemon[1605]: time=2026-09-23T05:15:03.648Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7827worker2 # [ 210.460360] nix-grpc-daemon[1605]: time=2026-09-23T05:15:03.649Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7828lb # [ 210.455591] niks3-server[1146]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7829lb # [ 210.469799] niks3-server[1146]: 2026/09/23 05:15:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7830lb # [ 210.478510] niks3-server[1146]: 2026/09/23 05:15:02 INFO Signed narinfos id=88 count=17831worker2 # [ 210.505710] nix-grpc-daemon[1605]: time=2026-09-23T05:15:03.695Z level=INFO msg="Uploading 1 narinfos"7832lb # [ 210.497123] niks3-server[1146]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7833lb # [ 210.503246] niks3-server[1146]: 2026/09/23 05:15:02 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7834lb # [ 210.505786] niks3-server[1146]: 2026/09/23 05:15:02 INFO Completed upload id=887835worker2 # [ 210.534425] nix-grpc-daemon[1605]: time=2026-09-23T05:15:03.724Z level=INFO msg="Upload complete. (198ms)"7836lb # [ 210.508829] niks3-server[1146]: 2026/09/23 05:15:02 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7837worker2 # [ 210.539574] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=5127838worker2 # [ 210.547458] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767839worker2 # [ 210.558190] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7840worker2 # [ 210.568245] nix-daemon[873]: reaped child process 1921, status = succeeded7841client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7842worker2 # [ 210.577252] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7843client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7844client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7845client # farm-a> LOG-farm-a-drain7846worker2 # [ 211.810290] nix-daemon[873]: reaped child process 1930, status = succeeded7847worker2 # [ 211.902453] nix-daemon[873]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)7848worker2 # [ 211.915090] nix-daemon[873]: reaped child process 1949, status = succeeded7849lb # [ 211.894836] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received uploads request method=POST path=/api/pending_closures7850worker2 # [ 211.933467] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.123Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7851worker2 # [ 211.934943] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.125Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7852lb # [ 211.942472] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7853lb # [ 211.946223] niks3-server[1146]: 2026/09/23 05:15:04 INFO Signed narinfos id=89 count=17854worker2 # [ 211.973233] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.163Z level=INFO msg="Uploading 1 narinfos"7855lb # [ 211.947286] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7856lb # [ 211.952500] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7857lb # [ 211.955571] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7858lb # [ 211.958765] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7859lb # [ 211.964057] niks3-server[1146]: 2026/09/23 05:15:04 INFO Completed upload id=897860worker2 # [ 211.991325] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.181Z level=INFO msg="Upload complete. (206ms)"7861lb # [ 211.965795] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7862worker2 # [ 211.996652] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17863worker2 # [ 212.008308] nix-daemon[873]: reaped child process 1928, status = succeeded7864worker2 # [ 212.044723] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07865worker2 # [ 212.049893] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17866worker2 # [ 212.089400] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767867worker2 # [ 212.102154] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767868worker2 # [ 212.117131] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.307Z level=INFO msg="All 1 paths already cached"7869client # copying 1 paths to 'lb:50051'...7870client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7871worker2 # [ 212.126655] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767872worker2 # [ 212.137134] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7873worker2 # [ 212.238373] nix-daemon[873]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)7874worker2 # [ 212.251641] nix-daemon[873]: reaped child process 1958, status = succeeded7875lb # [ 212.231944] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received uploads request method=POST path=/api/pending_closures7876worker2 # [ 212.268942] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.459Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7877worker2 # [ 212.270476] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.460Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7878lb # [ 212.266124] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7879lb # [ 212.272764] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7880lb # [ 212.278469] niks3-server[1146]: 2026/09/23 05:15:04 INFO Signed narinfos id=90 count=17881worker2 # [ 212.308225] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.498Z level=INFO msg="Uploading 1 narinfos"7882lb # [ 212.286153] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7883lb # [ 212.297552] niks3-server[1146]: 2026/09/23 05:15:04 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7884lb # [ 212.301762] niks3-server[1146]: 2026/09/23 05:15:04 INFO Completed upload id=907885worker2 # [ 212.329212] nix-grpc-daemon[1605]: time=2026-09-23T05:15:05.519Z level=INFO msg="Upload complete. (179ms)"7886lb # [ 212.303957] niks3-server[1146]: 2026/09/23 05:15:04 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7887worker2 # [ 212.333326] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=6247888worker2 # [ 212.338489] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767889worker2 # [ 212.347583] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7890worker2 # [ 212.354902] nix-daemon[873]: reaped child process 1953, status = succeeded7891worker2 # [ 212.360163] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7892client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7893client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7894client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7895client # farm-b> LOG-farm-b-drain7896worker2 # [ 213.397758] nix-daemon[873]: reaped child process 1962, status = succeeded7897worker2 # [ 213.485093] nix-daemon[873]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)7898worker2 # [ 213.496597] nix-daemon[873]: reaped child process 1981, status = succeeded7899lb # [ 213.479171] niks3-server[1146]: 2026/09/23 05:15:05 INFO Received uploads request method=POST path=/api/pending_closures7900worker2 # [ 213.515711] nix-grpc-daemon[1605]: time=2026-09-23T05:15:06.705Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7901worker2 # [ 213.518393] nix-grpc-daemon[1605]: time=2026-09-23T05:15:06.707Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7902lb # [ 213.514283] niks3-server[1146]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7903lb # [ 213.528648] niks3-server[1146]: 2026/09/23 05:15:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7904lb # [ 213.533317] niks3-server[1146]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7905lb # [ 213.541519] niks3-server[1146]: 2026/09/23 05:15:05 INFO Signed narinfos id=91 count=17906worker2 # [ 213.568650] nix-grpc-daemon[1605]: time=2026-09-23T05:15:06.758Z level=INFO msg="Uploading 1 narinfos"7907lb # [ 213.546479] niks3-server[1146]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7908lb # [ 213.558968] niks3-server[1146]: 2026/09/23 05:15:05 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7909lb # [ 213.563870] niks3-server[1146]: 2026/09/23 05:15:05 INFO Completed upload id=917910worker2 # [ 213.591269] nix-grpc-daemon[1605]: time=2026-09-23T05:15:06.781Z level=INFO msg="Upload complete. (223ms)"7911lb # [ 213.566649] niks3-server[1146]: 2026/09/23 05:15:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7912worker2 # [ 213.597744] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17913worker2 # [ 213.609383] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07914worker2 # [ 213.613287] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17915worker2 # [ 213.623365] nix-daemon[873]: reaped child process 1960, status = succeeded7916worker2 # [ 213.654073] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767917worker2 # [ 213.666529] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:602767918worker2 # [ 213.679391] nix-grpc-daemon[1605]: time=2026-09-23T05:15:06.869Z level=INFO msg="All 2 paths already cached"7919client # copying 1 paths to 'lb:50051'...7920client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7921worker2 # [ 213.688639] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:602767922worker2 # [ 213.697146] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7923worker2 # [ 213.797504] nix-daemon[873]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)7924worker2 # [ 213.810946] nix-daemon[873]: reaped child process 1990, status = succeeded7925lb # [ 213.791931] niks3-server[1146]: 2026/09/23 05:15:06 INFO Received uploads request method=POST path=/api/pending_closures7926worker2 # [ 213.827082] nix-grpc-daemon[1605]: time=2026-09-23T05:15:07.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7927worker2 # [ 213.828612] nix-grpc-daemon[1605]: time=2026-09-23T05:15:07.018Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7928lb # [ 213.821237] niks3-server[1146]: 2026/09/23 05:15:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7929lb # [ 213.827078] niks3-server[1146]: 2026/09/23 05:15:06 INFO Signed narinfos id=92 count=17930worker2 # [ 213.858693] nix-grpc-daemon[1605]: time=2026-09-23T05:15:07.048Z level=INFO msg="Uploading 1 narinfos"7931lb # [ 213.835732] niks3-server[1146]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7932lb # [ 213.841984] niks3-server[1146]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7933lb # [ 213.854848] niks3-server[1146]: 2026/09/23 05:15:06 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7934lb # [ 213.860108] niks3-server[1146]: 2026/09/23 05:15:06 INFO Completed upload id=927935worker2 # [ 213.886975] nix-grpc-daemon[1605]: time=2026-09-23T05:15:07.077Z level=INFO msg="Upload complete. (176ms)"7936worker2 # [ 213.891165] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=0 paths=1 nar_bytes_in=7527937lb # [ 213.865687] niks3-server[1146]: 2026/09/23 05:15:06 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7938worker2 # [ 213.898314] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:602767939worker2 # [ 213.906347] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7940worker2 # [ 213.911069] nix-daemon[873]: reaped child process 1985, status = succeeded7941worker2 # [ 213.916268] nix-daemon[873]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)7942client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7943client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7944client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7945client # farm-top> LOG-farm-top-drain7946worker2 # [ 214.930808] nix-daemon[873]: reaped child process 1994, status = succeeded7947worker2 # [ 215.007211] nix-daemon[873]: accepted connection from pid 2010, user nix-grpc-daemon (trusted)7948worker2 # [ 215.020131] nix-daemon[873]: reaped child process 2013, status = succeeded7949lb # [ 215.002206] niks3-server[1146]: 2026/09/23 05:15:07 INFO Received uploads request method=POST path=/api/pending_closures7950worker2 # [ 215.038031] nix-grpc-daemon[1605]: time=2026-09-23T05:15:08.227Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7951worker2 # [ 215.039829] nix-grpc-daemon[1605]: time=2026-09-23T05:15:08.229Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7952lb # [ 215.047847] niks3-server[1146]: 2026/09/23 05:15:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7953lb # [ 215.052587] niks3-server[1146]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7954worker2 # [ 215.086611] nix-grpc-daemon[1605]: time=2026-09-23T05:15:08.276Z level=INFO msg="Uploading 1 narinfos"7955lb # [ 215.059126] niks3-server[1146]: 2026/09/23 05:15:07 INFO Signed narinfos id=93 count=17956lb # [ 215.063844] niks3-server[1146]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7957lb # [ 215.075142] niks3-server[1146]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7958lb # [ 215.081335] niks3-server[1146]: 2026/09/23 05:15:07 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7959lb # [ 215.088699] niks3-server[1146]: 2026/09/23 05:15:07 INFO Completed upload id=937960worker2 # [ 215.116066] nix-grpc-daemon[1605]: time=2026-09-23T05:15:08.306Z level=INFO msg="Upload complete. (193ms)"7961lb # [ 215.090797] niks3-server[1146]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7962worker2 # [ 215.122829] nix-grpc-daemon[1516]: ts=2026-09-23T05:15:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60276 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=17963worker2 # [ 215.137350] nix-daemon[873]: reaped child process 1992, status = succeeded7964client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.08 seconds)7965client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7966client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7967worker1: must succeed: rm /nix/.rw-store/fill7968worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7969worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7970worker1 # [ 215.637815] nix-grpc-daemon[1845]: ts=2026-09-23T05:15:08Z level=info event=healthy reason=min_free7971lb # [ 215.709726] envoy[910]: conn peer=[2001:db8:1::1]:34026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300437972lb # [ 215.713614] envoy[910]: conn peer=[2001:db8:1::1]:56444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9469 tx=4344 ms=49247973worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7974(finished: subtest: low disk drains a worker and builds go to the other, in 8.52 seconds)7975(finished: run the VM test script, in 217.51 seconds)7976test script finished in 217.69s7977cleanup7978kill QemuMachine (pid 49)7979client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7980client # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7981client # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7982client # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7983kill QemuMachine (pid 50)7984lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7985lb # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7986lb # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7987lb # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7988kill QemuMachine (pid 51)7989worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7990worker1 # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7991worker1 # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7992worker1 # [2026-09-23T05:15:10Z INFO virtiofsd] Client disconnected, shutting down7993kill QemuMachine (pid 52)7994worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7995worker2 # [2026-09-23T05:15:11Z INFO virtiofsd] Client disconnected, shutting down7996worker2 # [2026-09-23T05:15:11Z INFO virtiofsd] Client disconnected, shutting down7997worker2 # [2026-09-23T05:15:11Z INFO virtiofsd] Client disconnected, shutting down7998(finished: cleanup, in 1.14 seconds)7999additionally exposed symbols:8000 client, lb, worker1, worker2,8001 vlan1,8002 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh