vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #515
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb: QEMU running (pid 52)14client # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16client # Formatting '/build/vm-state-client/tmp.3GtZFRt3KY', fmt=raw size=107374182417worker1: QEMU running (pid 50)18lb # Formatting '/build/vm-state-lb/tmp.aasnhgx7QI', fmt=raw size=107374182419worker1 # Disk image does not exist, creating the virtualisation disk image...20worker2 # Disk image does not exist, creating the virtualisation disk image...21worker2: QEMU running (pid 51)22worker1 # Formatting '/build/vm-state-worker1/tmp.XHynxzKVED', fmt=raw size=107374182423worker2 # Formatting '/build/vm-state-worker2/tmp.3ylfWVNbWq', fmt=raw size=107374182424lb # mke2fs 1.47.4 (6-Mar-2025)25client # mke2fs 1.47.4 (6-Mar-2025)26worker1 # mke2fs 1.47.4 (6-Mar-2025)27lb # Discarding device blocks: 0/262144 done28client # Discarding device blocks: 0/262144 done29worker1 # Discarding device blocks: 0/262144 done30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31client # Creating filesystem with 262144 4k blocks and 65536 inodes32worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Filesystem UUID: acf17365-2a12-4f3c-bd6d-bdc5e24a1ca334client # Filesystem UUID: 99d54cb4-1d70-4ff6-9c84-e9c7f8573b2035worker1 # Filesystem UUID: f1004e8e-85db-4ece-afc6-7dd2a4f6198e36lb # Superblock backups stored on blocks:37client # Superblock backups stored on blocks:38worker1 # Superblock backups stored on blocks:39lb # 32768, 98304, 163840, 22937640client # 32768, 98304, 163840, 22937641worker1 # 32768, 98304, 163840, 22937642lb # 43client # 44(finished: start all VMs, in 0.23 seconds)45worker1 # 46client # Allocating group tables: 0/8 done47client: waiting for unit network-addresses-eth1.service48worker1 # Allocating group tables: 0/8 done49client # Writing inode tables: 0/8 done50client: waiting for the VM to finish booting51worker1 # Writing inode tables: 0/8 done52client # Creating journal (8192 blocks): done53worker2 # mke2fs 1.47.4 (6-Mar-2025)54lb # Allocating group tables: 0/8 done55worker1 # Creating journal (8192 blocks): done56client # Writing superblocks and filesystem accounting information: 0/8 done57worker2 # Discarding device blocks: 0/262144 done58lb # Writing inode tables: 0/8 done59worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes60client # 61worker1 # Writing superblocks and filesystem accounting information: 0/8 done62lb # Creating journal (8192 blocks): done63worker2 # Filesystem UUID: 8ca7ccfc-640c-4e9f-837e-77662eab02d164client # Virtualisation disk image created.65worker1 # 66lb # Writing superblocks and filesystem accounting information: 0/8 done67worker2 # Superblock backups stored on blocks:68client # Starting virtiofs daemons...69worker1 # Virtualisation disk image created.70lb # 71worker2 # 32768, 98304, 163840, 22937672client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73worker1 # Starting virtiofs daemons...74lb # Virtualisation disk image created.75worker2 # 76client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78lb # Starting virtiofs daemons...79worker2 # Allocating group tables: 0/8 done80client # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83worker2 # Writing inode tables: 0/8 done84client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...86lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87worker2 # Creating journal (8192 blocks): done88client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90lb # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker2 # Writing superblocks and filesystem accounting information: 0/8 done92client # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker2 # 96client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...98lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker2 # Virtualisation disk image created.100client # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)102lb # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...103worker2 # Starting virtiofs daemons...104client # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker1 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether106lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)107worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)108client # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests109worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...110lb # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether111worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether112client # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests113worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests114lb # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...115worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...116client # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests117worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests118lb # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests119worker1 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests120lb # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests121worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122lb # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests123worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker2 # [2026-09-23T04:42:07Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-23T04:42:07Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139client # 140client # 141client # 142client # 143client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144lb # 145lb # 146lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00147client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...148client # 149client # 150lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151lb # 152lb # 153lb # 154lb # 155lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157lb # 158lb # 159worker1 # 160worker1 # 161worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00162worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...163worker1 # 164worker1 # 165worker1 # 166worker1 # 167worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker2 # 169worker2 # 170worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00171worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...172worker1 # 173worker1 # 174client # Booting from ROM...175worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...176worker2 # 177worker2 # 178worker2 # 179worker2 # 180worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00181worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...182worker2 # 183worker2 # 184lb # Booting from ROM...185worker1 # Booting from ROM...186worker2 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4g8jh67d2wfg89946yl5bsm6viwgjv3b-nixos-system-client-test/init regInfo=/nix/store/q92shm13phfl94h0qljb58bl7yqsw0r8-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026199client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201client # [ 0.000000] NX (Execute Disable) protection: active202client # [ 0.000000] APIC: Static calls initialized203client # [ 0.000000] SMBIOS 2.8 present.204lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0205lb # [ 0.000000] BIOS-provided physical RAM map:206client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014207client # [ 0.000000] DMI: Memory slots populated: 1/1208lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable209client # [ 0.000000] Hypervisor detected: KVM210lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved211client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00213lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved214client # [ 0.000001] kvm-clock: using sched offset of 460079739 cycles215lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable216lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved217client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns218client # [ 0.000005] tsc: Detected 2400.012 MHz processor219lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved220client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved222client # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs223lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved224client # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT225lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved226client # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]227lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved228client # [ 0.002745] Using GB pages for direct mapping229lb # [ 0.000000] NX (Execute Disable) protection: active230client # [ 0.002903] RAMDISK: [mem 0x5e370000-0x5ffcffff]231lb # [ 0.000000] APIC: Static calls initialized232client # [ 0.002912] ACPI: Early table checksum verification disabled233lb # [ 0.000000] SMBIOS 2.8 present.234client # [ 0.002914] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )235lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014236worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026237client # [ 0.002918] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238lb # [ 0.000000] DMI: Memory slots populated: 1/1239lb # [ 0.000000] Hypervisor detected: KVM240client # [ 0.002922] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)241lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000242lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00243client # [ 0.002929] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)244client # [ 0.002931] ACPI: FACS 0x000000005FFE0000 000040245lb # [ 0.000000] kvm-clock: using sched offset of 470155088 cycles246client # [ 0.002933] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)247worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/53kdjxa67hi0pskimy635kxg9b1gs5nk-nixos-system-worker1-test/init regInfo=/nix/store/xprgpxkmsyza6mscqrlqixqhbif6ggrf-closure-info/registration console=ttyS0,115200n8 console=tty0248lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns249worker1 # [ 0.000000] BIOS-provided physical RAM map:250lb # [ 0.000005] tsc: Detected 2400.012 MHz processor251client # [ 0.002934] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)252worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable253lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000254client # [ 0.002936] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)255worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved256lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs257client # [ 0.002945] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)258worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved259lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT260client # [ 0.002947] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]261worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable262lb # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]263worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved264client # [ 0.002948] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]265lb # [ 0.002749] Using GB pages for direct mapping266lb # [ 0.002911] RAMDISK: [mem 0x5e36e000-0x5ffcffff]267client # [ 0.002948] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]268worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved269lb # [ 0.002922] ACPI: Early table checksum verification disabled270worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved271client # [ 0.002949] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]272lb # [ 0.002925] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved274client # [ 0.002949] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]275lb # [ 0.002929] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)276worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved277client # [ 0.002950] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]278lb # [ 0.002934] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)279worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved280client # [ 0.002950] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]281worker1 # [ 0.000000] NX (Execute Disable) protection: active282client # [ 0.003173] No NUMA configuration found283lb # [ 0.002941] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)284worker1 # [ 0.000000] APIC: Static calls initialized285client # [ 0.003174] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]286lb # [ 0.002943] ACPI: FACS 0x000000005FFE0000 000040287worker1 # [ 0.000000] SMBIOS 2.8 present.288client # [ 0.003176] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]289client # [ 0.005415] Zone ranges:290lb # [ 0.002944] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)291worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026292worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014293client # [ 0.005416] DMA [mem 0x0000000000001000-0x0000000000ffffff]294worker1 # [ 0.000000] DMI: Memory slots populated: 1/1295lb # [ 0.002946] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)296worker1 # [ 0.000000] Hypervisor detected: KVM297client # [ 0.005417] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]298client # [ 0.005418] Normal empty299worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000300client # [ 0.005419] Device empty301lb # [ 0.002948] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)302client # [ 0.005420] Movable zone start for each node303worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00304client # [ 0.005420] Early memory node ranges305lb # [ 0.002949] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)306worker1 # [ 0.000000] kvm-clock: using sched offset of 499082605 cycles307client # [ 0.005421] node 0: [mem 0x0000000000001000-0x000000000009efff]308worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h3d0pdqn1ya75a36qnamc6yf4g4lplhf-nixos-system-worker2-test/init regInfo=/nix/store/xlrn9dg9l4miz93y7jm4a6swalmr42vd-closure-info/registration console=ttyS0,115200n8 console=tty0309lb # [ 0.002950] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]310worker2 # [ 0.000000] BIOS-provided physical RAM map:311client # [ 0.005422] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]312worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns313lb # [ 0.002952] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]314worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor315worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable316client # [ 0.005423] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]317lb # [ 0.002952] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]318worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000319client # [ 0.005442] On node 0, zone DMA: 1 pages in unavailable ranges320worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved321lb # [ 0.002953] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]322worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs323client # [ 0.005699] On node 0, zone DMA: 97 pages in unavailable ranges324worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved325lb # [ 0.002953] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]326client # [ 0.032471] On node 0, zone DMA32: 40 pages in unavailable ranges327worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT328worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable329client # [ 0.032921] ACPI: PM-Timer IO Port: 0x608330worker1 # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]331lb # [ 0.002954] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]332worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved333client # [ 0.032932] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])334worker1 # [ 0.002738] Using GB pages for direct mapping335lb # [ 0.002954] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]336worker1 # [ 0.002810] RAMDISK: [mem 0x5e370000-0x5ffcffff]337worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved338lb # [ 0.003177] No NUMA configuration found339client # [ 0.032965] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23340worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved341worker1 # [ 0.002818] ACPI: Early table checksum verification disabled342client # [ 0.032968] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)343lb # [ 0.003178] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]344client # [ 0.032969] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)345worker1 # [ 0.002821] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )346worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved347lb # [ 0.003181] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]348lb # [ 0.005400] Zone ranges:349client # [ 0.032970] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)350worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved351worker1 # [ 0.002824] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)352lb # [ 0.005401] DMA [mem 0x0000000000001000-0x0000000000ffffff]353client # [ 0.032971] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)354worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved355lb # [ 0.005402] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]356lb # [ 0.005403] Normal empty357worker1 # [ 0.002828] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)358worker2 # [ 0.000000] NX (Execute Disable) protection: active359client # [ 0.032972] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)360lb # [ 0.005404] Device empty361worker2 # [ 0.000000] APIC: Static calls initialized362client # [ 0.032975] ACPI: Using ACPI (MADT) for SMP configuration information363worker2 # [ 0.000000] SMBIOS 2.8 present.364lb # [ 0.005405] Movable zone start for each node365worker1 # [ 0.002835] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)366lb # [ 0.005405] Early memory node ranges367client # [ 0.032976] ACPI: HPET id: 0x8086a201 base: 0xfed00000368worker1 # [ 0.002837] ACPI: FACS 0x000000005FFE0000 000040369client # [ 0.032979] TSC deadline timer available370lb # [ 0.005406] node 0: [mem 0x0000000000001000-0x000000000009efff]371worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014372client # [ 0.032982] CPU topo: Max. logical packages: 1373worker2 # [ 0.000000] DMI: Memory slots populated: 1/1374worker1 # [ 0.002838] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)375lb # [ 0.005407] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]376client # [ 0.032983] CPU topo: Max. logical dies: 1377worker2 # [ 0.000000] Hypervisor detected: KVM378client # [ 0.032983] CPU topo: Max. dies per package: 1379worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000380worker1 # [ 0.002840] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)381lb # [ 0.005408] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]382client # [ 0.032987] CPU topo: Max. threads per core: 1383worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00384lb # [ 0.005428] On node 0, zone DMA: 1 pages in unavailable ranges385client # [ 0.032987] CPU topo: Num. cores per package: 1386worker1 # [ 0.002841] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)387worker2 # [ 0.000001] kvm-clock: using sched offset of 487653092 cycles388lb # [ 0.005688] On node 0, zone DMA: 97 pages in unavailable ranges389client # [ 0.032988] CPU topo: Num. threads per package: 1390client # [ 0.032988] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs391worker1 # [ 0.002843] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)392lb # [ 0.031981] On node 0, zone DMA32: 40 pages in unavailable ranges393worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns394lb # [ 0.032439] ACPI: PM-Timer IO Port: 0x608395worker1 # [ 0.002844] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]396client # [ 0.033005] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()397worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor398lb # [ 0.032449] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])399worker1 # [ 0.002845] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]400worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000401client # [ 0.033035] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]402lb # [ 0.032477] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23403worker1 # [ 0.002846] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]404worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs405client # [ 0.033037] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]406lb # [ 0.032479] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)407worker1 # [ 0.002846] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]408client # [ 0.033038] [mem 0x60000000-0xafffffff] available for PCI devices409worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT410lb # [ 0.032481] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)411worker1 # [ 0.002846] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]412client # [ 0.033039] Booting paravirtualized kernel on KVM413worker2 # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]414lb # [ 0.032482] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)415worker2 # [ 0.002741] Using GB pages for direct mapping416worker1 # [ 0.002847] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]417worker2 # [ 0.002897] RAMDISK: [mem 0x5e36f000-0x5ffcffff]418client # [ 0.033042] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns419lb # [ 0.032483] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)420worker1 # [ 0.002847] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]421worker2 # [ 0.002906] ACPI: Early table checksum verification disabled422client # [ 0.037524] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1423lb # [ 0.032484] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)424worker1 # [ 0.003076] No NUMA configuration found425worker2 # [ 0.002909] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )426client # [ 0.039640] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152427lb # [ 0.032486] ACPI: Using ACPI (MADT) for SMP configuration information428worker1 # [ 0.003077] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]429lb # [ 0.032487] ACPI: HPET id: 0x8086a201 base: 0xfed00000430client # [ 0.039680] kvm-guest: PV spinlocks disabled, single CPU431worker1 # [ 0.003080] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]432worker2 # [ 0.002912] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)433lb # [ 0.032491] TSC deadline timer available434worker1 # [ 0.005616] Zone ranges:435lb # [ 0.032495] CPU topo: Max. logical packages: 1436worker2 # [ 0.002916] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)437worker1 # [ 0.005617] DMA [mem 0x0000000000001000-0x0000000000ffffff]438lb # [ 0.032496] CPU topo: Max. logical dies: 1439worker1 # [ 0.005619] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]440worker2 # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)441lb # [ 0.032496] CPU topo: Max. dies per package: 1442worker1 # [ 0.005620] Normal empty443worker1 # [ 0.005620] Device empty444worker2 # [ 0.002925] ACPI: FACS 0x000000005FFE0000 000040445lb # [ 0.032499] CPU topo: Max. threads per core: 1446worker1 # [ 0.005621] Movable zone start for each node447lb # [ 0.032500] CPU topo: Num. cores per package: 1448worker1 # [ 0.005622] Early memory node ranges449client # [ 0.039682] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4g8jh67d2wfg89946yl5bsm6viwgjv3b-nixos-system-client-test/init regInfo=/nix/store/q92shm13phfl94h0qljb58bl7yqsw0r8-closure-info/registration console=ttyS0,115200n8 console=tty0450worker2 # [ 0.002926] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)451lb # [ 0.032500] CPU topo: Num. threads per package: 1452worker1 # [ 0.005622] node 0: [mem 0x0000000000001000-0x000000000009efff]453lb # [ 0.032500] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs454worker2 # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)455worker1 # [ 0.005623] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]456lb # [ 0.032519] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()457client # [ 0.039794] Unknown kernel command line parameters "regInfo=/nix/store/q92shm13phfl94h0qljb58bl7yqsw0r8-closure-info/registration", will be passed to user space.458worker2 # [ 0.002930] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)459client # [ 0.039806] random: crng init done460worker1 # [ 0.005625] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]461lb # [ 0.032549] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]462client # [ 0.039807] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes463worker2 # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)464lb # [ 0.032550] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]465worker1 # [ 0.005654] On node 0, zone DMA: 1 pages in unavailable ranges466worker2 # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]467client # [ 0.042082] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)468lb # [ 0.032551] [mem 0x60000000-0xafffffff] available for PCI devices469worker1 # [ 0.005911] On node 0, zone DMA: 97 pages in unavailable ranges470worker2 # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]471lb # [ 0.032553] Booting paravirtualized kernel on KVM472client # [ 0.043126] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)473client # [ 0.043156] Fallback order for Node 0: 0474worker1 # [ 0.032190] On node 0, zone DMA32: 40 pages in unavailable ranges475worker2 # [ 0.002934] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]476lb # [ 0.032556] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns477client # [ 0.043159] Built 1 zonelists, mobility grouping on. Total pages: 393078478worker1 # [ 0.032655] ACPI: PM-Timer IO Port: 0x608479worker2 # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]480client # [ 0.043160] Policy zone: DMA32481lb # [ 0.037039] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1482worker1 # [ 0.032666] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])483worker2 # [ 0.002935] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]484client # [ 0.045581] mem auto-init: stack:all(zero), heap alloc:on, heap free:off485worker1 # [ 0.032696] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23486lb # [ 0.039196] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152487client # [ 0.048591] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1488worker2 # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]489lb # [ 0.039257] kvm-guest: PV spinlocks disabled, single CPU490worker1 # [ 0.032698] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)491client # [ 0.052133] allocated 3145728 bytes of page_ext492worker2 # [ 0.002936] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]493client # [ 0.061813] ftrace: allocating 48787 entries in 192 pages494worker1 # [ 0.032700] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)495worker2 # [ 0.003166] No NUMA configuration found496client # [ 0.061815] ftrace: allocated 192 pages with 2 groups497worker1 # [ 0.032701] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)498worker2 # [ 0.003167] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]499client # [ 0.062641] Dynamic Preempt: lazy500worker1 # [ 0.032702] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)501client # [ 0.062806] rcu: Preemptible hierarchical RCU implementation.502worker2 # [ 0.003169] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]503worker2 # [ 0.005444] Zone ranges:504client # [ 0.062807] rcu: RCU event tracing is enabled.505worker1 # [ 0.032703] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)506worker2 # [ 0.005445] DMA [mem 0x0000000000001000-0x0000000000ffffff]507lb # [ 0.039259] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0508client # [ 0.062808] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.509worker1 # [ 0.032706] ACPI: Using ACPI (MADT) for SMP configuration information510worker2 # [ 0.005447] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]511client # [ 0.062809] Trampoline variant of Tasks RCU enabled.512worker1 # [ 0.032707] ACPI: HPET id: 0x8086a201 base: 0xfed00000513worker2 # [ 0.005448] Normal empty514client # [ 0.062809] Rude variant of Tasks RCU enabled.515worker2 # [ 0.005449] Device empty516worker1 # [ 0.032710] TSC deadline timer available517lb # [ 0.039353] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.518worker1 # [ 0.032714] CPU topo: Max. logical packages: 1519client # [ 0.062810] Tracing variant of Tasks RCU enabled.520worker2 # [ 0.005450] Movable zone start for each node521lb # [ 0.039365] random: crng init done522worker2 # [ 0.005450] Early memory node ranges523worker1 # [ 0.032714] CPU topo: Max. logical dies: 1524client # [ 0.062810] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.525lb # [ 0.039366] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes526worker1 # [ 0.032715] CPU topo: Max. dies per package: 1527worker2 # [ 0.005451] node 0: [mem 0x0000000000001000-0x000000000009efff]528worker1 # [ 0.032718] CPU topo: Max. threads per core: 1529client # [ 0.062811] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1530worker2 # [ 0.005452] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]531lb # [ 0.041507] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)532worker1 # [ 0.032719] CPU topo: Num. cores per package: 1533client # [ 0.062830] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.534worker2 # [ 0.005453] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]535worker1 # [ 0.032719] CPU topo: Num. threads per package: 1536lb # [ 0.042584] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)537lb # [ 0.042615] Fallback order for Node 0: 0538worker2 # [ 0.005474] On node 0, zone DMA: 1 pages in unavailable ranges539worker1 # [ 0.032719] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs540client # [ 0.062832] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.541worker2 # [ 0.005744] On node 0, zone DMA: 97 pages in unavailable ranges542lb # [ 0.042618] Built 1 zonelists, mobility grouping on. Total pages: 393078543worker1 # [ 0.032737] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()544lb # [ 0.042619] Policy zone: DMA32545client # [ 0.062833] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546worker2 # [ 0.032237] On node 0, zone DMA32: 40 pages in unavailable ranges547worker1 # [ 0.032767] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]548worker2 # [ 0.032706] ACPI: PM-Timer IO Port: 0x608549lb # [ 0.045288] mem auto-init: stack:all(zero), heap alloc:on, heap free:off550client # [ 0.067197] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16551worker2 # [ 0.032717] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])552worker1 # [ 0.032769] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]553lb # [ 0.048401] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1554client # [ 0.067465] rcu: srcu_init: Setting srcu_struct sizes based on contention.555lb # [ 0.051720] allocated 3145728 bytes of page_ext556worker1 # [ 0.032770] [mem 0x60000000-0xafffffff] available for PCI devices557worker2 # [ 0.032747] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23558lb # [ 0.061566] ftrace: allocating 48787 entries in 192 pages559worker1 # [ 0.032772] Booting paravirtualized kernel on KVM560client # [ 0.067471] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns561worker2 # [ 0.032750] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)562lb # [ 0.061568] ftrace: allocated 192 pages with 2 groups563lb # [ 0.062466] Dynamic Preempt: lazy564worker2 # [ 0.032752] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)565worker1 # [ 0.032775] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns566client # [ 0.067572] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)567lb # [ 0.062616] rcu: Preemptible hierarchical RCU implementation.568client # [ 0.071109] Console: colour VGA+ 80x25569worker2 # [ 0.032753] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)570worker1 # [ 0.037242] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1571lb # [ 0.062617] rcu: RCU event tracing is enabled.572client # [ 0.071112] printk: legacy console [tty0] enabled573worker2 # [ 0.032754] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)574client # [ 0.100577] printk: legacy console [ttyS0] enabled575worker1 # [ 0.039404] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152576lb # [ 0.062617] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.577worker2 # [ 0.032754] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)578worker1 # [ 0.039454] kvm-guest: PV spinlocks disabled, single CPU579lb # [ 0.062619] Trampoline variant of Tasks RCU enabled.580client # [ 0.198202] ACPI: Core revision 20250807581lb # [ 0.062619] Rude variant of Tasks RCU enabled.582worker2 # [ 0.032757] ACPI: Using ACPI (MADT) for SMP configuration information583lb # [ 0.062620] Tracing variant of Tasks RCU enabled.584worker2 # [ 0.032758] ACPI: HPET id: 0x8086a201 base: 0xfed00000585client # [ 0.199061] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns586worker2 # [ 0.032761] TSC deadline timer available587lb # [ 0.062620] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.588worker2 # [ 0.032765] CPU topo: Max. logical packages: 1589client # [ 0.200534] APIC: Switch to symmetric I/O mode setup590worker2 # [ 0.032766] CPU topo: Max. logical dies: 1591lb # [ 0.062621] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1592client # [ 0.201499] x2apic enabled593worker2 # [ 0.032766] CPU topo: Max. dies per package: 1594worker1 # [ 0.039456] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/53kdjxa67hi0pskimy635kxg9b1gs5nk-nixos-system-worker1-test/init regInfo=/nix/store/xprgpxkmsyza6mscqrlqixqhbif6ggrf-closure-info/registration console=ttyS0,115200n8 console=tty0595lb # [ 0.062677] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.596worker2 # [ 0.032770] CPU topo: Max. threads per core: 1597client # [ 0.202235] APIC: Switched APIC routing to: physical x2apic598worker2 # [ 0.032770] CPU topo: Num. cores per package: 1599lb # [ 0.062679] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.600worker2 # [ 0.032771] CPU topo: Num. threads per package: 1601worker1 # [ 0.039552] Unknown kernel command line parameters "regInfo=/nix/store/xprgpxkmsyza6mscqrlqixqhbif6ggrf-closure-info/registration", will be passed to user space.602worker1 # [ 0.039564] random: crng init done603worker2 # [ 0.032771] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs604lb # [ 0.062680] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.605client # [ 0.204029] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1606worker2 # [ 0.032790] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()607worker1 # [ 0.039565] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes608lb # [ 0.067086] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16609lb # [ 0.067361] rcu: srcu_init: Setting srcu_struct sizes based on contention.610worker2 # [ 0.032820] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]611worker1 # [ 0.041697] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)612client # [ 0.205024] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns613worker2 # [ 0.032821] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]614worker1 # [ 0.042758] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)615lb # [ 0.067367] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns616worker1 # [ 0.042790] Fallback order for Node 0: 0617worker2 # [ 0.032823] [mem 0x60000000-0xafffffff] available for PCI devices618client # [ 0.206707] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)619worker2 # [ 0.032825] Booting paravirtualized kernel on KVM620worker1 # [ 0.042793] Built 1 zonelists, mobility grouping on. Total pages: 393078621lb # [ 0.067469] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)622worker1 # [ 0.042794] Policy zone: DMA32623lb # [ 0.070984] Console: colour VGA+ 80x25624client # [ 0.207791] x86/cpu: User Mode Instruction Prevention (UMIP) activated625lb # [ 0.070987] printk: legacy console [tty0] enabled626worker1 # [ 0.045525] mem auto-init: stack:all(zero), heap alloc:on, heap free:off627worker2 # [ 0.032827] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns628lb # [ 0.100848] printk: legacy console [ttyS0] enabled629worker1 # [ 0.048612] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1630client # [ 0.208869] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127631worker2 # [ 0.037292] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1632worker1 # [ 0.051907] allocated 3145728 bytes of page_ext633lb # [ 0.198886] ACPI: Core revision 20250807634worker2 # [ 0.039451] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152635client # [ 0.209705] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0636worker1 # [ 0.061840] ftrace: allocating 48787 entries in 192 pages637worker2 # [ 0.039496] kvm-guest: PV spinlocks disabled, single CPU638worker1 # [ 0.061841] ftrace: allocated 192 pages with 2 groups639lb # [ 0.199801] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns640worker1 # [ 0.062668] Dynamic Preempt: lazy641client # [ 0.210708] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto642worker1 # [ 0.062843] rcu: Preemptible hierarchical RCU implementation.643lb # [ 0.201297] APIC: Switch to symmetric I/O mode setup644worker1 # [ 0.062844] rcu: RCU event tracing is enabled.645lb # [ 0.202292] x2apic enabled646client # [ 0.212704] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl647worker1 # [ 0.062844] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.648worker1 # [ 0.062846] Trampoline variant of Tasks RCU enabled.649lb # [ 0.203070] APIC: Switched APIC routing to: physical x2apic650client # [ 0.213705] Transient Scheduler Attacks: Vulnerable: No microcode651worker1 # [ 0.062846] Rude variant of Tasks RCU enabled.652worker2 # [ 0.039497] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h3d0pdqn1ya75a36qnamc6yf4g4lplhf-nixos-system-worker2-test/init regInfo=/nix/store/xlrn9dg9l4miz93y7jm4a6swalmr42vd-closure-info/registration console=ttyS0,115200n8 console=tty0653worker1 # [ 0.062847] Tracing variant of Tasks RCU enabled.654client # [ 0.214704] Spectre V2 : Mitigation: Enhanced / Automatic IBRS655worker1 # [ 0.062847] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.656lb # [ 0.204864] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1657client # [ 0.215704] Speculative Return Stack Overflow: Mitigation: Safe RET658worker2 # [ 0.039601] Unknown kernel command line parameters "regInfo=/nix/store/xlrn9dg9l4miz93y7jm4a6swalmr42vd-closure-info/registration", will be passed to user space.659worker2 # [ 0.039613] random: crng init done660worker1 # [ 0.062848] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1661worker2 # [ 0.039614] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes662worker1 # [ 0.062864] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.663lb # [ 0.205840] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns664worker2 # [ 0.041733] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)665worker1 # [ 0.062865] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.666lb # [ 0.207488] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)667worker2 # [ 0.043023] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)668worker2 # [ 0.043054] Fallback order for Node 0: 0669worker1 # [ 0.062866] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.670client # [ 0.216704] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization671lb # [ 0.208572] x86/cpu: User Mode Instruction Prevention (UMIP) activated672worker1 # [ 0.067247] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16673worker2 # [ 0.043058] Built 1 zonelists, mobility grouping on. Total pages: 393078674client # [ 0.218712] Spectre V2 : Enabling IBPB for BPF675worker2 # [ 0.043059] Policy zone: DMA32676lb # [ 0.209623] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127677worker1 # [ 0.067529] rcu: srcu_init: Setting srcu_struct sizes based on contention.678worker2 # [ 0.045676] mem auto-init: stack:all(zero), heap alloc:on, heap free:off679client # [ 0.219705] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier680lb # [ 0.210485] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0681worker1 # [ 0.067534] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns682worker2 # [ 0.048837] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1683client # [ 0.220705] active return thunk: srso_alias_return_thunk684worker2 # [ 0.052135] allocated 3145728 bytes of page_ext685worker1 # [ 0.067637] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)686worker2 # [ 0.062065] ftrace: allocating 48787 entries in 192 pages687lb # [ 0.211488] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto688worker1 # [ 0.071180] Console: colour VGA+ 80x25689client # [ 0.221725] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'690worker2 # [ 0.062067] ftrace: allocated 192 pages with 2 groups691worker1 # [ 0.071183] printk: legacy console [tty0] enabled692worker2 # [ 0.062914] Dynamic Preempt: lazy693worker1 # [ 0.100586] printk: legacy console [ttyS0] enabled694client # [ 0.222704] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695lb # [ 0.213485] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl696worker2 # [ 0.063063] rcu: Preemptible hierarchical RCU implementation.697worker1 # [ 0.198707] ACPI: Core revision 20250807698worker2 # [ 0.063064] rcu: RCU event tracing is enabled.699client # [ 0.223704] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'700lb # [ 0.214486] Transient Scheduler Attacks: Vulnerable: No microcode701worker2 # [ 0.063065] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.702client # [ 0.224704] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'703lb # [ 0.215485] Spectre V2 : Mitigation: Enhanced / Automatic IBRS704worker2 # [ 0.063066] Trampoline variant of Tasks RCU enabled.705worker1 # [ 0.199591] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns706worker2 # [ 0.063066] Rude variant of Tasks RCU enabled.707lb # [ 0.216485] Speculative Return Stack Overflow: Mitigation: Safe RET708client # [ 0.225704] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'709worker1 # [ 0.201082] APIC: Switch to symmetric I/O mode setup710worker2 # [ 0.063067] Tracing variant of Tasks RCU enabled.711worker1 # [ 0.202018] x2apic enabled712client # [ 0.226704] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'713worker2 # [ 0.063067] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.714worker2 # [ 0.063068] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1715worker1 # [ 0.202755] APIC: Switched APIC routing to: physical x2apic716client # [ 0.227704] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'717worker2 # [ 0.063082] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.718client # [ 0.228705] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256719lb # [ 0.217484] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization720worker1 # [ 0.204581] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1721worker2 # [ 0.063084] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.722client # [ 0.230447] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64723lb # [ 0.219494] Spectre V2 : Enabling IBPB for BPF724client # [ 0.231414] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512725worker2 # [ 0.063085] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.726worker1 # [ 0.205531] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns727worker2 # [ 0.067513] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16728lb # [ 0.220486] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier729client # [ 0.231704] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024730worker2 # [ 0.067984] rcu: srcu_init: Setting srcu_struct sizes based on contention.731worker1 # [ 0.207136] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)732lb # [ 0.221485] active return thunk: srso_alias_return_thunk733client # [ 0.232704] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8734worker2 # [ 0.067991] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns735worker1 # [ 0.208221] x86/cpu: User Mode Instruction Prevention (UMIP) activated736lb # [ 0.222505] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'737client # [ 0.233705] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.738worker2 # [ 0.068097] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)739worker1 # [ 0.209295] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127740lb # [ 0.223484] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'741worker2 # [ 0.071666] Console: colour VGA+ 80x25742worker2 # [ 0.071670] printk: legacy console [tty0] enabled743worker1 # [ 0.210133] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0744lb # [ 0.224484] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'745worker2 # [ 0.101718] printk: legacy console [ttyS0] enabled746lb # [ 0.225484] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'747worker2 # [ 0.198351] ACPI: Core revision 20250807748worker1 # [ 0.211136] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto749lb # [ 0.226484] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'750worker2 # [ 0.199232] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns751worker1 # [ 0.212133] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl752lb # [ 0.227485] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'753worker2 # [ 0.200813] APIC: Switch to symmetric I/O mode setup754worker1 # [ 0.213133] Transient Scheduler Attacks: Vulnerable: No microcode755lb # [ 0.228484] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'756worker2 # [ 0.201820] x2apic enabled757worker1 # [ 0.214132] Spectre V2 : Mitigation: Enhanced / Automatic IBRS758lb # [ 0.230412] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256759worker2 # [ 0.202604] APIC: Switched APIC routing to: physical x2apic760worker1 # [ 0.215132] Speculative Return Stack Overflow: Mitigation: Safe RET761lb # [ 0.231484] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64762lb # [ 0.232484] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512763worker2 # [ 0.204509] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1764lb # [ 0.233485] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024765worker2 # [ 0.205523] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns766lb # [ 0.234485] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8767worker1 # [ 0.216132] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization768worker1 # [ 0.218139] Spectre V2 : Enabling IBPB for BPF769worker2 # [ 0.207199] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)770lb # [ 0.235485] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.771worker2 # [ 0.208290] x86/cpu: User Mode Instruction Prevention (UMIP) activated772worker1 # [ 0.219133] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier773worker1 # [ 0.220133] active return thunk: srso_alias_return_thunk774worker2 # [ 0.210104] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127775worker2 # [ 0.210907] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0776worker1 # [ 0.221154] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'777worker1 # [ 0.222133] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'778worker2 # [ 0.211979] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto779worker1 # [ 0.223133] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'780worker1 # [ 0.224132] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'781worker2 # [ 0.213196] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl782worker1 # [ 0.225132] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'783worker2 # [ 0.215197] Transient Scheduler Attacks: Vulnerable: No microcode784worker1 # [ 0.226132] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'785worker2 # [ 0.216196] Spectre V2 : Mitigation: Enhanced / Automatic IBRS786worker2 # [ 0.217196] Speculative Return Stack Overflow: Mitigation: Safe RET787worker1 # [ 0.227132] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'788worker1 # [ 0.229132] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256789worker1 # [ 0.230132] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64790worker1 # [ 0.231132] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512791worker2 # [ 0.218196] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization792worker1 # [ 0.232132] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024793worker2 # [ 0.220195] Spectre V2 : Enabling IBPB for BPF794worker1 # [ 0.233132] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8795worker2 # [ 0.221197] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier796worker1 # [ 0.234132] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.797worker2 # [ 0.222196] active return thunk: srso_alias_return_thunk798worker2 # [ 0.223217] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799worker2 # [ 0.224196] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800worker2 # [ 0.225196] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801worker2 # [ 0.226974] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802client # [ 0.267683] Freeing SMP alternatives memory: 44K803worker2 # [ 0.227979] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'804client # [ 0.268334] pid_max: default: 32768 minimum: 301805worker2 # [ 0.228963] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'806client # [ 0.269354] LSM: initializing lsm=capability,landlock,yama,bpf,ima807client # [ 0.269797] landlock: Up and running.808worker2 # [ 0.230012] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'809client # [ 0.270705] Yama: becoming mindful.810worker2 # [ 0.231197] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256811client # [ 0.271916] LSM support for eBPF active812worker2 # [ 0.232196] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64813client # [ 0.272805] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)814worker2 # [ 0.233196] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512815client # [ 0.273740] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)816worker2 # [ 0.234196] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024817worker2 # [ 0.235196] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8818client # [ 0.276361] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)819worker2 # [ 0.236196] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.820client # [ 0.277210] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.277709] ... version: 2822client # [ 0.278478] ... bit width: 48823lb # [ 0.270510] Freeing SMP alternatives memory: 44K824client # [ 0.278707] ... generic counters: 6825lb # [ 0.271292] pid_max: default: 32768 minimum: 301826client # [ 0.279424] ... generic bitmap: 000000000000003f827lb # [ 0.272147] LSM: initializing lsm=capability,landlock,yama,bpf,ima828client # [ 0.279755] ... fixed-purpose counters: 0829lb # [ 0.272580] landlock: Up and running.830client # [ 0.280464] ... fixed-purpose bitmap: 0000000000000000831lb # [ 0.273945] Yama: becoming mindful.832client # [ 0.280707] ... value mask: 0000ffffffffffff833lb # [ 0.274698] LSM support for eBPF active834client # [ 0.281671] ... max period: 00007fffffffffff835lb # [ 0.275607] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)836client # [ 0.282471] ... global_ctrl mask: 000000000000003f837client # [ 0.282827] signal: max sigframe size: 3376838lb # [ 0.276539] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)839client # [ 0.283662] rcu: Hierarchical SRCU implementation.840client # [ 0.284335] rcu: Max phase no-delay instances is 400.841lb # [ 0.279624] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)842lb # [ 0.281067] Performance Events: Fam17h+ core perfctr, AMD PMU driver.843lb # [ 0.281495] ... version: 2844lb # [ 0.282246] ... bit width: 48845client # [ 0.289342] smp: Bringing up secondary CPUs ...846lb # [ 0.282488] ... generic counters: 6847client # [ 0.289720] smp: Brought up 1 node, 1 CPU848lb # [ 0.283207] ... generic bitmap: 000000000000003f849worker1 # [ 0.268391] Freeing SMP alternatives memory: 44K850lb # [ 0.283487] ... fixed-purpose counters: 0851client # [ 0.290486] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)852worker1 # [ 0.269135] pid_max: default: 32768 minimum: 301853lb # [ 0.284171] ... fixed-purpose bitmap: 0000000000000000854worker1 # [ 0.269948] LSM: initializing lsm=capability,landlock,yama,bpf,ima855lb # [ 0.284487] ... value mask: 0000ffffffffffff856worker1 # [ 0.270961] landlock: Up and running.857client # [ 0.291662] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)858lb # [ 0.285359] ... max period: 00007fffffffffff859worker1 # [ 0.271558] Yama: becoming mindful.860client # [ 0.292931] devtmpfs: initialized861lb # [ 0.286135] ... global_ctrl mask: 000000000000003f862worker1 # [ 0.272353] LSM support for eBPF active863client # [ 0.293723] x86/mm: Memory block size: 128MB864lb # [ 0.286585] signal: max sigframe size: 3376865worker1 # [ 0.273241] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)866lb # [ 0.287364] rcu: Hierarchical SRCU implementation.867lb # [ 0.288097] rcu: Max phase no-delay instances is 400.868worker1 # [ 0.274179] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)869client # [ 0.295586] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)870client # [ 0.296628] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).871client # [ 0.297773] pinctrl core: initialized pinctrl subsystem872worker1 # [ 0.277032] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)873client # [ 0.298890] PM: RTC time: 04:42:07, date: 2026-09-23874worker1 # [ 0.277648] Performance Events: Fam17h+ core perfctr, AMD PMU driver.875lb # [ 0.293005] smp: Bringing up secondary CPUs ...876worker1 # [ 0.278138] ... version: 2877lb # [ 0.293496] smp: Brought up 1 node, 1 CPU878worker1 # [ 0.278857] ... bit width: 48879worker1 # [ 0.279135] ... generic counters: 6880lb # [ 0.294185] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)881client # [ 0.302382] NET: Registered PF_NETLINK/PF_ROUTE protocol family882worker1 # [ 0.279857] ... generic bitmap: 000000000000003f883worker1 # [ 0.280180] ... fixed-purpose counters: 0884worker2 # [ 0.270928] Freeing SMP alternatives memory: 44K885client # [ 0.303217] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations886lb # [ 0.294672] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)887worker1 # [ 0.280865] ... fixed-purpose bitmap: 0000000000000000888worker2 # [ 0.271199] pid_max: default: 32768 minimum: 301889lb # [ 0.295736] devtmpfs: initialized890worker1 # [ 0.281135] ... value mask: 0000ffffffffffff891client # [ 0.303727] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations892worker2 # [ 0.272294] LSM: initializing lsm=capability,landlock,yama,bpf,ima893lb # [ 0.296483] x86/mm: Memory block size: 128MB894worker1 # [ 0.281987] ... max period: 00007fffffffffff895worker2 # [ 0.273291] landlock: Up and running.896worker1 # [ 0.282807] ... global_ctrl mask: 000000000000003f897client # [ 0.304982] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations898worker2 # [ 0.274197] Yama: becoming mindful.899worker1 # [ 0.283260] signal: max sigframe size: 3376900client # [ 0.305716] audit: initializing netlink subsys (disabled)901worker2 # [ 0.274992] LSM support for eBPF active902lb # [ 0.298254] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)903worker1 # [ 0.284154] rcu: Hierarchical SRCU implementation.904worker2 # [ 0.275759] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)905client # [ 0.306709] audit: type=2000 audit(1790138528.369:1): state=initialized audit_enabled=0 res=1906worker1 # [ 0.284901] rcu: Max phase no-delay instances is 400.907lb # [ 0.299387] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).908client # [ 0.307835] thermal_sys: Registered thermal governor 'fair_share'909worker2 # [ 0.277087] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)910lb # [ 0.300581] pinctrl core: initialized pinctrl subsystem911client # [ 0.307837] thermal_sys: Registered thermal governor 'bang_bang'912lb # [ 0.301683] PM: RTC time: 04:42:07, date: 2026-09-23913client # [ 0.308708] thermal_sys: Registered thermal governor 'step_wise'914worker2 # [ 0.279315] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)915client # [ 0.309672] thermal_sys: Registered thermal governor 'user_space'916client # [ 0.310472] thermal_sys: Registered thermal governor 'power_allocator'917worker2 # [ 0.280785] Performance Events: Fam17h+ core perfctr, AMD PMU driver.918worker1 # [ 0.289789] smp: Bringing up secondary CPUs ...919client # [ 0.310720] cpuidle: using governor menu920worker2 # [ 0.281205] ... version: 2921lb # [ 0.305259] NET: Registered PF_NETLINK/PF_ROUTE protocol family922worker1 # [ 0.290149] smp: Brought up 1 node, 1 CPU923worker2 # [ 0.281886] ... bit width: 48924worker1 # [ 0.290799] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)925client # [ 0.312917] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5926worker2 # [ 0.282198] ... generic counters: 6927lb # [ 0.306674] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations928worker2 # [ 0.282894] ... generic bitmap: 000000000000003f929lb # [ 0.307514] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations930worker2 # [ 0.283199] ... fixed-purpose counters: 0931client # [ 0.313937] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]932worker1 # [ 0.291338] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)933worker2 # [ 0.283894] ... fixed-purpose bitmap: 0000000000000000934client # [ 0.314710] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry935worker1 # [ 0.292415] devtmpfs: initialized936lb # [ 0.308762] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations937worker2 # [ 0.284198] ... value mask: 0000ffffffffffff938client # [ 0.315719] PCI: Using configuration type 1 for base access939worker1 # [ 0.293149] x86/mm: Memory block size: 128MB940lb # [ 0.309499] audit: initializing netlink subsys (disabled)941worker2 # [ 0.285058] ... max period: 00007fffffffffff942lb # [ 0.310553] thermal_sys: Registered thermal governor 'fair_share'943worker2 # [ 0.285866] ... global_ctrl mask: 000000000000003f944client # [ 0.316787] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.945worker2 # [ 0.286307] signal: max sigframe size: 3376946lb # [ 0.310556] thermal_sys: Registered thermal governor 'bang_bang'947worker1 # [ 0.294875] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)948worker2 # [ 0.287111] rcu: Hierarchical SRCU implementation.949lb # [ 0.311488] thermal_sys: Registered thermal governor 'step_wise'950worker1 # [ 0.296007] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).951worker2 # [ 0.287818] rcu: Max phase no-delay instances is 400.952lb # [ 0.312385] thermal_sys: Registered thermal governor 'user_space'953worker1 # [ 0.297206] pinctrl core: initialized pinctrl subsystem954lb # [ 0.313177] audit: type=2000 audit(1790138528.383:1): state=initialized audit_enabled=0 res=1955worker1 # [ 0.298358] PM: RTC time: 04:42:07, date: 2026-09-23956lb # [ 0.314490] thermal_sys: Registered thermal governor 'power_allocator'957lb # [ 0.314504] cpuidle: using governor menu958lb # [ 0.316747] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5959worker2 # [ 0.292788] smp: Bringing up secondary CPUs ...960client # [ 0.324730] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages961worker1 # [ 0.301999] NET: Registered PF_NETLINK/PF_ROUTE protocol family962worker2 # [ 0.293213] smp: Brought up 1 node, 1 CPU963client # [ 0.325708] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page964lb # [ 0.317797] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]965worker2 # [ 0.293909] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)966worker1 # [ 0.303381] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations967lb # [ 0.318490] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry968lb # [ 0.319499] PCI: Using configuration type 1 for base access969worker1 # [ 0.304157] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations970worker2 # [ 0.294401] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)971worker1 # [ 0.305415] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations972lb # [ 0.320595] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.973worker2 # [ 0.295471] devtmpfs: initialized974worker1 # [ 0.306146] audit: initializing netlink subsys (disabled)975worker2 # [ 0.296354] x86/mm: Memory block size: 128MB976client # [ 0.330708] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages977worker1 # [ 0.307136] audit: type=2000 audit(1790138528.396:1): state=initialized audit_enabled=0 res=1978client # [ 0.331707] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page979worker1 # [ 0.308311] thermal_sys: Registered thermal governor 'fair_share'980worker2 # [ 0.298189] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)981worker1 # [ 0.308313] thermal_sys: Registered thermal governor 'bang_bang'982worker2 # [ 0.299139] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).983lb # [ 0.325717] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages984worker1 # [ 0.309136] thermal_sys: Registered thermal governor 'step_wise'985worker2 # [ 0.300290] pinctrl core: initialized pinctrl subsystem986lb # [ 0.326488] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page987worker1 # [ 0.310137] thermal_sys: Registered thermal governor 'user_space'988worker2 # [ 0.301483] PM: RTC time: 04:42:07, date: 2026-09-23989worker1 # [ 0.311121] thermal_sys: Registered thermal governor 'power_allocator'990worker1 # [ 0.311912] cpuidle: using governor menu991client # [ 0.337796] ACPI: Added _OSI(Module Device)992client # [ 0.338502] ACPI: Added _OSI(Processor Device)993worker1 # [ 0.314202] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5994client # [ 0.338708] ACPI: Added _OSI(Processor Aggregator Device)995worker2 # [ 0.305012] NET: Registered PF_NETLINK/PF_ROUTE protocol family996lb # [ 0.331489] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages997worker1 # [ 0.315378] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]998lb # [ 0.332487] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page999worker2 # [ 0.306453] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1000worker1 # [ 0.316137] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1001worker1 # [ 0.317147] PCI: Using configuration type 1 for base access1002worker2 # [ 0.307221] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1003worker1 # [ 0.318290] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1004worker2 # [ 0.308476] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1005worker2 # [ 0.309209] audit: initializing netlink subsys (disabled)1006worker2 # [ 0.310430] thermal_sys: Registered thermal governor 'fair_share'1007worker2 # [ 0.310432] thermal_sys: Registered thermal governor 'bang_bang'1008worker2 # [ 0.311200] thermal_sys: Registered thermal governor 'step_wise'1009worker2 # [ 0.312202] audit: type=2000 audit(1790138528.415:1): state=initialized audit_enabled=0 res=11010worker2 # [ 0.314202] thermal_sys: Registered thermal governor 'user_space'1011client # [ 0.350897] ACPI: 1 ACPI AML tables successfully acquired and loaded1012worker1 # [ 0.325148] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1013lb # [ 0.342892] ACPI: Added _OSI(Module Device)1014lb # [ 0.343489] ACPI: Added _OSI(Processor Device)1015worker2 # [ 0.314203] thermal_sys: Registered thermal governor 'power_allocator'1016worker1 # [ 0.326136] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1017client # [ 0.352658] ACPI: Interpreter enabled1018worker2 # [ 0.315191] cpuidle: using governor menu1019lb # [ 0.344252] ACPI: Added _OSI(Processor Aggregator Device)1020client # [ 0.353157] ACPI: PM: (supports S0 S3 S4 S5)1021worker2 # [ 0.317437] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51022worker2 # [ 0.318434] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1023client # [ 0.355708] ACPI: Using IOAPIC for interrupt routing1024worker2 # [ 0.319201] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1025worker1 # [ 0.331137] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1026client # [ 0.356621] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1027worker2 # [ 0.320211] PCI: Using configuration type 1 for base access1028worker1 # [ 0.332136] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1029worker2 # [ 0.321335] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1030client # [ 0.359707] PCI: Using E820 reservations for host bridge windows1031client # [ 0.360834] ACPI: Enabled 2 GPEs in block 00 to 3F1032lb # [ 0.353040] ACPI: 1 ACPI AML tables successfully acquired and loaded1033lb # [ 0.354462] ACPI: Interpreter enabled1034lb # [ 0.355023] ACPI: PM: (supports S0 S3 S4 S5)1035worker2 # [ 0.326462] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1036worker2 # [ 0.327200] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1037lb # [ 0.357488] ACPI: Using IOAPIC for interrupt routing1038worker1 # [ 0.343383] ACPI: Added _OSI(Module Device)1039lb # [ 0.358383] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1040worker1 # [ 0.344115] ACPI: Added _OSI(Processor Device)1041client # [ 0.369367] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1042worker1 # [ 0.344713] ACPI: Added _OSI(Processor Aggregator Device)1043worker2 # [ 0.332199] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1044client # [ 0.369715] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1045worker2 # [ 0.333198] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1046lb # [ 0.361488] PCI: Using E820 reservations for host bridge windows1047client # [ 0.370785] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1048lb # [ 0.362577] ACPI: Enabled 2 GPEs in block 00 to 3F1049client # [ 0.371838] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1050client # [ 0.373076] PCI host bridge to bus 0000:001051client # [ 0.373711] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1052worker1 # [ 0.350582] ACPI: 1 ACPI AML tables successfully acquired and loaded1053client # [ 0.374707] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1054client # [ 0.375707] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1055client # [ 0.376707] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1056client # [ 0.377707] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1057worker1 # [ 0.354537] ACPI: Interpreter enabled1058worker1 # [ 0.355151] ACPI: PM: (supports S0 S3 S4 S5)1059worker2 # [ 0.343540] ACPI: Added _OSI(Module Device)1060client # [ 0.378707] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1061worker2 # [ 0.344200] ACPI: Added _OSI(Processor Device)1062worker1 # [ 0.355891] ACPI: Using IOAPIC for interrupt routing1063lb # [ 0.372353] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1064client # [ 0.379707] pci_bus 0000:00: root bus resource [bus 00-ff]1065worker2 # [ 0.344891] ACPI: Added _OSI(Processor Aggregator Device)1066lb # [ 0.373277] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1067client # [ 0.380607] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1068lb # [ 0.374568] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1069client # [ 0.382147] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1070worker1 # [ 0.358220] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1071lb # [ 0.375617] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1072worker1 # [ 0.359136] PCI: Using E820 reservations for host bridge windows1073lb # [ 0.376839] PCI host bridge to bus 0000:001074lb # [ 0.377492] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1075lb # [ 0.378487] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1076worker1 # [ 0.362247] ACPI: Enabled 2 GPEs in block 00 to 3F1077lb # [ 0.379488] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1078lb # [ 0.380487] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1079worker2 # [ 0.353778] ACPI: 1 ACPI AML tables successfully acquired and loaded1080lb # [ 0.381487] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1081client # [ 0.384783] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1082client # [ 0.385720] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1083lb # [ 0.382487] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1084client # [ 0.386671] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1085lb # [ 0.383487] pci_bus 0000:00: root bus resource [bus 00-ff]1086worker2 # [ 0.357469] ACPI: Interpreter enabled1087lb # [ 0.384517] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1088worker2 # [ 0.358150] ACPI: PM: (supports S0 S3 S4 S5)1089client # [ 0.387749] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1090worker1 # [ 0.369888] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1091worker2 # [ 0.358687] ACPI: Using IOAPIC for interrupt routing1092worker1 # [ 0.370141] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1093lb # [ 0.385947] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1094worker2 # [ 0.359293] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1095client # [ 0.389643] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1096worker1 # [ 0.371214] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1097worker2 # [ 0.360199] PCI: Using E820 reservations for host bridge windows1098worker1 # [ 0.372258] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1099worker1 # [ 0.373504] PCI host bridge to bus 0000:001100client # [ 0.392727] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1101worker2 # [ 0.363328] ACPI: Enabled 2 GPEs in block 00 to 3F1102worker1 # [ 0.374139] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1103client # [ 0.393592] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1104worker1 # [ 0.375135] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1105client # [ 0.394411] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1106client # [ 0.394714] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1107worker1 # [ 0.376135] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1108lb # [ 0.388591] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1109worker1 # [ 0.377135] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1110lb # [ 0.389500] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1111client # [ 0.396253] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1112worker1 # [ 0.378136] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1113lb # [ 0.390492] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1114worker1 # [ 0.379135] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1115worker1 # [ 0.380135] pci_bus 0000:00: root bus resource [bus 00-ff]1116lb # [ 0.391808] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1117client # [ 0.398697] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1118worker1 # [ 0.381072] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1119client # [ 0.399551] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1120worker2 # [ 0.371987] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1121lb # [ 0.393210] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1122client # [ 0.400465] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1123worker1 # [ 0.382574] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1124worker2 # [ 0.372927] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1125client # [ 0.401287] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1126worker2 # [ 0.374777] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1127lb # [ 0.396495] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1128worker2 # [ 0.375321] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1129lb # [ 0.397494] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1130worker2 # [ 0.376562] PCI host bridge to bus 0000:001131lb # [ 0.398509] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1132worker2 # [ 0.377202] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1133lb # [ 0.399494] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1134client # [ 0.403715] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1135worker2 # [ 0.378199] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1136client # [ 0.405345] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1137worker2 # [ 0.379199] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1138worker1 # [ 0.385232] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1139lb # [ 0.401545] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1140client # [ 0.405729] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1141worker2 # [ 0.380199] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1142worker1 # [ 0.386148] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1143client # [ 0.407256] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1144worker2 # [ 0.381198] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1145worker1 # [ 0.387213] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1146lb # [ 0.403545] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1147worker2 # [ 0.382199] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1148lb # [ 0.404405] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1149worker1 # [ 0.388363] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1150worker2 # [ 0.383199] pci_bus 0000:00: root bus resource [bus 00-ff]1151lb # [ 0.405191] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1152client # [ 0.409658] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1153worker2 # [ 0.384157] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1154client # [ 0.410358] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1155worker1 # [ 0.390105] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156lb # [ 0.406084] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1157client # [ 0.410729] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1158worker2 # [ 0.385635] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1159client # [ 0.411712] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1160worker1 # [ 0.393176] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1161worker1 # [ 0.394079] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1162client # [ 0.413253] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1163lb # [ 0.408496] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1164worker1 # [ 0.394927] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1165lb # [ 0.410191] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1166worker1 # [ 0.396001] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1167client # [ 0.414742] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1168lb # [ 0.410509] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1169client # [ 0.415712] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1170worker1 # [ 0.397945] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171lb # [ 0.412082] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1172worker2 # [ 0.388264] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1173client # [ 0.417323] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1174worker2 # [ 0.389210] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1175worker2 # [ 0.390214] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1176worker1 # [ 0.400142] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1177lb # [ 0.413543] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1178client # [ 0.418719] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1179worker1 # [ 0.400967] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1180lb # [ 0.414394] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1181worker2 # [ 0.391572] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1182worker1 # [ 0.401867] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1183lb # [ 0.415249] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1184client # [ 0.419666] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1185worker1 # [ 0.402719] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1186worker2 # [ 0.392917] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1187lb # [ 0.416351] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1188client # [ 0.421095] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1189lb # [ 0.418035] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1190client # [ 0.422720] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1191worker2 # [ 0.395241] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1192worker2 # [ 0.396129] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1193client # [ 0.423729] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1194worker1 # [ 0.405144] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1195lb # [ 0.419548] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1196worker2 # [ 0.396976] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1197worker1 # [ 0.406790] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1198lb # [ 0.420509] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1199client # [ 0.425274] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1200worker2 # [ 0.398046] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1201worker1 # [ 0.407161] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1202lb # [ 0.422092] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1203worker2 # [ 0.399500] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1204worker1 # [ 0.408689] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1205client # [ 0.426720] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1206lb # [ 0.423501] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1207client # [ 0.427758] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1208worker2 # [ 0.401239] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1209worker1 # [ 0.410143] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1210worker2 # [ 0.402073] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1211lb # [ 0.425136] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1212worker1 # [ 0.410995] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1213client # [ 0.429252] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1214worker2 # [ 0.402914] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1215worker1 # [ 0.411901] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1216lb # [ 0.426119] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1217worker2 # [ 0.403798] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1218worker1 # [ 0.412141] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1219client # [ 0.431564] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1220lb # [ 0.427500] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1221client # [ 0.432340] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1222client # [ 0.432729] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1223lb # [ 0.428503] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1224worker1 # [ 0.413767] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225client # [ 0.434288] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1226lb # [ 0.430058] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1227worker1 # [ 0.415148] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1228worker2 # [ 0.406381] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1229worker2 # [ 0.407206] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1230worker1 # [ 0.416104] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1231worker2 # [ 0.408198] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1232lb # [ 0.431500] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1233worker1 # [ 0.417639] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1234client # [ 0.436742] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1235lb # [ 0.432510] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1236worker2 # [ 0.409754] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1237client # [ 0.437567] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1238client # [ 0.438505] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1239worker1 # [ 0.419192] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1240lb # [ 0.434025] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1241worker2 # [ 0.411241] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1242worker2 # [ 0.412136] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1243worker1 # [ 0.420162] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1244client # [ 0.440087] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1245lb # [ 0.435523] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1246worker2 # [ 0.412937] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1247lb # [ 0.436420] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248worker1 # [ 0.421684] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1249client # [ 0.441501] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1250worker2 # [ 0.413205] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1251lb # [ 0.437192] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1252client # [ 0.442528] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1253worker1 # [ 0.423179] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1254worker2 # [ 0.414738] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1255lb # [ 0.438068] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1256worker1 # [ 0.424135] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1257client # [ 0.444457] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1258worker2 # [ 0.416125] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1259worker1 # [ 0.425682] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1260client # [ 0.444901] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1261worker2 # [ 0.416944] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1262lb # [ 0.440518] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1263lb # [ 0.441391] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1264client # [ 0.446555] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1265worker1 # [ 0.427181] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1266worker2 # [ 0.417801] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1267lb # [ 0.442232] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1268worker1 # [ 0.428118] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1269worker2 # [ 0.419244] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1270lb # [ 0.443057] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1271worker1 # [ 0.429586] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1272worker2 # [ 0.420204] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1273lb # [ 0.444152] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1274lb # [ 0.444684] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1275worker2 # [ 0.421818] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1276worker1 # [ 0.431189] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1277worker1 # [ 0.432120] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1278worker2 # [ 0.423097] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1279lb # [ 0.446106] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1280worker1 # [ 0.432940] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1281worker2 # [ 0.424039] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1282lb # [ 0.446691] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1283worker1 # [ 0.434538] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1284worker2 # [ 0.425617] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1285lb # [ 0.448204] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1286client # [ 0.447632] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1287worker2 # [ 0.427212] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1288worker1 # [ 0.437176] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1289client # [ 0.449434] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1290worker1 # [ 0.438141] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1291worker2 # [ 0.428204] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1292worker1 # [ 0.439104] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1293client # [ 0.449942] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1294worker2 # [ 0.429772] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1295client # [ 0.450998] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1296worker1 # [ 0.440546] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1297worker2 # [ 0.431224] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1298worker1 # [ 0.441755] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1299client # [ 0.451965] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1300worker2 # [ 0.432095] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1301worker1 # [ 0.442341] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1302worker2 # [ 0.432982] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1303client # [ 0.453749] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1304client # [ 0.454590] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1305worker1 # [ 0.443780] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1306worker2 # [ 0.434662] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1307client # [ 0.455734] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1308lb # [ 0.448706] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1309worker1 # [ 0.444352] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1310client # [ 0.457430] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1311lb # [ 0.451126] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1312worker1 # [ 0.445820] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1313worker2 # [ 0.437016] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1314lb # [ 0.451792] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1315worker2 # [ 0.437868] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1316worker2 # [ 0.438220] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1317lb # [ 0.452802] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1318client # [ 0.461296] ACPI: PCI: Interrupt link LNKA configured for IRQ 101319worker2 # [ 0.439773] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1320lb # [ 0.453735] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1321client # [ 0.461813] ACPI: PCI: Interrupt link LNKB configured for IRQ 101322worker2 # [ 0.440750] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1323client # [ 0.462806] ACPI: PCI: Interrupt link LNKC configured for IRQ 111324lb # [ 0.455510] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1325client # [ 0.463805] ACPI: PCI: Interrupt link LNKD configured for IRQ 111326worker2 # [ 0.441405] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1327lb # [ 0.456381] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1328client # [ 0.464805] ACPI: PCI: Interrupt link LNKE configured for IRQ 101329worker2 # [ 0.442740] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1330lb # [ 0.457547] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1331client # [ 0.465819] ACPI: PCI: Interrupt link LNKF configured for IRQ 101332client # [ 0.466808] ACPI: PCI: Interrupt link LNKG configured for IRQ 111333lb # [ 0.459170] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1334worker2 # [ 0.443400] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1335client # [ 0.467804] ACPI: PCI: Interrupt link LNKH configured for IRQ 111336worker2 # [ 0.444752] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1337worker1 # [ 0.446389] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1338client # [ 0.468741] ACPI: PCI: Interrupt link GSIA configured for IRQ 161339client # [ 0.469725] ACPI: PCI: Interrupt link GSIB configured for IRQ 171340worker1 # [ 0.448747] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1341client # [ 0.470721] ACPI: PCI: Interrupt link GSIC configured for IRQ 181342lb # [ 0.462950] ACPI: PCI: Interrupt link LNKA configured for IRQ 101343worker1 # [ 0.449403] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1344client # [ 0.471696] ACPI: PCI: Interrupt link GSID configured for IRQ 191345lb # [ 0.463593] ACPI: PCI: Interrupt link LNKB configured for IRQ 101346client # [ 0.472467] ACPI: PCI: Interrupt link GSIE configured for IRQ 201347lb # [ 0.464602] ACPI: PCI: Interrupt link LNKC configured for IRQ 111348worker1 # [ 0.450421] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1349client # [ 0.472721] ACPI: PCI: Interrupt link GSIF configured for IRQ 211350lb # [ 0.465602] ACPI: PCI: Interrupt link LNKD configured for IRQ 111351client # [ 0.473686] ACPI: PCI: Interrupt link GSIG configured for IRQ 221352worker1 # [ 0.451399] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1353lb # [ 0.466585] ACPI: PCI: Interrupt link LNKE configured for IRQ 101354client # [ 0.474441] ACPI: PCI: Interrupt link GSIH configured for IRQ 231355lb # [ 0.467583] ACPI: PCI: Interrupt link LNKF configured for IRQ 101356worker1 # [ 0.453158] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1357lb # [ 0.468589] ACPI: PCI: Interrupt link LNKG configured for IRQ 111358client # [ 0.475751] iommu: Default domain type: Translated1359worker1 # [ 0.454039] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1360lb # [ 0.469587] ACPI: PCI: Interrupt link LNKH configured for IRQ 111361client # [ 0.476534] iommu: DMA domain TLB invalidation policy: lazy mode1362worker1 # [ 0.455150] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1363client # [ 0.477697] ACPI: bus type USB registered1364lb # [ 0.470524] ACPI: PCI: Interrupt link GSIA configured for IRQ 161365client # [ 0.478427] usbcore: registered new interface driver usbfs1366lb # [ 0.471505] ACPI: PCI: Interrupt link GSIB configured for IRQ 171367worker2 # [ 0.445412] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1368worker1 # [ 0.456826] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1369client # [ 0.478723] usbcore: registered new interface driver hub1370lb # [ 0.472492] ACPI: PCI: Interrupt link GSIC configured for IRQ 181371client # [ 0.479560] usbcore: registered new device driver usb1372worker2 # [ 0.447689] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1373lb # [ 0.473464] ACPI: PCI: Interrupt link GSID configured for IRQ 191374client # [ 0.481141] NetLabel: Initializing1375lb # [ 0.474276] ACPI: PCI: Interrupt link GSIE configured for IRQ 201376worker2 # [ 0.448484] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1377client # [ 0.481708] NetLabel: domain hash size = 1281378lb # [ 0.475212] ACPI: PCI: Interrupt link GSIF configured for IRQ 211379client # [ 0.482437] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1380worker2 # [ 0.449488] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1381worker1 # [ 0.460716] ACPI: PCI: Interrupt link LNKA configured for IRQ 101382lb # [ 0.475502] ACPI: PCI: Interrupt link GSIG configured for IRQ 221383client # [ 0.482739] NetLabel: unlabeled traffic allowed by default1384lb # [ 0.476431] ACPI: PCI: Interrupt link GSIH configured for IRQ 231385worker1 # [ 0.461252] ACPI: PCI: Interrupt link LNKB configured for IRQ 101386client # [ 0.483605] PCI: Using ACPI for IRQ routing1387worker2 # [ 0.450454] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1388worker1 # [ 0.462235] ACPI: PCI: Interrupt link LNKC configured for IRQ 111389lb # [ 0.478224] iommu: Default domain type: Translated1390worker1 # [ 0.463231] ACPI: PCI: Interrupt link LNKD configured for IRQ 111391worker2 # [ 0.452107] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1392lb # [ 0.478495] iommu: DMA domain TLB invalidation policy: lazy mode1393worker1 # [ 0.464231] ACPI: PCI: Interrupt link LNKE configured for IRQ 101394worker2 # [ 0.452865] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1395lb # [ 0.479748] ACPI: bus type USB registered1396worker1 # [ 0.465234] ACPI: PCI: Interrupt link LNKF configured for IRQ 101397lb # [ 0.480531] usbcore: registered new interface driver usbfs1398worker2 # [ 0.453517] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1399worker1 # [ 0.466234] ACPI: PCI: Interrupt link LNKG configured for IRQ 111400lb # [ 0.481425] usbcore: registered new interface driver hub1401lb # [ 0.482119] usbcore: registered new device driver usb1402worker1 # [ 0.467241] ACPI: PCI: Interrupt link LNKH configured for IRQ 111403worker2 # [ 0.455033] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1404worker1 # [ 0.468171] ACPI: PCI: Interrupt link GSIA configured for IRQ 161405lb # [ 0.483303] NetLabel: Initializing1406lb # [ 0.483963] NetLabel: domain hash size = 1281407worker1 # [ 0.469149] ACPI: PCI: Interrupt link GSIB configured for IRQ 171408lb # [ 0.484487] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1409worker1 # [ 0.470152] ACPI: PCI: Interrupt link GSIC configured for IRQ 181410lb # [ 0.485468] NetLabel: unlabeled traffic allowed by default1411worker1 # [ 0.471149] ACPI: PCI: Interrupt link GSID configured for IRQ 191412lb # [ 0.486165] PCI: Using ACPI for IRQ routing1413worker2 # [ 0.458921] ACPI: PCI: Interrupt link LNKA configured for IRQ 101414worker1 # [ 0.472149] ACPI: PCI: Interrupt link GSIE configured for IRQ 201415worker2 # [ 0.459306] ACPI: PCI: Interrupt link LNKB configured for IRQ 101416worker1 # [ 0.473159] ACPI: PCI: Interrupt link GSIF configured for IRQ 211417worker2 # [ 0.460295] ACPI: PCI: Interrupt link LNKC configured for IRQ 111418worker1 # [ 0.474147] ACPI: PCI: Interrupt link GSIG configured for IRQ 221419worker2 # [ 0.461300] ACPI: PCI: Interrupt link LNKD configured for IRQ 111420worker1 # [ 0.475147] ACPI: PCI: Interrupt link GSIH configured for IRQ 231421worker2 # [ 0.462295] ACPI: PCI: Interrupt link LNKE configured for IRQ 101422worker1 # [ 0.477183] iommu: Default domain type: Translated1423worker2 # [ 0.463297] ACPI: PCI: Interrupt link LNKF configured for IRQ 101424worker1 # [ 0.478037] iommu: DMA domain TLB invalidation policy: lazy mode1425worker2 # [ 0.464299] ACPI: PCI: Interrupt link LNKG configured for IRQ 111426worker1 # [ 0.479188] ACPI: bus type USB registered1427worker2 # [ 0.465315] ACPI: PCI: Interrupt link LNKH configured for IRQ 111428worker1 # [ 0.479954] usbcore: registered new interface driver usbfs1429worker2 # [ 0.466237] ACPI: PCI: Interrupt link GSIA configured for IRQ 161430worker1 # [ 0.480846] usbcore: registered new interface driver hub1431worker2 # [ 0.467211] ACPI: PCI: Interrupt link GSIB configured for IRQ 171432worker1 # [ 0.481145] usbcore: registered new device driver usb1433worker2 # [ 0.468214] ACPI: PCI: Interrupt link GSIC configured for IRQ 181434worker1 # [ 0.482893] NetLabel: Initializing1435worker2 # [ 0.469214] ACPI: PCI: Interrupt link GSID configured for IRQ 191436worker1 # [ 0.483141] NetLabel: domain hash size = 1281437worker2 # [ 0.470211] ACPI: PCI: Interrupt link GSIE configured for IRQ 201438worker1 # [ 0.483897] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439worker2 # [ 0.471212] ACPI: PCI: Interrupt link GSIF configured for IRQ 211440worker1 # [ 0.484172] NetLabel: unlabeled traffic allowed by default1441worker1 # [ 0.485094] PCI: Using ACPI for IRQ routing1442worker2 # [ 0.472215] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443worker2 # [ 0.473211] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker2 # [ 0.475262] iommu: Default domain type: Translated1445worker2 # [ 0.476112] iommu: DMA domain TLB invalidation policy: lazy mode1446worker2 # [ 0.477327] ACPI: bus type USB registered1447worker2 # [ 0.478116] usbcore: registered new interface driver usbfs1448worker2 # [ 0.478947] usbcore: registered new interface driver hub1449worker2 # [ 0.479208] usbcore: registered new device driver usb1450worker2 # [ 0.481136] NetLabel: Initializing1451worker2 # [ 0.481707] NetLabel: domain hash size = 1281452worker2 # [ 0.482199] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker2 # [ 0.483310] NetLabel: unlabeled traffic allowed by default1454worker2 # [ 0.484214] PCI: Using ACPI for IRQ routing1455client # [ 0.527391] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.527703] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.527703] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.527712] vgaarb: loaded1459client # [ 0.528463] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.528707] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461client # [ 0.532793] clocksource: Switched to clocksource kvm-clock1462lb # [ 0.529688] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463lb # [ 0.530483] pci 0000:00:01.0: vgaarb: bridge control possible1464client # [ 0.536245] VFS: Disk quotas dquot_6.6.01465client # [ 0.536948] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1466lb # [ 0.530483] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1467lb # [ 0.530492] vgaarb: loaded1468client # [ 0.538296] pnp: PnP ACPI init1469lb # [ 0.531252] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470client # [ 0.539076] ACPI: IRQ 4 override to edge(!), high(!)1471lb # [ 0.531490] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1472client # [ 0.540027] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1473client # [ 0.541573] pnp: PnP ACPI: found 5 devices1474lb # [ 0.534577] clocksource: Switched to clocksource kvm-clock1475lb # [ 0.538171] VFS: Disk quotas dquot_6.6.01476lb # [ 0.538911] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1477lb # [ 0.540346] pnp: PnP ACPI init1478client # [ 0.549192] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1479worker1 # [ 0.528387] pci 0000:00:01.0: vgaarb: setting as boot VGA device1480lb # [ 0.541148] ACPI: IRQ 4 override to edge(!), high(!)1481client # [ 0.550628] clocksource: Switched to clocksource acpi_pm1482worker1 # [ 0.529131] pci 0000:00:01.0: vgaarb: bridge control possible1483client # [ 0.551581] NET: Registered PF_INET protocol family1484lb # [ 0.542071] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1485worker1 # [ 0.529131] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1486worker1 # [ 0.529139] vgaarb: loaded1487lb # [ 0.543574] pnp: PnP ACPI: found 5 devices1488client # [ 0.552891] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1489worker1 # [ 0.529899] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01490worker1 # [ 0.530136] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1491worker1 # [ 0.534219] clocksource: Switched to clocksource kvm-clock1492lb # [ 0.551183] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1493lb # [ 0.552674] clocksource: Switched to clocksource acpi_pm1494lb # [ 0.553683] NET: Registered PF_INET protocol family1495worker1 # [ 0.537992] VFS: Disk quotas dquot_6.6.01496worker2 # [ 0.527721] pci 0000:00:01.0: vgaarb: setting as boot VGA device1497worker2 # [ 0.528195] pci 0000:00:01.0: vgaarb: bridge control possible1498worker1 # [ 0.538709] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1499lb # [ 0.554870] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1500worker1 # [ 0.540047] pnp: PnP ACPI init1501worker2 # [ 0.528195] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1502worker2 # [ 0.528203] vgaarb: loaded1503worker1 # [ 0.540849] ACPI: IRQ 4 override to edge(!), high(!)1504worker2 # [ 0.528951] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01505worker1 # [ 0.541786] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1506worker2 # [ 0.529201] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1507worker1 # [ 0.543285] pnp: PnP ACPI: found 5 devices1508client # [ 0.568906] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1509client # [ 0.570465] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1510client # [ 0.571821] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1511worker2 # [ 0.532285] clocksource: Switched to clocksource kvm-clock1512client # [ 0.573167] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1513worker2 # [ 0.534198] VFS: Disk quotas dquot_6.6.01514client # [ 0.574572] TCP: Hash tables configured (established 16384 bind 16384)1515client # [ 0.575862] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1516worker1 # [ 0.550869] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1517client # [ 0.577334] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1518worker2 # [ 0.537150] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1519worker1 # [ 0.552508] clocksource: Switched to clocksource acpi_pm1520worker2 # [ 0.538684] pnp: PnP ACPI init1521client # [ 0.578556] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1522worker1 # [ 0.553666] NET: Registered PF_INET protocol family1523worker2 # [ 0.539565] ACPI: IRQ 4 override to edge(!), high(!)1524client # [ 0.579923] NET: Registered PF_UNIX/PF_LOCAL protocol family1525client # [ 0.581029] NET: Registered PF_XDP protocol family1526worker1 # [ 0.555033] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1527worker2 # [ 0.540695] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1528client # [ 0.581963] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1529lb # [ 0.572351] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1530worker2 # [ 0.542395] pnp: PnP ACPI: found 5 devices1531client # [ 0.583129] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1532lb # [ 0.574162] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1533client # [ 0.584365] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1534lb # [ 0.575694] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1535client # [ 0.585651] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1536lb # [ 0.577277] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1537client # [ 0.586957] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1538lb # [ 0.578720] TCP: Hash tables configured (established 16384 bind 16384)1539client # [ 0.588218] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1540lb # [ 0.579962] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1541client # [ 0.590105] ACPI: \_SB_.GSIA: Enabled at IRQ 161542lb # [ 0.581367] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1543worker2 # [ 0.550375] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1544lb # [ 0.582566] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1545worker2 # [ 0.551963] clocksource: Switched to clocksource acpi_pm1546client # [ 0.592833] ACPI: \_SB_.GSIB: Enabled at IRQ 171547lb # [ 0.583882] NET: Registered PF_UNIX/PF_LOCAL protocol family1548worker2 # [ 0.553034] NET: Registered PF_INET protocol family1549lb # [ 0.584880] NET: Registered PF_XDP protocol family1550client # [ 0.594746] ACPI: \_SB_.GSIC: Enabled at IRQ 181551worker2 # [ 0.554314] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1552lb # [ 0.585761] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1553lb # [ 0.586751] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554client # [ 0.596668] ACPI: \_SB_.GSID: Enabled at IRQ 191555lb # [ 0.587790] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1556client # [ 0.598349] PCI: CLS 0 bytes, default 641557lb # [ 0.588900] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1558worker1 # [ 0.572558] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1559lb # [ 0.589967] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1560client # [ 0.599184] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1561worker1 # [ 0.574079] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1562lb # [ 0.591033] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563client # [ 0.600873] Trying to unpack rootfs image as initramfs...1564worker1 # [ 0.575325] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1565lb # [ 0.592740] ACPI: \_SB_.GSIA: Enabled at IRQ 161566worker1 # [ 0.576631] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1567worker1 # [ 0.577893] TCP: Hash tables configured (established 16384 bind 16384)1568lb # [ 0.595332] ACPI: \_SB_.GSIB: Enabled at IRQ 171569worker1 # [ 0.579060] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1570worker1 # [ 0.580336] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1571lb # [ 0.597345] ACPI: \_SB_.GSIC: Enabled at IRQ 181572worker1 # [ 0.581463] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1573worker1 # [ 0.582706] NET: Registered PF_UNIX/PF_LOCAL protocol family1574lb # [ 0.599317] ACPI: \_SB_.GSID: Enabled at IRQ 191575worker1 # [ 0.583705] NET: Registered PF_XDP protocol family1576lb # [ 0.600969] PCI: CLS 0 bytes, default 641577worker1 # [ 0.584565] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1578worker1 # [ 0.585561] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1579worker2 # [ 0.570945] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1580lb # [ 0.601855] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1581worker1 # [ 0.586538] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1582lb # [ 0.603596] Trying to unpack rootfs image as initramfs...1583worker2 # [ 0.572486] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1584worker1 # [ 0.587638] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1585worker1 # [ 0.588716] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1586worker2 # [ 0.573845] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1587worker1 # [ 0.589819] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1588worker2 # [ 0.575246] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1589worker2 # [ 0.576525] TCP: Hash tables configured (established 16384 bind 16384)1590worker1 # [ 0.591512] ACPI: \_SB_.GSIA: Enabled at IRQ 161591worker2 # [ 0.577682] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1592worker2 # [ 0.579030] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1593worker1 # [ 0.594389] ACPI: \_SB_.GSIB: Enabled at IRQ 171594worker2 # [ 0.580294] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1595worker2 # [ 0.581582] NET: Registered PF_UNIX/PF_LOCAL protocol family1596worker1 # [ 0.596531] ACPI: \_SB_.GSIC: Enabled at IRQ 181597worker2 # [ 0.582594] NET: Registered PF_XDP protocol family1598worker2 # [ 0.583528] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1599worker1 # [ 0.598793] ACPI: \_SB_.GSID: Enabled at IRQ 191600worker2 # [ 0.584598] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1601worker1 # [ 0.600458] PCI: CLS 0 bytes, default 641602worker2 # [ 0.585656] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1603worker2 # [ 0.586825] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1604worker1 # [ 0.601349] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1605worker2 # [ 0.587962] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1606worker1 # [ 0.603112] Trying to unpack rootfs image as initramfs...1607worker2 # [ 0.589087] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker2 # [ 0.590900] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker2 # [ 0.593552] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker2 # [ 0.595591] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker2 # [ 0.597520] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker2 # [ 0.599173] PCI: CLS 0 bytes, default 641613worker2 # [ 0.600082] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614worker2 # [ 0.601902] Trying to unpack rootfs image as initramfs...1615client # [ 0.643864] Initialise system trusted keyrings1616client # [ 0.647317] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.648559] Initialise system trusted keyrings1618lb # [ 0.652197] workingset: timestamp_bits=40 max_order=19 bucket_order=01619client # [ 0.670413] Key type asymmetric registered1620client # [ 0.671144] Asymmetric key parser 'x509' registered1621client # [ 0.672061] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1622worker1 # [ 0.647732] Initialise system trusted keyrings1623client # [ 0.675383] io scheduler mq-deadline registered1624client # [ 0.676187] io scheduler kyber registered1625worker1 # [ 0.650679] workingset: timestamp_bits=40 max_order=19 bucket_order=01626client # [ 0.679888] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1627lb # [ 0.671562] Key type asymmetric registered1628client # [ 0.681209] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1629lb # [ 0.675172] Asymmetric key parser 'x509' registered1630client # [ 0.685026] Linux agpgart interface v0.1031631worker2 # [ 0.645457] Initialise system trusted keyrings1632client # [ 0.685790] ACPI: bus type drm_connector registered1633lb # [ 0.676040] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1634lb # [ 0.679292] io scheduler mq-deadline registered1635client # [ 0.688703] usbcore: registered new interface driver usbserial_generic1636lb # [ 0.680058] io scheduler kyber registered1637worker2 # [ 0.649392] workingset: timestamp_bits=40 max_order=19 bucket_order=01638client # [ 0.689785] usbserial: USB Serial support registered for generic1639lb # [ 0.682710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1640lb # [ 0.684025] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1641client # [ 0.693269] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1642client # [ 0.693269] Please enable it if your BIOS has the CPPC option.1643client # [ 0.695543] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1644worker1 # [ 0.672094] Key type asymmetric registered1645worker1 # [ 0.672864] Asymmetric key parser 'x509' registered1646client # [ 0.698388] drop_monitor: Initializing network drop monitor service1647lb # [ 0.689777] Linux agpgart interface v0.1031648client # [ 0.699565] NET: Registered PF_INET6 protocol family1649lb # [ 0.690542] ACPI: bus type drm_connector registered1650worker1 # [ 0.675675] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1651lb # [ 0.693582] usbcore: registered new interface driver usbserial_generic1652lb # [ 0.694726] usbserial: USB Serial support registered for generic1653worker1 # [ 0.678664] io scheduler mq-deadline registered1654client # [ 0.704715] Segment Routing with IPv61655worker1 # [ 0.679480] io scheduler kyber registered1656lb # [ 0.697160] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1657lb # [ 0.697160] Please enable it if your BIOS has the CPPC option.1658worker1 # [ 0.682250] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1659client # [ 0.708289] In-situ OAM (IOAM) with IPv61660lb # [ 0.699383] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1661client # [ 0.709397] IPI shorthand broadcast: enabled1662worker1 # [ 0.683500] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1663worker2 # [ 0.671547] Key type asymmetric registered1664lb # [ 0.702273] drop_monitor: Initializing network drop monitor service1665worker2 # [ 0.672287] Asymmetric key parser 'x509' registered1666lb # [ 0.703518] NET: Registered PF_INET6 protocol family1667worker1 # [ 0.687387] Linux agpgart interface v0.1031668worker1 # [ 0.688309] ACPI: bus type drm_connector registered1669worker2 # [ 0.673176] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1670worker1 # [ 0.691114] usbcore: registered new interface driver usbserial_generic1671lb # [ 0.708618] Segment Routing with IPv61672worker1 # [ 0.692185] usbserial: USB Serial support registered for generic1673client # [ 0.717695] sched_clock: Marking stable (582015019, 135232824)->(792877066, -75629223)1674lb # [ 0.709383] In-situ OAM (IOAM) with IPv61675worker2 # [ 0.678392] io scheduler mq-deadline registered1676worker2 # [ 0.679149] io scheduler kyber registered1677worker1 # [ 0.694636] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1678worker1 # [ 0.694636] Please enable it if your BIOS has the CPPC option.1679lb # [ 0.712437] IPI shorthand broadcast: enabled1680worker2 # [ 0.681847] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1681worker1 # [ 0.696914] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1682client # [ 0.723447] registered taskstats version 11683worker2 # [ 0.683121] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1684client # [ 0.724433] Loading compiled-in X.509 certificates1685worker2 # [ 0.688390] Linux agpgart interface v0.1031686worker1 # [ 0.702791] drop_monitor: Initializing network drop monitor service1687worker2 # [ 0.689114] ACPI: bus type drm_connector registered1688worker1 # [ 0.703958] NET: Registered PF_INET6 protocol family1689lb # [ 0.720798] sched_clock: Marking stable (584015064, 136116829)->(793771741, -73639848)1690worker1 # [ 0.707104] Segment Routing with IPv61691worker2 # [ 0.692365] usbcore: registered new interface driver usbserial_generic1692worker1 # [ 0.707802] In-situ OAM (IOAM) with IPv61693lb # [ 0.724346] registered taskstats version 11694worker2 # [ 0.693431] usbserial: USB Serial support registered for generic1695lb # [ 0.725360] Loading compiled-in X.509 certificates1696worker1 # [ 0.710913] IPI shorthand broadcast: enabled1697worker2 # [ 0.696243] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1698worker2 # [ 0.696243] Please enable it if your BIOS has the CPPC option.1699worker2 # [ 0.698463] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1700client # [ 0.740800] Demotion targets for Node 0: null1701worker2 # [ 0.701413] drop_monitor: Initializing network drop monitor service1702worker2 # [ 0.702577] NET: Registered PF_INET6 protocol family1703client # [ 0.744295] Key type .fscrypt registered1704client # [ 0.745021] Key type fscrypt-provisioning registered1705worker1 # [ 0.719364] sched_clock: Marking stable (583016345, 135597302)->(792277579, -73663932)1706client # [ 0.745959] ima: No TPM chip found, activating TPM-bypass!1707worker2 # [ 0.707344] Segment Routing with IPv61708worker2 # [ 0.707984] In-situ OAM (IOAM) with IPv61709client # [ 0.748259] ima: Allocated hash algorithm: sha11710worker1 # [ 0.722818] registered taskstats version 11711client # [ 0.749033] ima: No architecture policies found1712worker2 # [ 0.709066] IPI shorthand broadcast: enabled1713worker1 # [ 0.723802] Loading compiled-in X.509 certificates1714client # [ 0.751259] PM: Magic number: 10:852:7171715client # [ 0.752888] RAS: Correctable Errors collector initialized.1716lb # [ 0.744144] Demotion targets for Node 0: null1717lb # [ 0.745059] Key type .fscrypt registered1718lb # [ 0.747142] Key type fscrypt-provisioning registered1719lb # [ 0.748137] ima: No TPM chip found, activating TPM-bypass!1720worker2 # [ 0.717543] sched_clock: Marking stable (583014835, 134204204)->(791666212, -74447173)1721lb # [ 0.749056] ima: Allocated hash algorithm: sha11722client # [ 0.761927] clk: Disabling unused clocks1723lb # [ 0.753170] ima: No architecture policies found1724worker2 # [ 0.722478] registered taskstats version 11725lb # [ 0.754201] PM: Magic number: 10:852:7171726worker2 # [ 0.723421] Loading compiled-in X.509 certificates1727lb # [ 0.755854] RAS: Correctable Errors collector initialized.1728client # [ 0.766267] PM: genpd: Disabling unused power domains1729worker1 # [ 0.742628] Demotion targets for Node 0: null1730worker1 # [ 0.743511] Key type .fscrypt registered1731worker1 # [ 0.746618] Key type fscrypt-provisioning registered1732worker1 # [ 0.747643] ima: No TPM chip found, activating TPM-bypass!1733lb # [ 0.764851] clk: Disabling unused clocks1734worker1 # [ 0.748626] ima: Allocated hash algorithm: sha11735worker1 # [ 0.749386] ima: No architecture policies found1736lb # [ 0.768162] PM: genpd: Disabling unused power domains1737worker1 # [ 0.753623] PM: Magic number: 10:852:7171738worker1 # [ 0.755327] RAS: Correctable Errors collector initialized.1739worker2 # [ 0.741233] Demotion targets for Node 0: null1740worker2 # [ 0.742086] Key type .fscrypt registered1741worker2 # [ 0.745224] Key type fscrypt-provisioning registered1742worker2 # [ 0.746270] ima: No TPM chip found, activating TPM-bypass!1743worker2 # [ 0.747196] ima: Allocated hash algorithm: sha11744worker2 # [ 0.749250] ima: No architecture policies found1745worker1 # [ 0.764347] clk: Disabling unused clocks1746worker2 # [ 0.752278] PM: Magic number: 10:852:7171747worker1 # [ 0.766643] PM: genpd: Disabling unused power domains1748worker2 # [ 0.754057] RAS: Correctable Errors collector initialized.1749worker2 # [ 0.763019] clk: Disabling unused clocks1750worker2 # [ 0.765239] PM: genpd: Disabling unused power domains1751lb # [ 0.936608] Freeing initrd memory: 29064K1752client # [ 0.947045] Freeing initrd memory: 29056K1753lb # [ 0.939859] Freeing unused decrypted memory: 2028K1754client # [ 0.950714] Freeing unused decrypted memory: 2028K1755lb # [ 0.942606] Freeing unused kernel image (initmem) memory: 3652K1756lb # [ 0.943711] Write protecting the kernel read-only data: 32768k1757client # [ 0.953709] Freeing unused kernel image (initmem) memory: 3652K1758client # [ 0.954862] Write protecting the kernel read-only data: 32768k1759lb # [ 0.945650] Freeing unused kernel image (text/rodata gap) memory: 1184K1760lb # [ 0.947189] Freeing unused kernel image (rodata/data gap) memory: 720K1761client # [ 0.956858] Freeing unused kernel image (text/rodata gap) memory: 1184K1762client # [ 0.958489] Freeing unused kernel image (rodata/data gap) memory: 720K1763worker1 # [ 0.934332] Freeing initrd memory: 29056K1764worker1 # [ 0.937533] Freeing unused decrypted memory: 2028K1765worker1 # [ 0.940227] Freeing unused kernel image (initmem) memory: 3652K1766worker1 # [ 0.941341] Write protecting the kernel read-only data: 32768k1767worker1 # [ 0.943223] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker1 # [ 0.944765] Freeing unused kernel image (rodata/data gap) memory: 720K1769worker2 # [ 0.931437] Freeing initrd memory: 29060K1770worker2 # [ 0.934668] Freeing unused decrypted memory: 2028K1771worker2 # [ 0.937425] Freeing unused kernel image (initmem) memory: 3652K1772worker2 # [ 0.938558] Write protecting the kernel read-only data: 32768k1773worker2 # [ 0.940455] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker2 # [ 0.942028] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 0.997983] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 0.999093] Run /init as init process1777client # [ 1.009355] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778client # [ 1.010549] Run /init as init process1779lb # [ 1.009837] systemd[1]: Inserted module 'autofs4'1780worker1 # [ 0.995614] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781worker1 # [ 0.996737] Run /init as init process1782client # [ 1.022928] systemd[1]: Inserted module 'autofs4'1783worker1 # [ 1.007503] systemd[1]: Inserted module 'autofs4'1784worker2 # [ 0.993334] x86/mm: Checked W+X mappings: passed, no W+X pages found.1785worker2 # [ 0.994447] Run /init as init process1786lb # [ 1.029683] fuse: init (API version 7.45)1787client # [ 1.044421] fuse: init (API version 7.45)1788worker2 # [ 1.005019] systemd[1]: Inserted module 'autofs4'1789lb # [ 1.037075] ACPI: \_SB_.GSIG: Enabled at IRQ 221790lb # [ 1.039706] ACPI: \_SB_.GSIH: Enabled at IRQ 231791client # [ 1.052202] ACPI: \_SB_.GSIG: Enabled at IRQ 221792worker1 # [ 1.027280] fuse: init (API version 7.45)1793lb # [ 1.043690] ACPI: \_SB_.GSIE: Enabled at IRQ 201794client # [ 1.054928] ACPI: \_SB_.GSIH: Enabled at IRQ 231795lb # [ 1.045720] ACPI: \_SB_.GSIF: Enabled at IRQ 211796client # [ 1.058501] ACPI: \_SB_.GSIE: Enabled at IRQ 201797worker1 # [ 1.034699] ACPI: \_SB_.GSIG: Enabled at IRQ 221798lb # [ 1.051210] virtiofs virtio5: discovered new tag: nix-store1799client # [ 1.060780] ACPI: \_SB_.GSIF: Enabled at IRQ 211800lb # [ 1.052886] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1801worker1 # [ 1.037288] ACPI: \_SB_.GSIH: Enabled at IRQ 231802worker2 # [ 1.024878] fuse: init (API version 7.45)1803worker1 # [ 1.040486] ACPI: \_SB_.GSIE: Enabled at IRQ 201804client # [ 1.066784] virtiofs virtio5: discovered new tag: nix-store1805worker1 # [ 1.042706] ACPI: \_SB_.GSIF: Enabled at IRQ 211806client # [ 1.068598] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1807lb # [ 1.060874] virtiofs virtio6: discovered new tag: shared1808lb # [ 1.062467] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1809worker2 # [ 1.032382] ACPI: \_SB_.GSIG: Enabled at IRQ 221810worker1 # [ 1.048361] virtiofs virtio5: discovered new tag: nix-store1811worker2 # [ 1.035151] ACPI: \_SB_.GSIH: Enabled at IRQ 231812lb # [ 1.066079] virtiofs virtio7: discovered new tag: xchg1813worker1 # [ 1.049991] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1814client # [ 1.076153] virtiofs virtio6: discovered new tag: shared1815lb # [ 1.067803] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1816client # [ 1.077700] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.038540] ACPI: \_SB_.GSIE: Enabled at IRQ 201818worker2 # [ 1.040682] ACPI: \_SB_.GSIF: Enabled at IRQ 211819client # [ 1.081177] virtiofs virtio7: discovered new tag: xchg1820client # [ 1.082673] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1821worker1 # [ 1.057433] virtiofs virtio6: discovered new tag: shared1822worker1 # [ 1.058935] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823worker2 # [ 1.046243] virtiofs virtio5: discovered new tag: nix-store1824worker1 # [ 1.062463] virtiofs virtio7: discovered new tag: xchg1825worker2 # [ 1.047848] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1826worker1 # [ 1.064103] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1827worker2 # [ 1.055232] virtiofs virtio6: discovered new tag: shared1828worker2 # [ 1.056765] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1829lb # [ 1.090910] systemd[1]: Successfully made /usr/ read-only.1830worker2 # [ 1.061866] virtiofs virtio7: discovered new tag: xchg1831worker2 # [ 1.063808] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832client # [ 1.106088] systemd[1]: Successfully made /usr/ read-only.1833worker1 # [ 1.086406] systemd[1]: Successfully made /usr/ read-only.1834worker2 # [ 1.089612] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.441642] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.446678] systemd[1]: Detected virtualization kvm.1837client # [ 1.447586] systemd[1]: Detected architecture x86-64.1838client # [ 1.448538] systemd[1]: Running in initrd.1839lb # [ 1.427737] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840client # [ 1.449655] systemd[1]: Initializing machine ID from random generator.1841lb # [ 1.440306] systemd[1]: Detected virtualization kvm.1842client # [ 1.450909] systemd[1]: Hostname set to <client>.1843lb # [ 1.442492] systemd[1]: Detected architecture x86-64.1844lb # [ 1.444606] systemd[1]: Running in initrd.1845lb # [ 1.447072] systemd[1]: Initializing machine ID from random generator.1846worker1 # [ 1.422998] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1847lb # [ 1.449972] systemd[1]: Hostname set to <lb>.1848worker1 # [ 1.435106] systemd[1]: Detected virtualization kvm.1849worker1 # [ 1.437269] systemd[1]: Detected architecture x86-64.1850worker1 # [ 1.439421] systemd[1]: Running in initrd.1851worker1 # [ 1.441912] systemd[1]: Initializing machine ID from random generator.1852worker1 # [ 1.444854] systemd[1]: Hostname set to <worker1>.1853worker2 # [ 1.425853] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker2 # [ 1.432410] systemd[1]: Detected virtualization kvm.1855worker2 # [ 1.433549] systemd[1]: Detected architecture x86-64.1856worker2 # [ 1.434707] systemd[1]: Running in initrd.1857worker2 # [ 1.436056] systemd[1]: Initializing machine ID from random generator.1858worker2 # [ 1.437659] systemd[1]: Hostname set to <worker2>.1859client # [ 1.652774] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.663512] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker2 # [ 1.636615] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker1 # [ 1.667920] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.706574] systemd[1]: Queued start job for default target Initrd Default Target.1864client # [ 1.710536] systemd[1]: Created slice Slice /system/modprobe.1865client # [ 1.711755] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866client # [ 1.713180] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867client # [ 1.714369] systemd[1]: Reached target Path Units.1868worker2 # [ 1.674324] systemd[1]: Queued start job for default target Initrd Default Target.1869lb # [ 1.705308] systemd[1]: Queued start job for default target Initrd Default Target.1870client # [ 1.715254] systemd[1]: Reached target Slice Units.1871client # [ 1.716096] systemd[1]: Reached target Swaps.1872client # [ 1.716947] systemd[1]: Reached target Timer Units.1873client # [ 1.717908] systemd[1]: Listening on D-Bus System Message Bus Socket.1874worker2 # [ 1.677518] systemd[1]: Created slice Slice /system/modprobe.1875lb # [ 1.709778] systemd[1]: Created slice Slice /system/modprobe.1876client # [ 1.719107] systemd[1]: Listening on Journal Socket (/dev/log).1877client # [ 1.720310] systemd[1]: Listening on Journal Sockets.1878lb # [ 1.711029] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1879worker2 # [ 1.679894] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1880client # [ 1.721320] systemd[1]: Listening on udev Control Socket.1881lb # [ 1.712436] systemd[1]: Expecting device /dev/disk/by-label/nixos...1882worker2 # [ 1.681829] systemd[1]: Expecting device /dev/disk/by-label/nixos...1883client # [ 1.722361] systemd[1]: Listening on udev Kernel Socket.1884lb # [ 1.713560] systemd[1]: Reached target Path Units.1885worker2 # [ 1.683014] systemd[1]: Reached target Path Units.1886client # [ 1.723347] systemd[1]: Reached target Socket Units.1887lb # [ 1.714422] systemd[1]: Reached target Slice Units.1888worker2 # [ 1.683955] systemd[1]: Reached target Slice Units.1889lb # [ 1.715300] systemd[1]: Reached target Swaps.1890worker2 # [ 1.684903] systemd[1]: Reached target Swaps.1891client # [ 1.725077] systemd[1]: Starting Create List of Static Device Nodes...1892lb # [ 1.716155] systemd[1]: Reached target Timer Units.1893worker2 # [ 1.685716] systemd[1]: Reached target Timer Units.1894lb # [ 1.717108] systemd[1]: Listening on D-Bus System Message Bus Socket.1895worker2 # [ 1.686846] systemd[1]: Listening on D-Bus System Message Bus Socket.1896lb # [ 1.718363] systemd[1]: Listening on Journal Socket (/dev/log).1897worker2 # [ 1.688136] systemd[1]: Listening on Journal Socket (/dev/log).1898lb # [ 1.719572] systemd[1]: Listening on Journal Sockets.1899client # [ 1.729324] systemd[1]: Starting Load Kernel Module configfs...1900worker2 # [ 1.689400] systemd[1]: Listening on Journal Sockets.1901lb # [ 1.720561] systemd[1]: Listening on udev Control Socket.1902worker2 # [ 1.690452] systemd[1]: Listening on udev Control Socket.1903lb # [ 1.721550] systemd[1]: Listening on udev Kernel Socket.1904worker2 # [ 1.691461] systemd[1]: Listening on udev Kernel Socket.1905lb # [ 1.722495] systemd[1]: Reached target Socket Units.1906worker2 # [ 1.692408] systemd[1]: Reached target Socket Units.1907lb # [ 1.724297] systemd[1]: Starting Create List of Static Device Nodes...1908worker2 # [ 1.694607] systemd[1]: Starting Create List of Static Device Nodes...1909lb # [ 1.728203] systemd[1]: Starting Load Kernel Module configfs...1910worker2 # [ 1.699287] systemd[1]: Starting Load Kernel Module configfs...1911worker1 # [ 1.720508] systemd[1]: Queued start job for default target Initrd Default Target.1912client # [ 1.738833] systemd[1]: Starting Journal Service...1913worker1 # [ 1.724944] systemd[1]: Created slice Slice /system/modprobe.1914worker1 # [ 1.726141] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1915worker1 # [ 1.727588] systemd[1]: Expecting device /dev/disk/by-label/nixos...1916worker1 # [ 1.728821] systemd[1]: Reached target Path Units.1917worker1 # [ 1.729760] systemd[1]: Reached target Slice Units.1918worker1 # [ 1.730670] systemd[1]: Reached target Swaps.1919worker1 # [ 1.731463] systemd[1]: Reached target Timer Units.1920lb # [ 1.737200] systemd[1]: Starting Journal Service...1921worker1 # [ 1.732420] systemd[1]: Listening on D-Bus System Message Bus Socket.1922worker1 # [ 1.733625] systemd[1]: Listening on Journal Socket (/dev/log).1923worker2 # [ 1.708413] systemd[1]: Starting Journal Service...1924worker1 # [ 1.734855] systemd[1]: Listening on Journal Sockets.1925worker1 # [ 1.735853] systemd[1]: Listening on udev Control Socket.1926worker1 # [ 1.736941] systemd[1]: Listening on udev Kernel Socket.1927worker1 # [ 1.737949] systemd[1]: Reached target Socket Units.1928client # [ 1.764436] systemd[1]: Starting Load Kernel Modules...1929worker1 # [ 1.739862] systemd[1]: Starting Create List of Static Device Nodes...1930client # [ 1.768381] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1931worker1 # [ 1.743680] systemd[1]: Starting Load Kernel Module configfs...1932lb # [ 1.763360] systemd[1]: Starting Load Kernel Modules...1933worker2 # [ 1.734306] systemd[1]: Starting Load Kernel Modules...1934client # [ 1.775024] systemd[1]: Starting Coldplug All udev Devices...1935lb # [ 1.767232] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1936worker2 # [ 1.738332] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1937lb # [ 1.775318] systemd[1]: Starting Coldplug All udev Devices...1938worker2 # [ 1.746497] systemd[1]: Starting Coldplug All udev Devices...1939client # [ 1.788733] systemd-journald[66]: Collecting audit messages is disabled.1940client # [ 1.790353] systemd[1]: Finished Create List of Static Device Nodes.1941worker1 # [ 1.754688] systemd[1]: Starting Journal Service...1942client # [ 1.793889] systemd[1]: modprobe@configfs.service: Deactivated successfully.1943lb # [ 1.786743] systemd-journald[66]: Collecting audit messages is disabled.1944worker2 # [ 1.755834] systemd-journald[66]: Collecting audit messages is disabled.1945lb # [ 1.791232] systemd[1]: Finished Create List of Static Device Nodes.1946client # [ 1.802642] systemd[1]: Finished Load Kernel Module configfs.1947worker2 # [ 1.763446] systemd[1]: Finished Create List of Static Device Nodes.1948worker1 # [ 1.778805] systemd[1]: Starting Load Kernel Modules...1949lb # [ 1.795741] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker2 # [ 1.765891] systemd[1]: modprobe@configfs.service: Deactivated successfully.1951worker1 # [ 1.782757] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1952client # [ 1.808711] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1953lb # [ 1.803560] systemd[1]: Finished Load Kernel Module configfs.1954worker1 # [ 1.789420] systemd[1]: Starting Coldplug All udev Devices...1955worker2 # [ 1.775610] systemd[1]: Finished Load Kernel Module configfs.1956client # [ 1.818019] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1957lb # [ 1.809567] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1958worker2 # [ 1.780032] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1959worker1 # [ 1.801189] systemd-journald[66]: Collecting audit messages is disabled.1960client # [ 1.828098] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1961worker1 # [ 1.804009] systemd[1]: Finished Create List of Static Device Nodes.1962lb # [ 1.820236] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1963worker2 # [ 1.792296] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1964worker1 # [ 1.810209] systemd[1]: modprobe@configfs.service: Deactivated successfully.1965client # [ 1.837429] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966worker2 # [ 1.797919] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967worker1 # [ 1.816044] systemd[1]: Finished Load Kernel Module configfs.1968lb # [ 1.833148] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969worker1 # [ 1.822019] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1970lb # [ 1.841319] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971worker2 # [ 1.810467] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1972client # [ 1.859846] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1973worker1 # [ 1.834681] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1974client # [ 1.867463] systemd[1]: Starting Create Static Device Nodes in /dev...1975worker1 # [ 1.845943] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1976lb # [ 1.863713] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977worker2 # [ 1.833869] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1978worker1 # [ 1.852644] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1979lb # [ 1.871345] systemd[1]: Starting Create Static Device Nodes in /dev...1980client # [ 1.881366] systemd[1]: Finished Load Kernel Modules.1981worker2 # [ 1.844470] systemd[1]: Starting Create Static Device Nodes in /dev...1982client # [ 1.890460] systemd[1]: Starting Apply Kernel Variables...1983worker2 # [ 1.854765] systemd[1]: Finished Load Kernel Modules.1984lb # [ 1.886205] systemd[1]: Finished Load Kernel Modules.1985lb # [ 1.892333] systemd[1]: Starting Apply Kernel Variables...1986worker1 # [ 1.876237] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1987client # [ 1.906573] systemd[1]: Finished Create Static Device Nodes in /dev.1988worker2 # [ 1.868314] systemd[1]: Starting Apply Kernel Variables...1989worker1 # [ 1.884809] systemd[1]: Starting Create Static Device Nodes in /dev...1990client # [ 1.911932] systemd[1]: Reached target Preparation for Local File Systems.1991worker2 # [ 1.876510] systemd[1]: Started Journal Service.1992worker2 # [ 1.739797] systemd-modules-load[67]: Inserted module 'dm_mod'1993client # [ 1.779923] systemd-modules-load[68]: Inserted module 'dm_mod'1994client # [ 1.783213] systemd-modules-load[68]: Inserted module 'virtio_balloon'1995lb # [ 1.909607] systemd[1]: Started Journal Service.1996client # [ 1.919375] systemd[1]: Reached target Local File Systems.1997worker2 # [ 1.743983] systemd-modules-load[67]: Inserted module 'virtio_balloon'1998worker2 # [ 1.746182] systemd-modules-load[67]: Inserted module 'virtio_gpu'1999client # [ 1.785334] systemd-modules-load[68]: Inserted module 'virtio_gpu'2000lb # [ 1.772975] systemd-modules-load[67]: Inserted module 'dm_mod'
loading 2,844 hidden lines…
4845client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.12 seconds)4846client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4847client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4848lb: must succeed: curl -sf localhost:9901/clusters4849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4850lb: must succeed: curl -sf localhost:9901/clusters4851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4852worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4853worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4854(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.45 seconds)4855subtest: outputs read back through any worker carry the cache signature4856client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24857client # don't know how to build these paths:4858client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4859worker1 # [ 41.138455] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448784860worker1 # [ 41.144446] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448784861worker1 # [ 41.164664] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04862worker1 # [ 41.170705] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14863worker1 # [ 41.175463] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784864worker1 # [ 41.189390] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448784865client # copying 1 paths to 'lb:50051'...4866client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4867worker1 # [ 41.202709] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448784868worker1 # [ 41.214192] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4869worker1 # [ 41.233653] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=info event=niks3_push_started pid=9174870worker1 # [ 41.352620] nix-grpc-daemon[917]: time=2026-09-23T04:42:49.393Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4871worker1 # [ 41.444897] nix-daemon[863]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4872worker1 # [ 41.457567] nix-daemon[863]: reaped child process 932, status = succeeded4873lb # [ 41.481114] niks3-server[966]: 2026/09/23 04:42:49 INFO Received uploads request method=POST path=/api/pending_closures4874worker1 # [ 41.473357] nix-grpc-daemon[917]: time=2026-09-23T04:42:49.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4875worker1 # [ 41.474837] nix-grpc-daemon[917]: time=2026-09-23T04:42:49.516Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4876lb # [ 41.514552] niks3-server[966]: 2026/09/23 04:42:49 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4877lb # [ 41.522979] niks3-server[966]: 2026/09/23 04:42:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4878lb # [ 41.529974] niks3-server[966]: 2026/09/23 04:42:49 INFO Signed narinfos id=9 count=14879worker1 # [ 41.517421] nix-grpc-daemon[917]: time=2026-09-23T04:42:49.558Z level=INFO msg="Uploading 1 narinfos"4880lb # [ 41.539533] niks3-server[966]: 2026/09/23 04:42:49 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4881lb # [ 41.550677] niks3-server[966]: 2026/09/23 04:42:49 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4882lb # [ 41.559535] niks3-server[966]: 2026/09/23 04:42:49 INFO Completed upload id=94883lb # [ 41.561241] niks3-server[966]: 2026/09/23 04:42:49 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4884worker1 # [ 41.545448] nix-grpc-daemon[917]: time=2026-09-23T04:42:49.586Z level=INFO msg="Upload complete. (188ms)"4885worker1 # [ 41.549988] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=5044886worker1 # [ 41.554680] nix-grpc-daemon[780]: ts=2026-09-23T04:42:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784887worker1 # [ 41.563560] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4888worker1 # [ 41.569231] nix-daemon[863]: reaped child process 915, status = succeeded4889worker1 # [ 41.573814] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4890client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4891client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4892client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4893client # farm-a> LOG-farm-a-sig4894worker1 # [ 42.186988] nix-daemon[863]: reaped child process 936, status = succeeded4895worker1 # [ 42.262469] nix-daemon[863]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4896worker1 # [ 42.274598] nix-daemon[863]: reaped child process 955, status = succeeded4897lb # [ 42.296772] niks3-server[966]: 2026/09/23 04:42:50 INFO Received uploads request method=POST path=/api/pending_closures4898worker1 # [ 42.289303] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.330Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4899worker1 # [ 42.290775] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.332Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4900lb # [ 42.328584] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4901lb # [ 42.342401] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4902lb # [ 42.356127] niks3-server[966]: 2026/09/23 04:42:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4903lb # [ 42.360389] niks3-server[966]: 2026/09/23 04:42:50 INFO Signed narinfos id=10 count=14904worker1 # [ 42.348608] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.389Z level=INFO msg="Uploading 1 narinfos"4905lb # [ 42.371518] niks3-server[966]: 2026/09/23 04:42:50 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4906lb # [ 42.380237] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4907lb # [ 42.383399] niks3-server[966]: 2026/09/23 04:42:50 INFO Completed upload id=104908lb # [ 42.385625] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4909worker1 # [ 42.371406] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.409Z level=INFO msg="Upload complete. (188ms)"4910worker1 # [ 42.376937] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14911worker1 # [ 42.387679] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04912worker1 # [ 42.392296] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14913worker1 # [ 42.400090] nix-daemon[863]: reaped child process 934, status = succeeded4914worker1 # [ 42.430870] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784915worker1 # [ 42.444100] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448784916worker1 # [ 42.456832] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.498Z level=INFO msg="All 1 paths already cached"4917client # copying 1 paths to 'lb:50051'...4918client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4919worker1 # [ 42.465422] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448784920worker1 # [ 42.477569] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4921worker1 # [ 42.575841] nix-daemon[863]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4922worker1 # [ 42.591405] nix-daemon[863]: reaped child process 964, status = succeeded4923lb # [ 42.612060] niks3-server[966]: 2026/09/23 04:42:50 INFO Received uploads request method=POST path=/api/pending_closures4924worker1 # [ 42.604386] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.645Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4925worker1 # [ 42.605912] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.647Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4926lb # [ 42.643693] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4927lb # [ 42.650418] niks3-server[966]: 2026/09/23 04:42:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4928lb # [ 42.657736] niks3-server[966]: 2026/09/23 04:42:50 INFO Signed narinfos id=11 count=14929worker1 # [ 42.645079] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.686Z level=INFO msg="Uploading 1 narinfos"4930lb # [ 42.665053] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4931lb # [ 42.678149] niks3-server[966]: 2026/09/23 04:42:50 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4932lb # [ 42.686183] niks3-server[966]: 2026/09/23 04:42:50 INFO Completed upload id=114933worker1 # [ 42.670757] nix-grpc-daemon[917]: time=2026-09-23T04:42:50.712Z level=INFO msg="Upload complete. (179ms)"4934lb # [ 42.688298] niks3-server[966]: 2026/09/23 04:42:50 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4935worker1 # [ 42.675618] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=6244936worker1 # [ 42.681529] nix-grpc-daemon[780]: ts=2026-09-23T04:42:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784937worker1 # [ 42.692118] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4938worker1 # [ 42.699840] nix-daemon[863]: reaped child process 959, status = succeeded4939worker1 # [ 42.704722] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4940client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4941client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4942client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4943client # farm-b> LOG-farm-b-sig4944worker1 # [ 43.519095] nix-daemon[863]: reaped child process 968, status = succeeded4945worker1 # [ 43.591959] nix-daemon[863]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4946worker1 # [ 43.605964] nix-daemon[863]: reaped child process 987, status = succeeded4947lb # [ 43.628216] niks3-server[966]: 2026/09/23 04:42:51 INFO Received uploads request method=POST path=/api/pending_closures4948worker1 # [ 43.620389] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.661Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4949worker1 # [ 43.622269] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.663Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4950lb # [ 43.669989] niks3-server[966]: 2026/09/23 04:42:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4951lb # [ 43.675773] niks3-server[966]: 2026/09/23 04:42:51 INFO Signed narinfos id=12 count=14952worker1 # [ 43.663183] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.704Z level=INFO msg="Uploading 1 narinfos"4953lb # [ 43.681724] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4954lb # [ 43.685727] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4955lb # [ 43.695721] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4956lb # [ 43.699527] niks3-server[966]: 2026/09/23 04:42:51 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4957lb # [ 43.702368] niks3-server[966]: 2026/09/23 04:42:51 INFO Completed upload id=124958worker1 # [ 43.686860] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.728Z level=INFO msg="Upload complete. (176ms)"4959lb # [ 43.705483] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4960worker1 # [ 43.691527] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14961worker1 # [ 43.702407] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04962worker1 # [ 43.706216] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14963worker1 # [ 43.711605] nix-daemon[863]: reaped child process 966, status = succeeded4964worker1 # [ 43.746303] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784965worker1 # [ 43.758148] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448784966worker1 # [ 43.770303] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.811Z level=INFO msg="All 2 paths already cached"4967client # copying 1 paths to 'lb:50051'...4968client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4969worker1 # [ 43.779238] nix-grpc-daemon[780]: ts=2026-09-23T04:42:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448784970worker1 # [ 43.788299] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4971worker1 # [ 43.882063] nix-daemon[863]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4972worker1 # [ 43.896868] nix-daemon[863]: reaped child process 996, status = succeeded4973lb # [ 43.917781] niks3-server[966]: 2026/09/23 04:42:51 INFO Received uploads request method=POST path=/api/pending_closures4974worker1 # [ 43.910538] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.951Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4975worker1 # [ 43.911974] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.951Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4976lb # [ 43.948260] niks3-server[966]: 2026/09/23 04:42:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4977lb # [ 43.954260] niks3-server[966]: 2026/09/23 04:42:51 INFO Signed narinfos id=13 count=14978worker1 # [ 43.942088] nix-grpc-daemon[917]: time=2026-09-23T04:42:51.983Z level=INFO msg="Uploading 1 narinfos"4979lb # [ 43.959466] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4980lb # [ 43.971149] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4981lb # [ 43.979957] niks3-server[966]: 2026/09/23 04:42:51 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4982lb # [ 43.988793] niks3-server[966]: 2026/09/23 04:42:51 INFO Completed upload id=134983worker1 # [ 43.973369] nix-grpc-daemon[917]: time=2026-09-23T04:42:52.014Z level=INFO msg="Upload complete. (173ms)"4984lb # [ 43.992368] niks3-server[966]: 2026/09/23 04:42:51 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4985worker1 # [ 43.976993] nix-grpc-daemon[780]: ts=2026-09-23T04:42:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=7524986worker1 # [ 43.981993] nix-grpc-daemon[780]: ts=2026-09-23T04:42:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448784987worker1 # [ 43.991690] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4988worker1 # [ 43.999293] nix-daemon[863]: reaped child process 991, status = succeeded4989worker1 # [ 44.004439] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4990client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4991client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4992client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4993client # farm-top> LOG-farm-top-sig4994worker1 # [ 44.618670] nix-daemon[863]: reaped child process 1000, status = succeeded4995lb # [ 44.664976] envoy[904]: conn peer=[2001:db8:1::1]:39086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9490 tx=4356 ms=51954996lb # [ 44.668699] envoy[904]: conn peer=[2001:db8:1::1]:42578 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=478 ms=464997lb # [ 44.672161] envoy[904]: conn peer=[2001:db8:1::1]:42588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1879 ms=19514998worker1 # [ 44.693772] nix-daemon[863]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4999worker1 # [ 44.705514] nix-daemon[863]: reaped child process 1019, status = succeeded5000lb # [ 44.729750] niks3-server[966]: 2026/09/23 04:42:52 INFO Received uploads request method=POST path=/api/pending_closures5001worker1 # [ 44.725505] nix-grpc-daemon[917]: time=2026-09-23T04:42:52.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5002worker1 # [ 44.728061] nix-grpc-daemon[917]: time=2026-09-23T04:42:52.768Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5003lb # [ 44.764523] niks3-server[966]: 2026/09/23 04:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5004lb # [ 44.767968] niks3-server[966]: 2026/09/23 04:42:52 INFO Signed narinfos id=14 count=15005worker1 # [ 44.753625] nix-grpc-daemon[917]: time=2026-09-23T04:42:52.795Z level=INFO msg="Uploading 1 narinfos"5006lb # [ 44.773200] niks3-server[966]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5007lb # [ 44.776079] niks3-server[966]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5008lb # [ 44.778490] niks3-server[966]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5009lb # [ 44.783275] niks3-server[966]: 2026/09/23 04:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5010lb # [ 44.786192] niks3-server[966]: 2026/09/23 04:42:52 INFO Completed upload id=145011worker1 # [ 44.770596] nix-grpc-daemon[917]: time=2026-09-23T04:42:52.812Z level=INFO msg="Upload complete. (160ms)"5012worker1 # [ 44.773229] nix-grpc-daemon[780]: ts=2026-09-23T04:42:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15013lb # [ 44.794756] niks3-server[966]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5014worker1 # [ 44.780743] nix-daemon[863]: reaped child process 998, status = succeeded5015client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 3.82 seconds)5016client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5017client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)5018client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5019worker1 # [ 45.029864] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:543305020worker1 # [ 45.072170] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:543305021client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5022client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5023worker2 # [ 45.242487] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:490265024client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5025client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5026client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5027client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5028worker2 # [ 45.269127] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:490265029worker2 # [ 45.275742] nix-grpc-daemon[785]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5030worker2 # [ 45.289884] nix-grpc-daemon[785]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5031worker2 # [ 45.301839] nix-grpc-daemon[785]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5032client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)5033client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5034client # finding garbage collector roots...5035client # removing stale temporary roots file "/nix/var/nix/temproots/849"5036client # removing stale temporary roots file "/nix/var/nix/temproots/871"5037client # removing stale temporary roots file "/nix/var/nix/temproots/799"5038client # removing stale temporary roots file "/nix/var/nix/temproots/837"5039client # removing stale temporary roots file "/nix/var/nix/temproots/894"5040client # removing stale temporary roots file "/nix/var/nix/temproots/816"5041client # removing stale temporary roots file "/nix/var/nix/temproots/857"5042client # removing stale temporary roots file "/nix/var/nix/temproots/879"5043client # deleting specified paths...5044client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5045client # deleting unused links...5046client # note: hard linking is currently saving 0.0 KiB5047worker2 # [ 45.584734] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485085048worker1 # [ 45.602091] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785049worker1 # [ 45.611860] nix-grpc-daemon[917]: time=2026-09-23T04:42:53.653Z level=INFO msg="All 1 paths already cached"5050worker1 # [ 45.618782] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:448785051worker2 # [ 45.613459] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485085052client # copying 3 paths...5053worker2 # [ 45.623651] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485085054client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5055worker2 # [ 45.639721] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:485085056worker2 # [ 45.645367] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_out=2325057worker1 # [ 45.660473] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785058worker1 # [ 45.667870] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1765059worker1 # [ 45.672445] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785060worker1 # [ 45.676093] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1285061client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5062client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5063client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.40 seconds)5064(finished: subtest: outputs read back through any worker carry the cache signature, in 4.76 seconds)5065subtest: two clients wanting the same drv build it once5066client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5067client # [ 45.804913] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5068client # Running as unit: dedup1.service; invocation ID: 3a60f88bdfc0478f9f46181a2d1c5c5c5069client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5070client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5071worker2 # [ 45.896259] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485085072client # [ 45.941552] nix[947]: don't know how to build these paths:5073client # [ 45.943168] nix[947]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5074worker1 # [ 45.925977] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05075worker1 # [ 45.931172] nix-grpc-daemon[780]: ts=2026-09-23T04:42:53Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15076worker2 # [ 45.958355] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085077worker2 # [ 45.971187] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085078client # [ 46.021257] nix[947]: copying 1 paths to 'lb:50051'...5079worker2 # [ 45.983205] nix-grpc-daemon[785]: ts=2026-09-23T04:42:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085080client # [ 46.024171] nix[947]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5081worker2 # [ 45.996128] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5082worker2 # [ 46.105763] nix-daemon[854]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)5083worker2 # [ 46.120364] nix-daemon[854]: reaped child process 1022, status = succeeded5084lb # [ 46.157308] niks3-server[966]: 2026/09/23 04:42:54 INFO Received uploads request method=POST path=/api/pending_closures5085worker2 # [ 46.136274] nix-grpc-daemon[867]: time=2026-09-23T04:42:53.957Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5086worker2 # [ 46.138256] nix-grpc-daemon[867]: time=2026-09-23T04:42:53.958Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5087lb # [ 46.192337] niks3-server[966]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5088lb # [ 46.203175] niks3-server[966]: 2026/09/23 04:42:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5089worker2 # [ 46.179652] nix-grpc-daemon[867]: time=2026-09-23T04:42:54.000Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 46.208808] niks3-server[966]: 2026/09/23 04:42:54 INFO Signed narinfos id=15 count=15091lb # [ 46.219996] niks3-server[966]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5092lb # [ 46.231668] niks3-server[966]: 2026/09/23 04:42:54 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5093lb # [ 46.235261] niks3-server[966]: 2026/09/23 04:42:54 INFO Completed upload id=155094worker2 # [ 46.206756] nix-grpc-daemon[867]: time=2026-09-23T04:42:54.027Z level=INFO msg="Upload complete. (199ms)"5095lb # [ 46.239544] niks3-server[966]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5096worker2 # [ 46.211547] nix-grpc-daemon[785]: ts=2026-09-23T04:42:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_in=4325097worker2 # [ 46.217756] nix-grpc-daemon[785]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085098worker2 # [ 46.228129] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5099worker2 # [ 46.244360] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5100client # [ 46.282159] nix[947]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5101worker2 # [ 46.245723] nix-daemon[854]: reaped child process 1017, status = succeeded5102client # [ 46.863321] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5103client # Running as unit: dedup2.service; invocation ID: ae10a1e3c1c24e46834004b4f4f5480a5104client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5105client # [ 47.026095] nix[963]: don't know how to build these paths:5106client # [ 47.026986] nix[963]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5107worker1 # [ 46.999128] nix-grpc-daemon[780]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448785108worker1 # [ 47.009408] nix-grpc-daemon[780]: ts=2026-09-23T04:42:55Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05109worker2 # [ 47.002268] nix-grpc-daemon[785]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085110client # [ 47.046773] nix[963]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5111client: must succeed: sleep 25112client: (finished: must succeed: sleep 2, in 2.04 seconds)5113client: waiting for success: ! systemctl is-active dedup1 dedup25114worker2 # [ 50.280838] nix-daemon[854]: reaped child process 1026, status = succeeded5115worker2 # [ 50.367324] nix-daemon[854]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)5116worker2 # [ 50.378772] nix-daemon[854]: reaped child process 1059, status = succeeded5117lb # [ 50.417395] niks3-server[966]: 2026/09/23 04:42:58 INFO Received uploads request method=POST path=/api/pending_closures5118worker2 # [ 50.398301] nix-grpc-daemon[867]: time=2026-09-23T04:42:58.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5119worker2 # [ 50.401154] nix-grpc-daemon[867]: time=2026-09-23T04:42:58.221Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5120lb # [ 50.460339] niks3-server[966]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5121lb # [ 50.465270] niks3-server[966]: 2026/09/23 04:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5122lb # [ 50.470484] niks3-server[966]: 2026/09/23 04:42:58 INFO Signed narinfos id=16 count=15123worker2 # [ 50.444588] nix-grpc-daemon[867]: time=2026-09-23T04:42:58.265Z level=INFO msg="Uploading 1 narinfos"5124lb # [ 50.482553] niks3-server[966]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5125lb # [ 50.488443] niks3-server[966]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5126lb # [ 50.494471] niks3-server[966]: 2026/09/23 04:42:58 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5127lb # [ 50.502657] niks3-server[966]: 2026/09/23 04:42:58 INFO Completed upload id=165128worker2 # [ 50.474135] nix-grpc-daemon[867]: time=2026-09-23T04:42:58.294Z level=INFO msg="Upload complete. (218ms)"5129lb # [ 50.504870] niks3-server[966]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5130worker2 # [ 50.481078] nix-grpc-daemon[785]: ts=2026-09-23T04:42:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15131worker2 # [ 50.488282] nix-grpc-daemon[785]: ts=2026-09-23T04:42:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15132worker2 # [ 50.501518] nix-daemon[854]: reaped child process 1024, status = succeeded5133client # [ 50.568342] systemd[1]: dedup2.service: Deactivated successfully.5134client # [ 50.572414] systemd[1]: dedup2.service: Consumed 138ms CPU time over 3.709s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5135client # [ 50.580541] systemd[1]: dedup1.service: Deactivated successfully.5136client # [ 50.582640] systemd[1]: dedup1.service: Consumed 122ms CPU time over 4.774s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.5137client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)5138client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5139client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5140worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5141worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5142worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5143worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5144worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5145worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5146worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5147worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5148(finished: subtest: two clients wanting the same drv build it once, in 5.70 seconds)5149subtest: --timeout and --max-silent-time reach the builder5150worker1 # [ 51.609793] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448785151worker1 # [ 51.620716] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05152worker1 # [ 51.626185] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15153worker1 # [ 51.632393] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785154worker1 # [ 51.640393] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785155worker1 # [ 51.652310] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448785156worker1 # [ 51.663086] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5157worker1 # [ 51.761356] nix-daemon[863]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)5158worker1 # [ 51.775274] nix-daemon[863]: reaped child process 1053, status = succeeded5159lb # [ 51.801320] niks3-server[966]: 2026/09/23 04:42:59 INFO Received uploads request method=POST path=/api/pending_closures5160worker1 # [ 51.794911] nix-grpc-daemon[917]: time=2026-09-23T04:42:59.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5161worker1 # [ 51.796903] nix-grpc-daemon[917]: time=2026-09-23T04:42:59.837Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5162lb # [ 51.833449] niks3-server[966]: 2026/09/23 04:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5163lb # [ 51.838959] niks3-server[966]: 2026/09/23 04:42:59 INFO Signed narinfos id=17 count=15164worker1 # [ 51.825891] nix-grpc-daemon[917]: time=2026-09-23T04:42:59.867Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 51.845213] niks3-server[966]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5166lb # [ 51.859163] niks3-server[966]: 2026/09/23 04:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5167lb # [ 51.862970] niks3-server[966]: 2026/09/23 04:42:59 INFO Completed upload id=175168worker1 # [ 51.847493] nix-grpc-daemon[917]: time=2026-09-23T04:42:59.888Z level=INFO msg="Upload complete. (172ms)"5169lb # [ 51.864274] niks3-server[966]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5170lb # [ 51.870092] niks3-server[966]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5171worker1 # [ 51.851464] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=4245172worker1 # [ 51.857310] nix-grpc-daemon[780]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785173worker1 # [ 51.866220] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5174worker1 # [ 51.872308] nix-daemon[863]: reaped child process 1048, status = succeeded5175worker1 # [ 51.876694] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5176lb # [ 54.664980] envoy[904]: conn peer=[2001:db8:1::1]:42600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9421 tx=4315 ms=36515177lb # [ 54.668877] envoy[904]: conn peer=[2001:db8:1::1]:39064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1425178lb # [ 54.672348] envoy[904]: conn peer=[2001:db8:1::1]:39090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=715179lb # [ 54.675765] envoy[904]: conn peer=[2001:db8:1::1]:39080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=855180lb # [ 54.680142] envoy[904]: conn peer=[2001:db8:1::1]:39084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=805181lb # [ 54.683572] envoy[904]: conn peer=[2001:db8:1::1]:39092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3037 tx=1562 ms=45975182lb # [ 54.687072] envoy[904]: conn peer=[2001:db8:1::1]:39098 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=35145183worker1 # [ 58.490270] nix-grpc-daemon[780]: ts=2026-09-23T04:43:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05184worker1 # [ 58.502736] nix-daemon[863]: reaped child process 1055, status = succeeded5185worker1 # [ 58.503748] nix-daemon[863]: reaped child process 1057, status = succeeded5186worker1 # [ 58.631923] nix-grpc-daemon[780]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448785187worker1 # [ 58.643307] nix-grpc-daemon[780]: ts=2026-09-23T04:43:06Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05188worker1 # [ 58.647387] nix-grpc-daemon[780]: ts=2026-09-23T04:43:06Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15189worker2 # [ 58.642797] nix-grpc-daemon[785]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085190worker2 # [ 58.657598] nix-grpc-daemon[785]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085191worker2 # [ 58.668415] nix-grpc-daemon[785]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085192worker2 # [ 58.679486] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5193worker2 # [ 58.780634] nix-daemon[854]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5194worker2 # [ 58.793993] nix-daemon[854]: reaped child process 1075, status = succeeded5195lb # [ 58.829851] niks3-server[966]: 2026/09/23 04:43:06 INFO Received uploads request method=POST path=/api/pending_closures5196worker2 # [ 58.809525] nix-grpc-daemon[867]: time=2026-09-23T04:43:06.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5197worker2 # [ 58.811448] nix-grpc-daemon[867]: time=2026-09-23T04:43:06.631Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5198lb # [ 58.865287] niks3-server[966]: 2026/09/23 04:43:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5199lb # [ 58.870066] niks3-server[966]: 2026/09/23 04:43:06 INFO Signed narinfos id=18 count=15200worker2 # [ 58.844202] nix-grpc-daemon[867]: time=2026-09-23T04:43:06.665Z level=INFO msg="Uploading 1 narinfos"5201lb # [ 58.885979] niks3-server[966]: 2026/09/23 04:43:06 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5202lb # [ 59.159722] niks3-server[966]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5203lb # [ 59.180966] niks3-server[966]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5204lb # [ 59.192210] niks3-server[966]: 2026/09/23 04:43:07 INFO Completed upload id=185205worker2 # [ 59.163603] nix-grpc-daemon[867]: time=2026-09-23T04:43:06.984Z level=INFO msg="Upload complete. (473ms)"5206worker2 # [ 59.168488] nix-grpc-daemon[785]: ts=2026-09-23T04:43:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_in=4245207worker2 # [ 59.175565] nix-grpc-daemon[785]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085208worker2 # [ 59.185608] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5209lb # [ 59.219846] niks3-server[966]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5210worker2 # [ 59.202284] nix-daemon[854]: reaped child process 1070, status = succeeded5211worker2 # [ 59.211168] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5212lb # [ 64.664634] envoy[904]: conn peer=[2001:db8:1::1]:39110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=934 ms=68965213worker2 # [ 65.631084] nix-grpc-daemon[785]: ts=2026-09-23T04:43:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05214worker2 # [ 65.656413] nix-daemon[854]: reaped child process 1077, status = succeeded5215worker2 # [ 65.658823] nix-daemon[854]: reaped child process 1079, status = succeeded5216(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.32 seconds)5217subtest: the first client leaving does not fail the second5218client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5219client # [ 65.847563] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5220client # Running as unit: share1.service; invocation ID: 6095a47eef2b4ad8bf1fb07662611e8c5221client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5222client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5223client # [ 65.992662] nix[1036]: don't know how to build these paths:5224client # [ 65.993549] nix[1036]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5225worker2 # [ 65.952138] nix-grpc-daemon[785]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485085226worker1 # [ 65.977711] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05227worker1 # [ 65.982381] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15228worker1 # [ 65.987295] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785229worker1 # [ 65.999061] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785230client # [ 66.034777] nix[1036]: copying 1 paths to 'lb:50051'...5231client # [ 66.035633] nix[1036]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5232worker1 # [ 66.009668] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448785233worker1 # [ 66.021369] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5234worker1 # [ 66.127440] nix-daemon[863]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)5235worker1 # [ 66.142739] nix-daemon[863]: reaped child process 1088, status = succeeded5236lb # [ 66.166432] niks3-server[966]: 2026/09/23 04:43:14 INFO Received uploads request method=POST path=/api/pending_closures5237worker1 # [ 66.158963] nix-grpc-daemon[917]: time=2026-09-23T04:43:14.199Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5238worker1 # [ 66.160335] nix-grpc-daemon[917]: time=2026-09-23T04:43:14.199Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5239lb # [ 66.196852] niks3-server[966]: 2026/09/23 04:43:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5240worker1 # [ 66.187953] nix-grpc-daemon[917]: time=2026-09-23T04:43:14.229Z level=INFO msg="Uploading 1 narinfos"5241lb # [ 66.203683] niks3-server[966]: 2026/09/23 04:43:14 INFO Signed narinfos id=19 count=15242lb # [ 66.212812] niks3-server[966]: 2026/09/23 04:43:14 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5243lb # [ 66.225134] niks3-server[966]: 2026/09/23 04:43:14 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5244lb # [ 66.233083] niks3-server[966]: 2026/09/23 04:43:14 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5245lb # [ 66.242233] niks3-server[966]: 2026/09/23 04:43:14 INFO Completed upload id=195246worker1 # [ 66.226729] nix-grpc-daemon[917]: time=2026-09-23T04:43:14.268Z level=INFO msg="Upload complete. (193ms)"5247lb # [ 66.244224] niks3-server[966]: 2026/09/23 04:43:14 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5248worker1 # [ 66.231916] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=4325249worker1 # [ 66.238111] nix-grpc-daemon[780]: ts=2026-09-23T04:43:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785250worker1 # [ 66.251116] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5251worker1 # [ 66.258074] nix-daemon[863]: reaped child process 1082, status = succeeded5252worker1 # [ 66.263151] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5253client # [ 66.288894] nix[1036]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5254client # [ 66.913089] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5255client # Running as unit: share2.service; invocation ID: 82b70665a8a14391be8d33a845b296495256client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5257client # [ 67.063389] nix[1052]: don't know how to build these paths:5258client # [ 67.064409] nix[1052]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5259worker2 # [ 67.023193] nix-grpc-daemon[785]: ts=2026-09-23T04:43:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:485085260worker1 # [ 67.043910] nix-grpc-daemon[780]: ts=2026-09-23T04:43:15Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05261worker1 # [ 67.046508] nix-grpc-daemon[780]: ts=2026-09-23T04:43:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785262client # [ 67.075097] nix[1052]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5263client: must succeed: sleep 2; systemctl kill -s INT share15264client # [ 70.221655] systemd[1]: share1.service: Sent signal SIGINT to main process 1036 (nix) on client request.5265client # [ 70.226681] nix[1036]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5266client # [ 70.232945] nix[1036]: error: interrupted by the user5267client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5268client: waiting for success: ! systemctl is-active share15269client # [ 70.276831] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5270client # [ 70.278430] systemd[1]: share1.service: Failed with result 'exit-code'.5271client # [ 70.280528] systemd[1]: share1.service: Consumed 120ms CPU time over 4.432s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5272client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5273client: waiting for success: ! systemctl is-active share25274worker1 # [ 71.481762] nix-daemon[863]: reaped child process 1092, status = succeeded5275worker1 # [ 71.557772] nix-daemon[863]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)5276worker1 # [ 71.570070] nix-daemon[863]: reaped child process 1131, status = succeeded5277lb # [ 71.596855] niks3-server[966]: 2026/09/23 04:43:19 INFO Received uploads request method=POST path=/api/pending_closures5278worker1 # [ 71.588955] nix-grpc-daemon[917]: time=2026-09-23T04:43:19.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5279worker1 # [ 71.591303] nix-grpc-daemon[917]: time=2026-09-23T04:43:19.632Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5280lb # [ 71.640148] niks3-server[966]: 2026/09/23 04:43:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5281lb # [ 71.646146] niks3-server[966]: 2026/09/23 04:43:19 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5282lb # [ 71.649754] niks3-server[966]: 2026/09/23 04:43:19 INFO Signed narinfos id=20 count=15283worker1 # [ 71.637028] nix-grpc-daemon[917]: time=2026-09-23T04:43:19.678Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 71.656397] niks3-server[966]: 2026/09/23 04:43:19 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5285lb # [ 71.661772] niks3-server[966]: 2026/09/23 04:43:19 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5286lb # [ 71.666162] niks3-server[966]: 2026/09/23 04:43:19 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5287lb # [ 71.669090] niks3-server[966]: 2026/09/23 04:43:19 INFO Completed upload id=205288worker1 # [ 71.653640] nix-grpc-daemon[917]: time=2026-09-23T04:43:19.695Z level=INFO msg="Upload complete. (180ms)"5289lb # [ 71.671529] niks3-server[966]: 2026/09/23 04:43:19 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5290worker1 # [ 71.660508] nix-grpc-daemon[780]: ts=2026-09-23T04:43:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15291worker1 # [ 71.665243] nix-grpc-daemon[780]: ts=2026-09-23T04:43:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15292worker1 # [ 71.675792] nix-daemon[863]: reaped child process 1090, status = succeeded5293client # [ 71.759959] systemd[1]: share2.service: Deactivated successfully.5294client # [ 71.763878] systemd[1]: share2.service: Consumed 122ms CPU time over 4.849s wall clock time, 33.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5295client: (finished: waiting for success: ! systemctl is-active share2, in 1.09 seconds)5296client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5297client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5298(finished: subtest: the first client leaving does not fail the second, in 6.81 seconds)5299subtest: build hook: nix-daemon with builders = grpc://lb5300client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5301client # [ 72.702413] systemd[1]: Started Nix Daemon.5302client # [ 72.791443] nix-daemon[1104]: accepted connection from pid 1101, user root (trusted)5303worker2 # [ 72.873228] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:485085304worker1 # [ 72.890831] nix-grpc-daemon[780]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785305worker1 # [ 72.906205] nix-grpc-daemon[780]: ts=2026-09-23T04:43:20Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05306worker1 # [ 72.910176] nix-grpc-daemon[780]: ts=2026-09-23T04:43:20Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15307worker2 # [ 72.904555] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085308worker2 # [ 72.919136] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085309worker2 # [ 72.930684] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085310worker2 # [ 72.942239] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5311worker2 # [ 73.043487] nix-daemon[854]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)5312worker2 # [ 73.055569] nix-daemon[854]: reaped child process 1121, status = succeeded5313lb # [ 73.093697] niks3-server[966]: 2026/09/23 04:43:21 INFO Received uploads request method=POST path=/api/pending_closures5314worker2 # [ 73.074272] nix-grpc-daemon[867]: time=2026-09-23T04:43:20.894Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5315worker2 # [ 73.076304] nix-grpc-daemon[867]: time=2026-09-23T04:43:20.896Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5316lb # [ 73.130213] niks3-server[966]: 2026/09/23 04:43:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5317lb # [ 73.139136] niks3-server[966]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5318worker2 # [ 73.115971] nix-grpc-daemon[867]: time=2026-09-23T04:43:20.936Z level=INFO msg="Uploading 1 narinfos"5319lb # [ 73.146150] niks3-server[966]: 2026/09/23 04:43:21 INFO Signed narinfos id=21 count=15320lb # [ 73.151974] niks3-server[966]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5321lb # [ 73.166797] niks3-server[966]: 2026/09/23 04:43:21 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5322worker2 # [ 73.144220] nix-grpc-daemon[867]: time=2026-09-23T04:43:20.965Z level=INFO msg="Upload complete. (190ms)"5323lb # [ 73.173136] niks3-server[966]: 2026/09/23 04:43:21 INFO Completed upload id=215324worker2 # [ 73.146496] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_in=5125325worker2 # [ 73.150247] nix-grpc-daemon[785]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085326lb # [ 73.179571] niks3-server[966]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5327worker2 # [ 73.156745] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5328worker2 # [ 73.158979] nix-daemon[854]: reaped child process 1116, status = succeeded5329worker2 # [ 73.164047] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5330worker2 # [ 74.003552] nix-daemon[854]: reaped child process 1125, status = succeeded5331worker2 # [ 74.084638] nix-daemon[854]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)5332worker2 # [ 74.096648] nix-daemon[854]: reaped child process 1144, status = succeeded5333lb # [ 74.132345] niks3-server[966]: 2026/09/23 04:43:22 INFO Received uploads request method=POST path=/api/pending_closures5334worker2 # [ 74.112335] nix-grpc-daemon[867]: time=2026-09-23T04:43:21.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5335worker2 # [ 74.114517] nix-grpc-daemon[867]: time=2026-09-23T04:43:21.934Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5336lb # [ 74.170299] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5337lb # [ 74.184193] niks3-server[966]: 2026/09/23 04:43:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5338lb # [ 74.194352] niks3-server[966]: 2026/09/23 04:43:22 INFO Signed narinfos id=22 count=15339worker2 # [ 74.167983] nix-grpc-daemon[867]: time=2026-09-23T04:43:21.988Z level=INFO msg="Uploading 1 narinfos"5340lb # [ 74.200856] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5341lb # [ 74.205754] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5342lb # [ 74.215463] niks3-server[966]: 2026/09/23 04:43:22 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5343lb # [ 74.220119] niks3-server[966]: 2026/09/23 04:43:22 INFO Completed upload id=225344lb # [ 74.221527] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5345worker2 # [ 74.191439] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.012Z level=INFO msg="Upload complete. (220ms)"5346worker2 # [ 74.197838] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15347worker1 # [ 74.216802] nix-grpc-daemon[780]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:448785348worker2 # [ 74.212348] nix-daemon[854]: reaped child process 1123, status = succeeded5349worker1 # [ 74.226274] nix-grpc-daemon[780]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5350worker1 # [ 74.250077] nix-grpc-daemon[780]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785351worker1 # [ 74.255446] nix-grpc-daemon[780]: ts=2026-09-23T04:43:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1285352worker2 # [ 74.451559] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:485085353worker2 # [ 74.457421] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085354worker2 # [ 74.462773] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.282Z level=INFO msg="All 1 paths already cached"5355worker1 # [ 74.486376] nix-grpc-daemon[780]: ts=2026-09-23T04:43:22Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05356worker1 # [ 74.490455] nix-grpc-daemon[780]: ts=2026-09-23T04:43:22Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15357worker2 # [ 74.484710] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085358worker2 # [ 74.496776] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085359worker2 # [ 74.509827] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.330Z level=INFO msg="All 1 paths already cached"5360worker2 # [ 74.517506] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085361worker2 # [ 74.525868] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5362worker2 # [ 74.628232] nix-daemon[854]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)5363lb # [ 74.666168] envoy[904]: conn peer=[2001:db8:1::1]:51394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=975 ms=70265364worker2 # [ 74.642175] nix-daemon[854]: reaped child process 1151, status = succeeded5365lb # [ 74.669894] envoy[904]: conn peer=[2001:db8:1::1]:46646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=794 ms=42535366lb # [ 74.673379] envoy[904]: conn peer=[2001:db8:1::1]:46660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=46805367lb # [ 74.677844] envoy[904]: conn peer=[2001:db8:1::1]:60320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1921 ms=14225368lb # [ 74.681374] envoy[904]: conn peer=[2001:db8:1::1]:60324 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=555369lb # [ 74.686842] niks3-server[966]: 2026/09/23 04:43:22 INFO Received uploads request method=POST path=/api/pending_closures5370worker2 # [ 74.666099] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5371worker2 # [ 74.667631] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.488Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5372lb # [ 74.717759] niks3-server[966]: 2026/09/23 04:43:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5373lb # [ 74.726266] niks3-server[966]: 2026/09/23 04:43:22 INFO Signed narinfos id=23 count=15374worker2 # [ 74.699908] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.520Z level=INFO msg="Uploading 1 narinfos"5375lb # [ 74.729090] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5376lb # [ 74.736642] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5377lb # [ 74.741909] niks3-server[966]: 2026/09/23 04:43:22 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5378lb # [ 74.745389] niks3-server[966]: 2026/09/23 04:43:22 INFO Completed upload id=235379worker2 # [ 74.717951] nix-grpc-daemon[867]: time=2026-09-23T04:43:22.538Z level=INFO msg="Upload complete. (177ms)"5380lb # [ 74.749287] niks3-server[966]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5381worker2 # [ 74.721985] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_in=6245382worker2 # [ 74.727265] nix-grpc-daemon[785]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085383worker2 # [ 74.737116] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5384worker2 # [ 74.743074] nix-daemon[854]: reaped child process 1146, status = succeeded5385worker2 # [ 74.747181] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5386worker2 # [ 75.780480] nix-daemon[854]: reaped child process 1155, status = succeeded5387worker2 # [ 75.872171] nix-daemon[854]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5388worker2 # [ 75.884581] nix-daemon[854]: reaped child process 1174, status = succeeded5389lb # [ 75.920803] niks3-server[966]: 2026/09/23 04:43:23 INFO Received uploads request method=POST path=/api/pending_closures5390worker2 # [ 75.902836] nix-grpc-daemon[867]: time=2026-09-23T04:43:23.723Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5391worker2 # [ 75.905058] nix-grpc-daemon[867]: time=2026-09-23T04:43:23.725Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5392lb # [ 75.963863] niks3-server[966]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5393lb # [ 75.973113] niks3-server[966]: 2026/09/23 04:43:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5394lb # [ 75.982793] niks3-server[966]: 2026/09/23 04:43:23 INFO Signed narinfos id=24 count=15395worker2 # [ 75.954444] nix-grpc-daemon[867]: time=2026-09-23T04:43:23.775Z level=INFO msg="Uploading 1 narinfos"5396lb # [ 75.988132] niks3-server[966]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5397lb # [ 75.995450] niks3-server[966]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5398lb # [ 76.002661] niks3-server[966]: 2026/09/23 04:43:23 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5399lb # [ 76.006722] niks3-server[966]: 2026/09/23 04:43:23 INFO Completed upload id=245400worker2 # [ 75.978187] nix-grpc-daemon[867]: time=2026-09-23T04:43:23.799Z level=INFO msg="Upload complete. (222ms)"5401lb # [ 76.009154] niks3-server[966]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5402worker2 # [ 75.984825] nix-grpc-daemon[785]: ts=2026-09-23T04:43:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15403worker1 # [ 76.003855] nix-grpc-daemon[780]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:448785404worker2 # [ 75.999308] nix-daemon[854]: reaped child process 1153, status = succeeded5405worker1 # [ 76.012655] nix-grpc-daemon[780]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5406worker1 # [ 76.036815] nix-grpc-daemon[780]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785407worker1 # [ 76.041846] nix-grpc-daemon[780]: ts=2026-09-23T04:43:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1765408worker2 # [ 76.241742] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:485085409worker2 # [ 76.248522] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085410worker2 # [ 76.254877] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.074Z level=INFO msg="All 2 paths already cached"5411worker1 # [ 76.278341] nix-grpc-daemon[780]: ts=2026-09-23T04:43:24Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05412worker1 # [ 76.283549] nix-grpc-daemon[780]: ts=2026-09-23T04:43:24Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15413worker2 # [ 76.272703] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085414worker2 # [ 76.285787] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085415worker2 # [ 76.299624] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.120Z level=INFO msg="All 2 paths already cached"5416worker2 # [ 76.306950] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085417worker2 # [ 76.315448] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5418worker2 # [ 76.420159] nix-daemon[854]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5419worker2 # [ 76.435554] nix-daemon[854]: reaped child process 1181, status = succeeded5420lb # [ 76.469475] niks3-server[966]: 2026/09/23 04:43:24 INFO Received uploads request method=POST path=/api/pending_closures5421worker2 # [ 76.449035] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.270Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5422worker2 # [ 76.450544] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.271Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5423lb # [ 76.504132] niks3-server[966]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5424lb # [ 76.511594] niks3-server[966]: 2026/09/23 04:43:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5425lb # [ 76.520182] niks3-server[966]: 2026/09/23 04:43:24 INFO Signed narinfos id=25 count=15426worker2 # [ 76.494825] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.315Z level=INFO msg="Uploading 1 narinfos"5427lb # [ 76.527496] niks3-server[966]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5428lb # [ 76.539455] niks3-server[966]: 2026/09/23 04:43:24 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5429lb # [ 76.545195] niks3-server[966]: 2026/09/23 04:43:24 INFO Completed upload id=255430worker2 # [ 76.517552] nix-grpc-daemon[867]: time=2026-09-23T04:43:24.338Z level=INFO msg="Upload complete. (188ms)"5431lb # [ 76.549064] niks3-server[966]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5432worker2 # [ 76.522154] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=0 paths=1 nar_bytes_in=7525433worker2 # [ 76.529159] nix-grpc-daemon[785]: ts=2026-09-23T04:43:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085434worker2 # [ 76.538080] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5435worker2 # [ 76.547289] nix-daemon[854]: reaped child process 1176, status = succeeded5436worker2 # [ 76.552569] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5437worker2 # [ 77.790761] nix-daemon[854]: reaped child process 1185, status = succeeded5438worker2 # [ 77.872339] nix-daemon[854]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5439worker2 # [ 77.885113] nix-daemon[854]: reaped child process 1204, status = succeeded5440lb # [ 77.922909] niks3-server[966]: 2026/09/23 04:43:25 INFO Received uploads request method=POST path=/api/pending_closures5441worker2 # [ 77.903079] nix-grpc-daemon[867]: time=2026-09-23T04:43:25.723Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5442worker2 # [ 77.904840] nix-grpc-daemon[867]: time=2026-09-23T04:43:25.725Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5443lb # [ 77.964916] niks3-server[966]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5444lb # [ 77.975496] niks3-server[966]: 2026/09/23 04:43:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5445lb # [ 77.980994] niks3-server[966]: 2026/09/23 04:43:25 INFO Signed narinfos id=26 count=15446worker2 # [ 77.952659] nix-grpc-daemon[867]: time=2026-09-23T04:43:25.773Z level=INFO msg="Uploading 1 narinfos"5447lb # [ 77.988928] niks3-server[966]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5448lb # [ 78.000851] niks3-server[966]: 2026/09/23 04:43:25 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5449lb # [ 78.003115] niks3-server[966]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5450lb # [ 78.005789] niks3-server[966]: 2026/09/23 04:43:25 INFO Completed upload id=265451worker2 # [ 77.977300] nix-grpc-daemon[867]: time=2026-09-23T04:43:25.798Z level=INFO msg="Upload complete. (216ms)"5452worker2 # [ 77.983857] nix-grpc-daemon[785]: ts=2026-09-23T04:43:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15453worker2 # [ 77.997781] nix-daemon[854]: reaped child process 1183, status = succeeded5454worker1 # [ 78.044536] nix-grpc-daemon[780]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:448785455worker1 # [ 78.056910] nix-grpc-daemon[780]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5456worker1 # [ 78.085741] nix-grpc-daemon[780]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785457worker1 # [ 78.090699] nix-grpc-daemon[780]: ts=2026-09-23T04:43:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=2325458client # [ 78.230063] nix-daemon[1104]: reaped child process 1106, status = succeeded5459client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.63 seconds)5460client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5461client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5462client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5463client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5464client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5465client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5466(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.69 seconds)5467subtest: an input only one worker has still reaches the builder5468client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5469client # this derivation will be built:5470client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5471client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5472client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds)5473client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5474client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5475worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5476worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5477worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5478worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5479client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25480client # [ 78.925935] nix-daemon[1104]: accepted connection from pid 1187, user root (trusted)5481client # this derivation will be built:5482client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5483worker1 # [ 79.023210] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:448785484client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5485worker2 # [ 79.013918] nix-grpc-daemon[785]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085486client # copying 1 paths...5487client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5488worker1 # [ 79.039693] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448785489worker1 # [ 79.051187] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5490worker1 # [ 79.155100] nix-daemon[863]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)5491worker1 # [ 79.168173] nix-daemon[863]: reaped child process 1152, status = succeeded5492lb # [ 79.192516] niks3-server[966]: 2026/09/23 04:43:27 INFO Received uploads request method=POST path=/api/pending_closures5493worker1 # [ 79.185796] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.227Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5494worker1 # [ 79.187729] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.228Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5495lb # [ 79.224697] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5496lb # [ 79.231673] niks3-server[966]: 2026/09/23 04:43:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5497lb # [ 79.236803] niks3-server[966]: 2026/09/23 04:43:27 INFO Signed narinfos id=27 count=15498worker1 # [ 79.228317] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.269Z level=INFO msg="Uploading 1 narinfos"5499lb # [ 79.250707] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5500lb # [ 79.259138] niks3-server[966]: 2026/09/23 04:43:27 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5501lb # [ 79.267442] niks3-server[966]: 2026/09/23 04:43:27 INFO Completed upload id=275502lb # [ 79.269223] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5503worker1 # [ 79.253247] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.294Z level=INFO msg="Upload complete. (192ms)"5504worker1 # [ 79.258816] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=1205505worker1 # [ 79.273263] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05506worker1 # [ 79.277291] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15507worker1 # [ 79.282888] nix-daemon[863]: reaped child process 1147, status = succeeded5508worker1 # [ 79.317723] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785509worker1 # [ 79.330667] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785510client # copying 2 paths to 'lb:50051'...5511client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5512worker1 # [ 79.346403] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448785513client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5514worker1 # [ 79.360136] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5515worker1 # [ 79.458769] nix-daemon[863]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)5516worker1 # [ 79.472915] nix-daemon[863]: reaped child process 1161, status = succeeded5517lb # [ 79.494374] niks3-server[966]: 2026/09/23 04:43:27 INFO Received uploads request method=POST path=/api/pending_closures5518lb # [ 79.503537] niks3-server[966]: 2026/09/23 04:43:27 INFO Received uploads request method=POST path=/api/pending_closures5519worker1 # [ 79.497324] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.538Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5520worker1 # [ 79.498807] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.540Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5521worker1 # [ 79.500550] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.542Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5522lb # [ 79.553758] niks3-server[966]: 2026/09/23 04:43:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5523lb # [ 79.555993] niks3-server[966]: 2026/09/23 04:43:27 INFO Signed narinfos id=28 count=15524lb # [ 79.559300] niks3-server[966]: 2026/09/23 04:43:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5525lb # [ 79.560781] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5526lb # [ 79.562766] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5527lb # [ 79.565699] niks3-server[966]: 2026/09/23 04:43:27 INFO Signed narinfos id=29 count=25528worker1 # [ 79.550982] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.591Z level=INFO msg="Uploading 3 narinfos"5529lb # [ 79.567737] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5530lb # [ 79.570940] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5531lb # [ 79.583861] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5532lb # [ 79.586969] niks3-server[966]: 2026/09/23 04:43:27 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5533lb # [ 79.590085] niks3-server[966]: 2026/09/23 04:43:27 INFO Completed upload id=285534lb # [ 79.591331] niks3-server[966]: 2026/09/23 04:43:27 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5535lb # [ 79.594401] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5536lb # [ 79.596556] niks3-server[966]: 2026/09/23 04:43:27 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5537lb # [ 79.599232] niks3-server[966]: 2026/09/23 04:43:27 INFO Completed upload id=295538worker1 # [ 79.583885] nix-grpc-daemon[917]: time=2026-09-23T04:43:27.625Z level=INFO msg="Upload complete. (211ms)"5539worker1 # [ 79.588862] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=2 nar_bytes_in=9765540worker1 # [ 79.594896] nix-grpc-daemon[780]: ts=2026-09-23T04:43:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785541worker1 # [ 79.606168] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5542worker1 # [ 79.615403] nix-daemon[863]: reaped child process 1156, status = succeeded5543worker1 # [ 79.620312] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5544client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5545client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5546client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5547worker1 # [ 80.460186] nix-daemon[863]: reaped child process 1165, status = succeeded5548worker1 # [ 80.541754] nix-daemon[863]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5549worker1 # [ 80.554529] nix-daemon[863]: reaped child process 1184, status = succeeded5550lb # [ 80.579331] niks3-server[966]: 2026/09/23 04:43:28 INFO Received uploads request method=POST path=/api/pending_closures5551worker1 # [ 80.574139] nix-grpc-daemon[917]: time=2026-09-23T04:43:28.615Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5552lb # [ 80.612628] niks3-server[966]: 2026/09/23 04:43:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5553worker1 # [ 80.606816] nix-grpc-daemon[917]: time=2026-09-23T04:43:28.648Z level=INFO msg="Uploading 1 narinfos"5554lb # [ 80.622388] niks3-server[966]: 2026/09/23 04:43:28 INFO Signed narinfos id=30 count=15555lb # [ 80.629589] niks3-server[966]: 2026/09/23 04:43:28 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5556lb # [ 80.636165] niks3-server[966]: 2026/09/23 04:43:28 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5557lb # [ 80.647097] niks3-server[966]: 2026/09/23 04:43:28 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5558lb # [ 80.652966] niks3-server[966]: 2026/09/23 04:43:28 INFO Completed upload id=305559worker1 # [ 80.637482] nix-grpc-daemon[917]: time=2026-09-23T04:43:28.678Z level=INFO msg="Upload complete. (210ms)"5560lb # [ 80.657023] niks3-server[966]: 2026/09/23 04:43:28 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5561worker1 # [ 80.643089] nix-grpc-daemon[780]: ts=2026-09-23T04:43:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15562client # copying 1 paths...5563worker2 # [ 80.642489] nix-grpc-daemon[785]: ts=2026-09-23T04:43:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485085564worker1 # [ 80.659178] nix-daemon[863]: reaped child process 1163, status = succeeded5565worker2 # [ 80.654224] nix-grpc-daemon[785]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5566client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5567worker1 # [ 80.694628] nix-grpc-daemon[780]: ts=2026-09-23T04:43:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785568worker1 # [ 80.699823] nix-grpc-daemon[780]: ts=2026-09-23T04:43:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1205569client # [ 80.796580] nix-daemon[1104]: reaped child process 1190, status = succeeded5570client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.00 seconds)5571client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25572client # [ 80.898106] nix-daemon[1104]: accepted connection from pid 1209, user root (trusted)5573client # this derivation will be built:5574client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5575worker1 # [ 80.990175] nix-grpc-daemon[780]: ts=2026-09-23T04:43:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:448785576client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5577worker1 # [ 80.996126] nix-grpc-daemon[780]: ts=2026-09-23T04:43:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785578worker1 # [ 81.000626] nix-grpc-daemon[917]: time=2026-09-23T04:43:29.042Z level=INFO msg="All 1 paths already cached"5579worker1 # [ 81.015251] nix-grpc-daemon[780]: ts=2026-09-23T04:43:29Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05580worker1 # [ 81.019644] nix-grpc-daemon[780]: ts=2026-09-23T04:43:29Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15581worker2 # [ 81.056598] nix-grpc-daemon[785]: ts=2026-09-23T04:43:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085582worker2 # [ 81.070227] nix-grpc-daemon[785]: ts=2026-09-23T04:43:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085583client # copying 1 paths to 'lb:50051'...5584worker2 # [ 81.089339] nix-grpc-daemon[785]: ts=2026-09-23T04:43:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485085585client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5586worker2 # [ 81.106636] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5587worker2 # [ 81.122653] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5588worker2 # [ 81.225553] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)5589worker2 # [ 81.333841] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 393 ms (attempt 3/5)5590worker2 # [ 81.729609] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 776 ms (attempt 4/5)5591worker2 # [ 82.507777] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5592worker2 # [ 82.510858] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5593worker2 # [ 82.516518] nix-grpc-daemon[785]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5594worker2 # [ 82.637697] nix-daemon[854]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5595worker2 # [ 82.651842] nix-daemon[854]: reaped child process 1220, status = succeeded5596lb # [ 82.688431] niks3-server[966]: 2026/09/23 04:43:30 INFO Received uploads request method=POST path=/api/pending_closures5597worker2 # [ 82.668662] nix-grpc-daemon[867]: time=2026-09-23T04:43:30.489Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5598worker2 # [ 82.670620] nix-grpc-daemon[867]: time=2026-09-23T04:43:30.491Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5599lb # [ 82.727759] niks3-server[966]: 2026/09/23 04:43:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5600lb # [ 82.730619] niks3-server[966]: 2026/09/23 04:43:30 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5601worker2 # [ 82.708731] nix-grpc-daemon[867]: time=2026-09-23T04:43:30.529Z level=INFO msg="Uploading 1 narinfos"5602lb # [ 82.737743] niks3-server[966]: 2026/09/23 04:43:30 INFO Signed narinfos id=31 count=15603lb # [ 82.748532] niks3-server[966]: 2026/09/23 04:43:30 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5604lb # [ 82.755345] niks3-server[966]: 2026/09/23 04:43:30 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5605lb # [ 82.758055] niks3-server[966]: 2026/09/23 04:43:30 INFO Completed upload id=315606worker2 # [ 82.729443] nix-grpc-daemon[867]: time=2026-09-23T04:43:30.550Z level=INFO msg="Upload complete. (188ms)"5607lb # [ 82.763583] niks3-server[966]: 2026/09/23 04:43:30 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5608worker2 # [ 82.733756] nix-grpc-daemon[785]: ts=2026-09-23T04:43:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=1 paths=1 nar_bytes_in=5605609worker2 # [ 82.739735] nix-grpc-daemon[785]: ts=2026-09-23T04:43:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485085610worker2 # [ 82.750253] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5611worker2 # [ 82.755453] nix-daemon[854]: reaped child process 1210, status = succeeded5612worker2 # [ 82.761488] nix-grpc-daemon[785]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5613worker2 # [ 82.768215] nix-daemon[854]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5614client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5615client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5616client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5617worker2 # [ 83.783043] nix-daemon[854]: reaped child process 1225, status = succeeded5618worker2 # [ 83.861714] nix-daemon[854]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5619worker2 # [ 83.874712] nix-daemon[854]: reaped child process 1244, status = succeeded5620lb # [ 83.912482] niks3-server[966]: 2026/09/23 04:43:31 INFO Received uploads request method=POST path=/api/pending_closures5621worker2 # [ 83.891981] nix-grpc-daemon[867]: time=2026-09-23T04:43:31.712Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5622lb # [ 83.942077] niks3-server[966]: 2026/09/23 04:43:31 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5623lb # [ 83.951499] niks3-server[966]: 2026/09/23 04:43:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5624worker2 # [ 83.931341] nix-grpc-daemon[867]: time=2026-09-23T04:43:31.752Z level=INFO msg="Uploading 1 narinfos"5625lb # [ 83.960116] niks3-server[966]: 2026/09/23 04:43:31 INFO Signed narinfos id=32 count=15626lb # [ 83.966652] niks3-server[966]: 2026/09/23 04:43:31 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5627lb # [ 83.979322] niks3-server[966]: 2026/09/23 04:43:31 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5628lb # [ 83.985708] niks3-server[966]: 2026/09/23 04:43:31 INFO Completed upload id=325629worker2 # [ 83.956629] nix-grpc-daemon[867]: time=2026-09-23T04:43:31.777Z level=INFO msg="Upload complete. (180ms)"5630lb # [ 83.988629] niks3-server[966]: 2026/09/23 04:43:31 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5631worker2 # [ 83.959754] nix-grpc-daemon[785]: ts=2026-09-23T04:43:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48508 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15632client # copying 1 paths...5633worker2 # [ 83.967695] nix-grpc-daemon[785]: ts=2026-09-23T04:43:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485085634client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5635worker2 # [ 83.976239] nix-daemon[854]: reaped child process 1222, status = succeeded5636worker1 # [ 84.000823] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785637worker1 # [ 84.011229] nix-grpc-daemon[780]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5638worker1 # [ 84.023608] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1205639client # [ 84.162426] nix-daemon[1104]: reaped child process 1212, status = succeeded5640client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.36 seconds)5641client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25642client # [ 84.268115] nix-daemon[1104]: accepted connection from pid 1231, user root (trusted)5643client # this derivation will be built:5644client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5645worker2 # [ 84.350602] nix-grpc-daemon[785]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:485085646client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5647worker2 # [ 84.356580] nix-grpc-daemon[785]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485085648worker2 # [ 84.361584] nix-grpc-daemon[867]: time=2026-09-23T04:43:32.181Z level=INFO msg="All 1 paths already cached"5649worker1 # [ 84.386222] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05650worker1 # [ 84.391948] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15651worker1 # [ 84.397647] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785652worker1 # [ 84.410206] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448785653worker1 # [ 84.424190] nix-grpc-daemon[917]: time=2026-09-23T04:43:32.465Z level=INFO msg="All 1 paths already cached"5654client # copying 1 paths to 'lb:50051'...5655client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5656worker1 # [ 84.432642] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448785657worker1 # [ 84.443690] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5658worker1 # [ 84.539607] nix-daemon[863]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5659worker1 # [ 84.553540] nix-daemon[863]: reaped child process 1196, status = succeeded5660lb # [ 84.574121] niks3-server[966]: 2026/09/23 04:43:32 INFO Received uploads request method=POST path=/api/pending_closures5661worker1 # [ 84.567900] nix-grpc-daemon[917]: time=2026-09-23T04:43:32.609Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5662worker1 # [ 84.569235] nix-grpc-daemon[917]: time=2026-09-23T04:43:32.609Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5663lb # [ 84.603345] niks3-server[966]: 2026/09/23 04:43:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5664lb # [ 84.609330] niks3-server[966]: 2026/09/23 04:43:32 INFO Signed narinfos id=33 count=15665worker1 # [ 84.598807] nix-grpc-daemon[917]: time=2026-09-23T04:43:32.640Z level=INFO msg="Uploading 1 narinfos"5666lb # [ 84.619927] niks3-server[966]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5667lb # [ 84.627166] niks3-server[966]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5668lb # [ 84.635747] niks3-server[966]: 2026/09/23 04:43:32 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5669lb # [ 84.642950] niks3-server[966]: 2026/09/23 04:43:32 INFO Completed upload id=335670worker1 # [ 84.627285] nix-grpc-daemon[917]: time=2026-09-23T04:43:32.668Z level=INFO msg="Upload complete. (172ms)"5671lb # [ 84.646056] niks3-server[966]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5672worker1 # [ 84.630996] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_in=5605673worker1 # [ 84.635648] nix-grpc-daemon[780]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448785674worker1 # [ 84.644874] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5675lb # [ 84.666034] envoy[904]: conn peer=[2001:db8:1::1]:60326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2138 ms=16305676worker1 # [ 84.651406] nix-daemon[863]: reaped child process 1191, status = succeeded5677lb # [ 84.667513] envoy[904]: conn peer=[2001:db8:1::1]:60340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=585678worker1 # [ 84.653903] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5679lb # [ 84.669277] envoy[904]: conn peer=[2001:db8:1::1]:60344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2370 ms=18885680lb # [ 84.670754] envoy[904]: conn peer=[2001:db8:1::1]:60346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=585681client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5682lb # [ 84.672123] envoy[904]: conn peer=[2001:db8:1::1]:60362 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4221 tx=2049 ms=16885683lb # [ 84.673603] envoy[904]: conn peer=[2001:db8:1::1]:60372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175684client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5685lb # [ 84.675041] envoy[904]: conn peer=[2001:db8:1::1]:53010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=765686lb # [ 84.676391] envoy[904]: conn peer=[2001:db8:1::1]:60386 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2126 ms=30885687client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5688worker1 # [ 85.493716] nix-daemon[863]: reaped child process 1200, status = succeeded5689worker1 # [ 85.562923] nix-daemon[863]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5690worker1 # [ 85.574377] nix-daemon[863]: reaped child process 1219, status = succeeded5691lb # [ 85.598539] niks3-server[966]: 2026/09/23 04:43:33 INFO Received uploads request method=POST path=/api/pending_closures5692worker1 # [ 85.593398] nix-grpc-daemon[917]: time=2026-09-23T04:43:33.634Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5693lb # [ 85.635829] niks3-server[966]: 2026/09/23 04:43:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5694worker1 # [ 85.627742] nix-grpc-daemon[917]: time=2026-09-23T04:43:33.669Z level=INFO msg="Uploading 1 narinfos"5695lb # [ 85.644098] niks3-server[966]: 2026/09/23 04:43:33 INFO Signed narinfos id=34 count=15696lb # [ 85.650230] niks3-server[966]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5697lb # [ 85.656532] niks3-server[966]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5698lb # [ 85.668131] niks3-server[966]: 2026/09/23 04:43:33 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5699lb # [ 85.675362] niks3-server[966]: 2026/09/23 04:43:33 INFO Completed upload id=345700worker1 # [ 85.659865] nix-grpc-daemon[917]: time=2026-09-23T04:43:33.701Z level=INFO msg="Upload complete. (197ms)"5701lb # [ 85.678506] niks3-server[966]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5702client # copying 1 paths...5703worker1 # [ 85.668321] nix-grpc-daemon[780]: ts=2026-09-23T04:43:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15704worker2 # [ 85.660264] nix-grpc-daemon[785]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:485085705worker1 # [ 85.683165] nix-daemon[863]: reaped child process 1198, status = succeeded5706worker2 # [ 85.669204] nix-grpc-daemon[785]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5707client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5708worker1 # [ 85.707968] nix-grpc-daemon[780]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:448785709worker1 # [ 85.713778] nix-grpc-daemon[780]: ts=2026-09-23T04:43:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44878 duration_s=0 paths=1 nar_bytes_out=1205710client # [ 85.807683] nix-daemon[1104]: reaped child process 1234, status = succeeded5711client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.65 seconds)5712(finished: subtest: an input only one worker has still reaches the builder, in 7.52 seconds)5713subtest: upload whose reference the worker lacks is completed from the cache5714client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5715client # this derivation will be built:5716client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5717client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5718client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5719client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5720client # this derivation will be built:5721client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5722client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5723client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.25 seconds)5724client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5725client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5726worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5727worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5728worker2 # [ 86.545474] systemd[1]: Stopping Nix worker-protocol over gRPC...5729worker2 # [ 87.344436] nix-grpc-daemon[785]: ts=2026-09-23T04:43:35Z level=info event=signal_exit5730worker2 # [ 88.657836] nix-grpc-daemon[785]: ts=2026-09-23T04:43:36Z level=info event=scheduler_restarting5731worker1 # [ 88.679661] nix-grpc-daemon[780]: ts=2026-09-23T04:43:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448765732worker1 # [ 88.683507] nix-grpc-daemon[780]: ts=2026-09-23T04:43:36Z level=info event=worker_gone addr=192.168.1.4:500515733worker2 # [ 88.680194] nix-grpc-daemon[785]: ts=2026-09-23T04:43:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5734worker2 # [ 88.704874] nix-daemon[854]: reaped child process 856, status = succeeded5735worker2 # [ 89.479144] nix-grpc-daemon[785]: E0000 00:00:1790138617.299821 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5736worker2 # [ 89.485430] nix-grpc-daemon[785]: E0000 00:00:1790138617.305582 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5737worker2 # [ 89.513115] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5738worker2 # [ 89.516930] systemd[1]: Stopped Nix worker-protocol over gRPC.5739worker2 # [ 89.521459] systemd[1]: nix-grpc-daemon.service: Consumed 2.414s CPU time over 1min 14.340s wall clock time, 61.7M memory peak, 100.6K incoming IP traffic, 226.3K outgoing IP traffic.5740worker2 # [ 89.533614] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5741worker2 # [ 89.537266] systemd[1]: Closed Nix worker-protocol over gRPC.5742worker2 # [ 89.540727] systemd[1]: nix-grpc-daemon.socket: Consumed 338us CPU time over 1min 21.655s wall clock time, 256K memory peak, 97.5K incoming IP traffic, 85.4K outgoing IP traffic.5743lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5744lb: must succeed: curl -sf localhost:9901/clusters5745lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5746lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5747client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25748worker1 # [ 89.764685] nix-grpc-daemon[780]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448785749client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5750worker1 # [ 89.769918] nix-grpc-daemon[780]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:448785751client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5752worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5753worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5754worker2 # [ 89.840448] systemd[1]: Listening on Nix worker-protocol over gRPC.5755worker1 # [ 89.902603] systemd[1]: Stopping Nix worker-protocol over gRPC...5756worker2 # [ 90.508191] systemd[1]: Starting scheduler-token.service...5757worker2 # [ 90.569333] systemd[1]: scheduler-token.service: Deactivated successfully.5758worker2 # [ 90.572602] systemd[1]: Finished scheduler-token.service.5759worker2 # [ 90.575440] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 63ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5760worker2 # [ 90.583363] systemd[1]: Starting Nix worker-protocol over gRPC...5761worker2 # [ 90.664327] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5762worker2 # [ 90.666457] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5763worker2 # [ 90.669806] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15764worker2 # [ 90.674055] nix-grpc-daemon[1269]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5765worker2 # [ 90.675552] nix-grpc-daemon[1269]: E0000 00:00:1790138618.494924 1269 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5766worker2 # [ 90.681424] nix-grpc-daemon[1269]: E0000 00:00:1790138618.500703 1269 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5767worker2 # [ 90.685112] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5768worker2 # [ 90.686841] systemd[1]: Started Nix worker-protocol over gRPC.5769worker2 # [ 90.694061] nix-grpc-daemon[1269]: E0000 00:00:1790138618.514974 1312 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5770worker2 # [ 90.709365] nix-grpc-daemon[1269]: E0000 00:00:1790138618.530292 1312 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5771worker2 # [ 90.714053] nix-grpc-daemon[1269]: E0000 00:00:1790138618.532684 1312 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5772worker2 # [ 90.723900] nix-grpc-daemon[1269]: E0000 00:00:1790138618.544840 1312 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5773worker2 # [ 90.733464] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:38Z level=info event=scheduler_connected addr=lb:500515774worker1 # [ 90.749445] nix-grpc-daemon[780]: ts=2026-09-23T04:43:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:448765775worker1 # [ 90.754461] nix-grpc-daemon[780]: ts=2026-09-23T04:43:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05776worker1 # [ 90.847123] nix-grpc-daemon[780]: ts=2026-09-23T04:43:38Z level=info event=signal_exit5777lb # [ 91.687983] niks3-server[966]: 2026/09/23 04:43:39 INFO lead: released remote=192.168.1.3:557165778worker1 # [ 92.670836] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=scheduler_restarting5779worker2 # [ 92.659882] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:40Z level=info event=scheduler_restarting addr=lb:500515780worker1 # [ 92.680848] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448765781worker1 # [ 92.684888] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=worker_gone addr=192.168.1.4:500515782worker1 # [ 92.687895] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=scheduler_restarting addr=lb:500515783worker1 # [ 92.690964] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448765784worker1 # [ 92.694572] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=worker_gone addr=192.168.1.3:500515785worker1 # [ 92.714535] nix-daemon[863]: reaped child process 865, status = succeeded5786worker2 # [ 92.860808] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:40Z level=info event=scheduler_connected addr=lb:500515787worker2 # [ 92.864805] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5788worker1 # [ 92.892371] nix-grpc-daemon[780]: ts=2026-09-23T04:43:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5789worker1 # [ 93.005258] nix-grpc-daemon[780]: E0000 00:00:1790138621.046246 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5790worker1 # [ 93.010176] nix-grpc-daemon[780]: E0000 00:00:1790138621.046498 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5791worker2 # [ 93.065875] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:40Z level=info event=scheduler_connected addr=lb:500515792worker2 # [ 93.070230] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5793worker2 # [ 93.271347] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_connected addr=lb:500515794worker2 # [ 93.275277] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5795worker1 # [ 93.482403] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5796worker1 # [ 93.486255] systemd[1]: Stopped Nix worker-protocol over gRPC.5797worker2 # [ 93.476429] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_connected addr=lb:500515798worker1 # [ 93.490616] systemd[1]: nix-grpc-daemon.service: Consumed 2.041s CPU time over 1min 18.834s wall clock time, 64.7M memory peak, 94.8K incoming IP traffic, 201.4K outgoing IP traffic.5799worker2 # [ 93.480313] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5800worker1 # [ 93.502420] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5801worker1 # [ 93.506624] systemd[1]: Closed Nix worker-protocol over gRPC.5802worker1 # [ 93.509970] systemd[1]: nix-grpc-daemon.socket: Consumed 328us CPU time over 1min 25.770s wall clock time, 256K memory peak, 114.8K incoming IP traffic, 99.1K outgoing IP traffic.5803lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5804lb: must succeed: curl -sf localhost:9901/clusters5805lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5806lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5807client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25808worker2 # [ 93.682973] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_connected addr=lb:500515809worker2 # [ 93.686577] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5810worker2 # [ 93.711946] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584305811worker2 # [ 93.723497] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5812worker2 # [ 93.740838] nix-grpc-daemon[1269]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5813worker2 # [ 93.845212] nix-grpc-daemon[1269]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 185 ms (attempt 2/5)5814worker2 # [ 93.884278] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 93.888505] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker2 # [ 94.032502] nix-grpc-daemon[1269]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 3/5)5817worker2 # [ 94.089735] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_connected addr=lb:500515818worker2 # [ 94.093654] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5819worker2 # [ 94.183617] nix-grpc-daemon[1269]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 601 ms (attempt 4/5)5820worker2 # [ 94.295234] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_connected addr=lb:500515821worker2 # [ 94.299167] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 94.500300] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 94.504221] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824lb # [ 94.667312] envoy[904]: conn peer=[2001:db8:1::1]:53030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=165825lb # [ 94.671380] envoy[904]: conn peer=[2001:db8:1::1]:53020 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3729 tx=2080 ms=13615826lb # [ 94.675080] envoy[904]: conn peer=[2001:db8:1::4]:37098 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=56586 tx=3268 ms=732945827lb # [ 94.678831] envoy[904]: conn peer=[2001:db8:1::1]:53044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165828lb # [ 94.682300] envoy[904]: conn peer=[2001:db8:1::3]:60506 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15858 tx=3392 ms=780135829worker2 # [ 94.708699] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 94.711924] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831worker2 # [ 94.787230] nix-grpc-daemon[1269]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5832worker2 # [ 94.790389] nix-grpc-daemon[1269]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5833client # copying 2 paths...5834client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5835worker2 # [ 94.812442] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584305836client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5837worker2 # [ 94.826685] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5838worker2 # [ 94.848439] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=niks3_push_started pid=13305839worker2 # [ 94.908722] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_connected addr=lb:500515840worker2 # [ 94.910834] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker2 # [ 94.957498] nix-grpc-daemon[1330]: time=2026-09-23T04:43:42.776Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5842worker2 # [ 95.043804] nix-daemon[854]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5843worker2 # [ 95.055503] nix-daemon[854]: reaped child process 1346, status = succeeded5844lb # [ 95.093397] niks3-server[966]: 2026/09/23 04:43:43 INFO Received uploads request method=POST path=/api/pending_closures5845lb # [ 95.104589] niks3-server[966]: 2026/09/23 04:43:43 INFO Received uploads request method=POST path=/api/pending_closures5846worker2 # [ 95.083434] nix-grpc-daemon[1330]: time=2026-09-23T04:43:42.904Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5847worker2 # [ 95.085032] nix-grpc-daemon[1330]: time=2026-09-23T04:43:42.906Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5848worker2 # [ 95.086926] nix-grpc-daemon[1330]: time=2026-09-23T04:43:42.907Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5849worker2 # [ 95.110950] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_connected addr=lb:500515850worker2 # [ 95.112735] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5851lb # [ 95.161152] niks3-server[966]: 2026/09/23 04:43:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5852lb # [ 95.162898] niks3-server[966]: 2026/09/23 04:43:43 INFO Signed narinfos id=35 count=15853lb # [ 95.165063] niks3-server[966]: 2026/09/23 04:43:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5854lb # [ 95.170474] niks3-server[966]: 2026/09/23 04:43:43 INFO Signed narinfos id=36 count=25855worker2 # [ 95.142449] nix-grpc-daemon[1330]: time=2026-09-23T04:43:42.962Z level=INFO msg="Uploading 3 narinfos"5856lb # [ 95.180680] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5857lb # [ 95.184481] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5858lb # [ 95.188198] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5859lb # [ 95.190875] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5860lb # [ 95.202792] niks3-server[966]: 2026/09/23 04:43:43 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5861lb # [ 95.205541] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5862lb # [ 95.207280] niks3-server[966]: 2026/09/23 04:43:43 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5863lb # [ 95.211541] niks3-server[966]: 2026/09/23 04:43:43 INFO Completed upload id=355864lb # [ 95.213293] niks3-server[966]: 2026/09/23 04:43:43 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5865lb # [ 95.216514] niks3-server[966]: 2026/09/23 04:43:43 INFO Completed upload id=365866worker2 # [ 95.188109] nix-grpc-daemon[1330]: time=2026-09-23T04:43:43.008Z level=INFO msg="Upload complete. (231ms)"5867worker2 # [ 95.193668] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=2 nar_bytes_in=3125868worker2 # [ 95.207217] nix-daemon[854]: reaped child process 1328, status = succeeded5869client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.67 seconds)5870worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5871worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5872worker1 # [ 95.310903] systemd[1]: Listening on Nix worker-protocol over gRPC.5873worker1 # [ 95.317918] systemd[1]: Starting Nix worker-protocol over gRPC...5874worker2 # [ 95.314363] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:43Z level=info event=scheduler_connected addr=lb:500515875worker2 # [ 95.318224] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5876worker1 # [ 95.404745] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5877worker1 # [ 95.406778] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5878worker1 # [ 95.410052] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15879worker1 # [ 95.414375] nix-grpc-daemon[1243]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5880worker1 # [ 95.417179] nix-grpc-daemon[1243]: E0000 00:00:1790138623.455780 1243 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5881worker1 # [ 95.422901] nix-grpc-daemon[1243]: E0000 00:00:1790138623.461262 1243 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5882worker1 # [ 95.426153] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5883worker1 # [ 95.429098] systemd[1]: Started Nix worker-protocol over gRPC.5884lb # [ 95.446718] niks3-server[966]: 2026/09/23 04:43:43 INFO lead: acquired remote=192.168.1.3:594685885worker1 # [ 95.433081] nix-grpc-daemon[1243]: E0000 00:00:1790138623.473292 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5886worker1 # [ 95.436126] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=scheduler_take_over5887lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5888lb: must succeed: curl -sf localhost:9901/clusters5889worker1 # [ 95.446757] nix-grpc-daemon[1243]: E0000 00:00:1790138623.488220 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5890worker1 # [ 95.450172] nix-grpc-daemon[1243]: E0000 00:00:1790138623.490374 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5891worker1 # [ 95.460760] nix-grpc-daemon[1243]: E0000 00:00:1790138623.502284 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5892worker1 # [ 95.472744] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=scheduler_connected addr=lb:500515893worker1 # [ 95.474353] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5894lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5895lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5896(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.70 seconds)5897subtest: a niks3 restart does not cost the leader its role5898lb: must succeed: curl -sf localhost:9901/clusters5899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5900lb: must succeed: curl -sf localhost:9901/clusters5901worker2 # [ 95.519222] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:43Z level=info event=scheduler_connected addr=lb:500515902worker1 # [ 95.540969] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=oidc_jwks provider=mock keys=15903lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5904worker1 # [ 95.543784] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:596285905worker1 # [ 95.546850] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05906lb # [ 95.582280] systemd[1]: Stopping niks3 server...5907lb # [ 95.583460] niks3-server[966]: 2026/09/23 04:43:43 INFO Shutdown signal received, draining in-flight requests timeout=10s5908lb # [ 95.588671] niks3-server[966]: 2026/09/23 04:43:43 INFO lead: released remote=192.168.1.3:594685909worker1 # [ 95.572086] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=scheduler_lock_lost status=2005910worker2 # [ 95.559992] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:43Z level=info event=scheduler_lock_lost status=2005911lb # [ 95.602400] systemd[1]: niks3.service: Deactivated successfully.5912lb # [ 95.603918] systemd[1]: Stopped niks3 server.5913lb # [ 95.605254] systemd[1]: niks3.service: Consumed 781ms CPU time over 1min 17.144s wall clock time, 28.9M memory peak, 97.2K incoming IP traffic, 139.6K outgoing IP traffic.5914lb # [ 95.613350] systemd[1]: Starting niks3 server...5915lb # [ 95.670692] niks3-server[1103]: 2026/09/23 04:43:43 goose: no migrations to run. current version: 202609200000005916lb # [ 95.677688] niks3-server[1103]: 2026/09/23 04:43:43 OK 1_commit_pending_closure.sql (4.91ms)5917lb # [ 95.681561] niks3-server[1103]: 2026/09/23 04:43:43 OK 2_object_stats_trigger.sql (3.46ms)5918lb # [ 95.682685] niks3-server[1103]: 2026/09/23 04:43:43 goose: up to current file version: 25919lb # [ 95.684206] niks3-server[1103]: 2026/09/23 04:43:43 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5920worker1 # [ 95.675391] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=scheduler_connected addr=lb:500515921lb # [ 95.695977] niks3-server[1103]: 2026/09/23 04:43:43 INFO Read proxy enabled — serving cache objects from S35922lb # [ 95.697242] niks3-server[1103]: 2026/09/23 04:43:43 INFO Using socket-activated listener address=0.0.0.0:57515923worker1 # [ 95.679950] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:596285924lb # [ 95.699593] niks3-server[1103]: 2026/09/23 04:43:43 INFO systemd watchdog enabled interval=15s5925lb # [ 95.701131] systemd[1]: Started niks3 server.5926lb # [ 95.703062] niks3-server[1103]: 2026/09/23 04:43:43 INFO Starting HTTP server address=0.0.0.0:57515927worker1 # [ 95.683942] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05928lb: waiting for unit niks3.service5929lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5930lb: must succeed: curl -sf localhost:9901/clusters5931lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5932worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5933worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5934lb: must succeed: curl -sf localhost:9901/clusters5935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5936lb: must succeed: curl -sf localhost:9901/clusters5937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5938worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5939worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5940client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25941worker1 # [ 96.030074] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:596425942worker1 # [ 96.041128] nix-daemon[863]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5943client # don't know how to build these paths:5944client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5945worker1 # [ 96.056583] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:596425946worker1 # [ 96.067178] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5947worker1 # [ 96.169631] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5948worker1 # [ 96.217357] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 3/5)5949worker1 # [ 96.411408] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 639 ms (attempt 4/5)5950lb # [ 96.595461] niks3-server[1103]: 2026/09/23 04:43:44 INFO lead: acquired remote=192.168.1.3:594705951worker1 # [ 97.052258] nix-grpc-daemon[1243]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5952worker1 # [ 97.055637] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5953worker1 # [ 97.093787] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:45Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05954worker1 # [ 97.098183] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:45Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15955worker2 # [ 97.127977] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584305956worker2 # [ 97.142408] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584305957client # copying 1 paths to 'lb:50051'...5958client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5959worker2 # [ 97.155488] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584305960worker2 # [ 97.167101] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5961worker2 # [ 97.275184] nix-daemon[854]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5962worker2 # [ 97.288243] nix-daemon[854]: reaped child process 1358, status = succeeded5963lb # [ 97.327257] niks3-server[1103]: 2026/09/23 04:43:45 INFO Received uploads request method=POST path=/api/pending_closures5964worker2 # [ 97.308608] nix-grpc-daemon[1330]: time=2026-09-23T04:43:45.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5965worker2 # [ 97.311042] nix-grpc-daemon[1330]: time=2026-09-23T04:43:45.131Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5966lb # [ 97.371146] niks3-server[1103]: 2026/09/23 04:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5967lb # [ 97.385872] niks3-server[1103]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5968lb # [ 97.400165] niks3-server[1103]: 2026/09/23 04:43:45 INFO Signed narinfos id=37 count=15969worker2 # [ 97.373632] nix-grpc-daemon[1330]: time=2026-09-23T04:43:45.193Z level=INFO msg="Uploading 1 narinfos"5970lb # [ 97.405919] niks3-server[1103]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5971lb # [ 97.411847] niks3-server[1103]: 2026/09/23 04:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5972lb # [ 97.417527] niks3-server[1103]: 2026/09/23 04:43:45 INFO Completed upload id=375973worker2 # [ 97.389130] nix-grpc-daemon[1330]: time=2026-09-23T04:43:45.209Z level=INFO msg="Upload complete. (210ms)"5974lb # [ 97.421297] niks3-server[1103]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5975worker2 # [ 97.394516] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=5365976worker2 # [ 97.401332] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584305977worker2 # [ 97.412546] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5978worker2 # [ 97.422689] nix-daemon[854]: reaped child process 1353, status = succeeded5979worker2 # [ 97.427559] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5980client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5981client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5982client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5983client # farm-a> LOG-farm-a-after-niks3-restart5984worker2 # [ 98.463838] nix-daemon[854]: reaped child process 1362, status = succeeded5985worker2 # [ 98.549366] nix-daemon[854]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5986worker2 # [ 98.561773] nix-daemon[854]: reaped child process 1381, status = succeeded5987lb # [ 98.601159] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received uploads request method=POST path=/api/pending_closures5988worker2 # [ 98.581425] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5989worker2 # [ 98.582947] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.403Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5990lb # [ 98.644708] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5991worker2 # [ 98.625950] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.446Z level=INFO msg="Uploading 1 narinfos"5992lb # [ 98.654916] niks3-server[1103]: 2026/09/23 04:43:46 INFO Signed narinfos id=38 count=15993lb # [ 98.663157] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5994lb # [ 98.667564] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5995lb # [ 98.678428] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5996lb # [ 98.680298] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5997lb # [ 98.684872] niks3-server[1103]: 2026/09/23 04:43:46 INFO Completed upload id=385998worker2 # [ 98.656335] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.477Z level=INFO msg="Upload complete. (221ms)"5999lb # [ 98.686982] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6000worker2 # [ 98.662932] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16001worker1 # [ 98.683967] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:46Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06002worker1 # [ 98.688137] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:46Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16003worker2 # [ 98.678309] nix-daemon[854]: reaped child process 1360, status = succeeded6004worker2 # [ 98.714213] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306005worker2 # [ 98.728433] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306006worker2 # [ 98.742485] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.563Z level=INFO msg="All 1 paths already cached"6007client # copying 1 paths to 'lb:50051'...6008client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6009worker2 # [ 98.751321] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306010worker2 # [ 98.759557] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6011worker2 # [ 98.856100] nix-daemon[854]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)6012worker2 # [ 98.868484] nix-daemon[854]: reaped child process 1388, status = succeeded6013lb # [ 98.904979] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received uploads request method=POST path=/api/pending_closures6014worker2 # [ 98.884403] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.705Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6015worker2 # [ 98.885972] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.706Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6016lb # [ 98.937527] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6017worker2 # [ 98.915060] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.736Z level=INFO msg="Uploading 1 narinfos"6018lb # [ 98.943916] niks3-server[1103]: 2026/09/23 04:43:46 INFO Signed narinfos id=39 count=16019lb # [ 98.953403] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6020lb # [ 98.959144] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6021lb # [ 98.972129] niks3-server[1103]: 2026/09/23 04:43:46 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6022lb # [ 98.984132] niks3-server[1103]: 2026/09/23 04:43:46 INFO Completed upload id=396023worker2 # [ 98.955452] nix-grpc-daemon[1330]: time=2026-09-23T04:43:46.776Z level=INFO msg="Upload complete. (182ms)"6024lb # [ 98.985368] niks3-server[1103]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6025worker2 # [ 98.959639] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=6566026worker2 # [ 98.964435] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306027worker2 # [ 98.973694] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6028worker2 # [ 98.983132] nix-daemon[854]: reaped child process 1383, status = succeeded6029worker2 # [ 98.987729] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6030client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6031client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6032client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6033client # farm-b> LOG-farm-b-after-niks3-restart6034worker2 # [ 99.803167] nix-daemon[854]: reaped child process 1392, status = succeeded6035worker2 # [ 99.876796] nix-daemon[854]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)6036worker2 # [ 99.889809] nix-daemon[854]: reaped child process 1411, status = succeeded6037lb # [ 99.925668] niks3-server[1103]: 2026/09/23 04:43:47 INFO Received uploads request method=POST path=/api/pending_closures6038worker2 # [ 99.906157] nix-grpc-daemon[1330]: time=2026-09-23T04:43:47.726Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6039worker2 # [ 99.907622] nix-grpc-daemon[1330]: time=2026-09-23T04:43:47.728Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6040lb # [ 99.969971] niks3-server[1103]: 2026/09/23 04:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6041lb # [ 99.976789] niks3-server[1103]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6042lb # [ 99.984098] niks3-server[1103]: 2026/09/23 04:43:47 INFO Signed narinfos id=40 count=16043worker2 # [ 99.955691] nix-grpc-daemon[1330]: time=2026-09-23T04:43:47.776Z level=INFO msg="Uploading 1 narinfos"6044lb # [ 99.989400] niks3-server[1103]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6045lb # [ 99.996376] niks3-server[1103]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6046lb # [ 100.007134] niks3-server[1103]: 2026/09/23 04:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6047lb # [ 100.011093] niks3-server[1103]: 2026/09/23 04:43:47 INFO Completed upload id=406048lb # [ 100.012928] niks3-server[1103]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6049worker2 # [ 99.983774] nix-grpc-daemon[1330]: time=2026-09-23T04:43:47.804Z level=INFO msg="Upload complete. (189ms)"6050worker2 # [ 99.991279] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16051worker1 # [ 100.010839] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:48Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06052worker1 # [ 100.015339] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:48Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16053worker2 # [ 100.006479] nix-daemon[854]: reaped child process 1390, status = succeeded6054worker2 # [ 100.041172] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306055worker2 # [ 100.054127] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306056worker2 # [ 100.067448] nix-grpc-daemon[1330]: time=2026-09-23T04:43:47.888Z level=INFO msg="All 2 paths already cached"6057client # copying 1 paths to 'lb:50051'...6058client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6059worker2 # [ 100.076793] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306060worker2 # [ 100.087109] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6061worker2 # [ 100.181957] nix-daemon[854]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6062worker2 # [ 100.194484] nix-daemon[854]: reaped child process 1418, status = succeeded6063lb # [ 100.230970] niks3-server[1103]: 2026/09/23 04:43:48 INFO Received uploads request method=POST path=/api/pending_closures6064worker2 # [ 100.210317] nix-grpc-daemon[1330]: time=2026-09-23T04:43:48.031Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6065worker2 # [ 100.211820] nix-grpc-daemon[1330]: time=2026-09-23T04:43:48.032Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6066lb # [ 100.263172] niks3-server[1103]: 2026/09/23 04:43:48 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6067lb # [ 100.270688] niks3-server[1103]: 2026/09/23 04:43:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6068worker2 # [ 100.249404] nix-grpc-daemon[1330]: time=2026-09-23T04:43:48.070Z level=INFO msg="Uploading 1 narinfos"6069lb # [ 100.278206] niks3-server[1103]: 2026/09/23 04:43:48 INFO Signed narinfos id=41 count=16070lb # [ 100.285068] niks3-server[1103]: 2026/09/23 04:43:48 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6071lb # [ 100.294236] niks3-server[1103]: 2026/09/23 04:43:48 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6072lb # [ 100.301476] niks3-server[1103]: 2026/09/23 04:43:48 INFO Completed upload id=416073worker2 # [ 100.272458] nix-grpc-daemon[1330]: time=2026-09-23T04:43:48.093Z level=INFO msg="Upload complete. (174ms)"6074lb # [ 100.306064] niks3-server[1103]: 2026/09/23 04:43:48 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6075worker2 # [ 100.276503] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=7846076worker2 # [ 100.282411] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306077worker2 # [ 100.292648] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6078worker2 # [ 100.299671] nix-daemon[854]: reaped child process 1413, status = succeeded6079worker2 # [ 100.304740] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6080client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6081client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6082client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6083client # farm-top> LOG-farm-top-after-niks3-restart6084worker2 # [ 101.545342] nix-daemon[854]: reaped child process 1422, status = succeeded6085worker2 # [ 101.626158] nix-daemon[854]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6086worker2 # [ 101.638511] nix-daemon[854]: reaped child process 1441, status = succeeded6087lb # [ 101.675864] niks3-server[1103]: 2026/09/23 04:43:49 INFO Received uploads request method=POST path=/api/pending_closures6088worker2 # [ 101.656360] nix-grpc-daemon[1330]: time=2026-09-23T04:43:49.477Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6089worker2 # [ 101.657848] nix-grpc-daemon[1330]: time=2026-09-23T04:43:49.478Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6090lb # [ 101.719625] niks3-server[1103]: 2026/09/23 04:43:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6091lb # [ 101.721141] niks3-server[1103]: 2026/09/23 04:43:49 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6092lb # [ 101.723054] niks3-server[1103]: 2026/09/23 04:43:49 INFO Signed narinfos id=42 count=16093worker2 # [ 101.694416] nix-grpc-daemon[1330]: time=2026-09-23T04:43:49.515Z level=INFO msg="Uploading 1 narinfos"6094lb # [ 101.726714] niks3-server[1103]: 2026/09/23 04:43:49 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6095lb # [ 101.732194] niks3-server[1103]: 2026/09/23 04:43:49 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6096lb # [ 101.737247] niks3-server[1103]: 2026/09/23 04:43:49 INFO Completed upload id=426097worker2 # [ 101.708523] nix-grpc-daemon[1330]: time=2026-09-23T04:43:49.529Z level=INFO msg="Upload complete. (195ms)"6098lb # [ 101.739516] niks3-server[1103]: 2026/09/23 04:43:49 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6099lb # [ 101.742311] niks3-server[1103]: 2026/09/23 04:43:49 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6100worker2 # [ 101.713641] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16101worker2 # [ 101.725753] nix-daemon[854]: reaped child process 1420, status = succeeded6102client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.89 seconds)6103client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6104client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6105(finished: subtest: a niks3 restart does not cost the leader its role, in 6.39 seconds)6106subtest: scheduler down: the other node takes the lock and keeps it6107lb: must succeed: curl -sf localhost:9901/clusters6108lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6109lb: must succeed: curl -sf localhost:9901/clusters6110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6111lb: must succeed: curl -sf localhost:9901/clusters6112lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6113lb: must succeed: curl -sf localhost:9901/clusters6114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6115worker1 # [ 102.058143] systemd[1]: Stopping Nix worker-protocol over gRPC...6116worker1 # [ 102.425236] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:50Z level=info event=signal_exit6117lb # [ 102.597688] niks3-server[1103]: 2026/09/23 04:43:50 INFO lead: released remote=192.168.1.3:594706118worker2 # [ 103.572084] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:51Z level=info event=scheduler_restarting addr=lb:500516119worker1 # [ 103.589127] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=scheduler_restarting6120worker1 # [ 103.592369] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=scheduler_restarting addr=lb:500516121worker1 # [ 103.596253] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:596286122worker1 # [ 103.600271] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=worker_gone addr=192.168.1.3:500516123worker1 # [ 103.603503] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:596286124worker1 # [ 103.607223] nix-grpc-daemon[1243]: ts=2026-09-23T04:43:51Z level=info event=worker_gone addr=192.168.1.4:500516125worker1 # [ 103.618776] nix-daemon[863]: reaped child process 1304, status = succeeded6126worker2 # [ 103.771995] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:51Z level=info event=scheduler_connected addr=lb:500516127worker2 # [ 103.776270] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6128worker2 # [ 103.977342] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:51Z level=info event=scheduler_connected addr=lb:500516129worker2 # [ 103.981329] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6130worker2 # [ 104.182493] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516131worker2 # [ 104.186675] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6132worker2 # [ 104.387979] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516133worker2 # [ 104.392259] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6134worker1 # [ 104.500774] nix-grpc-daemon[1243]: E0000 00:00:1790138632.542063 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6135worker1 # [ 104.506295] nix-grpc-daemon[1243]: E0000 00:00:1790138632.547444 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6136worker2 # [ 104.594403] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516137worker2 # [ 104.598272] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6138lb # [ 104.668315] envoy[904]: conn peer=[2001:db8:1::1]:46168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=282 ms=14996139lb # [ 104.672273] envoy[904]: conn peer=[2001:db8:1::1]:46176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4461 ms=57156140lb # [ 104.675820] envoy[904]: conn peer=[2001:db8:1::3]:60644 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=751 tx=265 ms=83216141worker2 # [ 104.799327] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516142worker2 # [ 104.803312] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6143worker2 # [ 105.004208] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516144worker2 # [ 105.008279] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6145worker2 # [ 105.209069] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516146worker2 # [ 105.213224] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6147worker2 # [ 105.414220] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516148worker2 # [ 105.418281] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6149worker2 # [ 105.620510] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516150worker2 # [ 105.624194] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6151worker1 # [ 105.815581] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6152worker1 # [ 105.820578] systemd[1]: Stopped Nix worker-protocol over gRPC.6153worker1 # [ 105.823577] systemd[1]: nix-grpc-daemon.service: Consumed 210ms CPU time over 10.501s wall clock time, 13.8M memory peak, 10.1K incoming IP traffic, 14.7K outgoing IP traffic.6154worker1 # [ 105.835526] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6155worker1 # [ 105.839694] systemd[1]: Closed Nix worker-protocol over gRPC.6156worker2 # [ 105.825463] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516157worker2 # [ 105.829532] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6158worker1 # [ 105.842573] systemd[1]: nix-grpc-daemon.socket: Consumed 518us CPU time over 10.532s wall clock time, 256K memory peak, 23.4K incoming IP traffic, 17.9K outgoing IP traffic.6159worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6160worker2 # [ 106.030625] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516161worker2 # [ 106.034755] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6162worker2 # [ 106.235786] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_connected addr=lb:500516163worker2 # [ 106.239806] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6164worker2 # [ 106.440804] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_connected addr=lb:500516165worker2 # [ 106.444816] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6166worker2 # [ 106.568333] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_take_over6167lb # [ 106.597352] niks3-server[1103]: 2026/09/23 04:43:54 INFO lead: acquired remote=192.168.1.4:609846168worker2 # [ 106.646710] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_connected addr=lb:500516169worker2 # [ 106.650505] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6170worker2 # [ 106.851685] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=scheduler_connected addr=lb:500516171worker2 # [ 106.868161] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=oidc_jwks provider=mock keys=16172worker2 # [ 106.872182] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:444286173worker2 # [ 106.876076] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06174worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.10 seconds)6175worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6176worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6177client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26178worker2 # [ 107.099439] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584306179client # don't know how to build these paths:6180client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6181worker2 # [ 107.108433] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306182worker2 # [ 107.132455] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06183worker2 # [ 107.137594] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16184worker2 # [ 107.177156] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306185worker2 # [ 107.189877] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306186client # copying 1 paths to 'lb:50051'...6187client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6188worker2 # [ 107.202830] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306189worker2 # [ 107.214456] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6190worker2 # [ 107.313542] nix-daemon[854]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)6191worker2 # [ 107.326304] nix-daemon[854]: reaped child process 1465, status = succeeded6192lb # [ 107.362162] niks3-server[1103]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures6193worker2 # [ 107.343640] nix-grpc-daemon[1330]: time=2026-09-23T04:43:55.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6194worker2 # [ 107.345596] nix-grpc-daemon[1330]: time=2026-09-23T04:43:55.166Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6195lb # [ 107.393716] niks3-server[1103]: 2026/09/23 04:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6196lb # [ 107.400093] niks3-server[1103]: 2026/09/23 04:43:55 INFO Signed narinfos id=43 count=16197worker2 # [ 107.374365] nix-grpc-daemon[1330]: time=2026-09-23T04:43:55.195Z level=INFO msg="Uploading 1 narinfos"6198lb # [ 107.409955] niks3-server[1103]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6199lb # [ 107.416863] niks3-server[1103]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6200lb # [ 107.427094] niks3-server[1103]: 2026/09/23 04:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6201lb # [ 107.432967] niks3-server[1103]: 2026/09/23 04:43:55 INFO Completed upload id=436202worker2 # [ 107.404350] nix-grpc-daemon[1330]: time=2026-09-23T04:43:55.225Z level=INFO msg="Upload complete. (178ms)"6203worker2 # [ 107.408118] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=5206204lb # [ 107.440185] niks3-server[1103]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6205worker2 # [ 107.413331] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306206worker2 # [ 107.426941] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6207worker2 # [ 107.430079] nix-daemon[854]: reaped child process 1460, status = succeeded6208worker2 # [ 107.434636] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6209client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6210client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6211client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6212client # farm-a> LOG-farm-a-on-standby6213worker2 # [ 108.649543] nix-daemon[854]: reaped child process 1469, status = succeeded6214worker2 # [ 108.724071] nix-daemon[854]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6215worker2 # [ 108.738255] nix-daemon[854]: reaped child process 1488, status = succeeded6216lb # [ 108.773826] niks3-server[1103]: 2026/09/23 04:43:56 INFO Received uploads request method=POST path=/api/pending_closures6217worker2 # [ 108.753618] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.574Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6218worker2 # [ 108.755162] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.576Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6219lb # [ 108.815664] niks3-server[1103]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6220lb # [ 108.823608] niks3-server[1103]: 2026/09/23 04:43:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6221lb # [ 108.833463] niks3-server[1103]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6222lb # [ 108.847805] niks3-server[1103]: 2026/09/23 04:43:56 INFO Signed narinfos id=44 count=16223lb # [ 108.849729] niks3-server[1103]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6224worker2 # [ 108.819381] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.640Z level=INFO msg="Uploading 1 narinfos"6225lb # [ 108.855491] niks3-server[1103]: 2026/09/23 04:43:56 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6226lb # [ 108.858260] niks3-server[1103]: 2026/09/23 04:43:56 INFO Completed upload id=446227worker2 # [ 108.829652] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.650Z level=INFO msg="Upload complete. (188ms)"6228worker2 # [ 108.836647] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16229worker2 # [ 108.850580] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06230worker2 # [ 108.854199] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16231worker2 # [ 108.864081] nix-daemon[854]: reaped child process 1467, status = succeeded6232worker2 # [ 108.894432] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306233worker2 # [ 108.906500] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306234worker2 # [ 108.920845] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.741Z level=INFO msg="All 1 paths already cached"6235client # copying 1 paths to 'lb:50051'...6236client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6237worker2 # [ 108.929599] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306238worker2 # [ 108.937882] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6239worker2 # [ 109.033753] nix-daemon[854]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)6240worker2 # [ 109.046792] nix-daemon[854]: reaped child process 1497, status = succeeded6241lb # [ 109.082390] niks3-server[1103]: 2026/09/23 04:43:57 INFO Received uploads request method=POST path=/api/pending_closures6242worker2 # [ 109.063913] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6243worker2 # [ 109.065483] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.886Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6244lb # [ 109.119179] niks3-server[1103]: 2026/09/23 04:43:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6245worker2 # [ 109.097056] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.917Z level=INFO msg="Uploading 1 narinfos"6246lb # [ 109.126115] niks3-server[1103]: 2026/09/23 04:43:57 INFO Signed narinfos id=45 count=16247lb # [ 109.134717] niks3-server[1103]: 2026/09/23 04:43:57 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6248lb # [ 109.145836] niks3-server[1103]: 2026/09/23 04:43:57 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6249lb # [ 109.150895] niks3-server[1103]: 2026/09/23 04:43:57 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6250lb # [ 109.160907] niks3-server[1103]: 2026/09/23 04:43:57 INFO Completed upload id=456251worker2 # [ 109.132334] nix-grpc-daemon[1330]: time=2026-09-23T04:43:56.953Z level=INFO msg="Upload complete. (182ms)"6252lb # [ 109.163827] niks3-server[1103]: 2026/09/23 04:43:57 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6253worker2 # [ 109.137346] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=6406254worker2 # [ 109.143446] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306255worker2 # [ 109.154471] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6256worker2 # [ 109.163851] nix-daemon[854]: reaped child process 1492, status = succeeded6257worker2 # [ 109.171628] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6258client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6259client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6260client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6261client # farm-b> LOG-farm-b-on-standby6262worker2 # [ 110.412256] nix-daemon[854]: reaped child process 1501, status = succeeded6263worker2 # [ 110.490444] nix-daemon[854]: accepted connection from pid 1517, user nix-grpc-daemon (trusted)6264worker2 # [ 110.501903] nix-daemon[854]: reaped child process 1520, status = succeeded6265lb # [ 110.538779] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures6266worker2 # [ 110.518549] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6267worker2 # [ 110.520102] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.341Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6268lb # [ 110.583259] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6269worker2 # [ 110.565869] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.386Z level=INFO msg="Uploading 1 narinfos"6270lb # [ 110.594891] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6271lb # [ 110.599276] niks3-server[1103]: 2026/09/23 04:43:58 INFO Signed narinfos id=46 count=16272lb # [ 110.607787] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6273lb # [ 110.611563] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6274lb # [ 110.624052] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6275lb # [ 110.627079] niks3-server[1103]: 2026/09/23 04:43:58 INFO Completed upload id=466276worker2 # [ 110.600199] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.421Z level=INFO msg="Upload complete. (220ms)"6277lb # [ 110.630979] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6278worker2 # [ 110.607200] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16279worker2 # [ 110.619853] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06280worker2 # [ 110.623669] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16281worker2 # [ 110.633134] nix-daemon[854]: reaped child process 1499, status = succeeded6282worker2 # [ 110.663813] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306283worker2 # [ 110.678908] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306284worker2 # [ 110.692278] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.513Z level=INFO msg="All 2 paths already cached"6285client # copying 1 paths to 'lb:50051'...6286client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6287worker2 # [ 110.702129] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306288worker2 # [ 110.710299] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6289worker2 # [ 110.811936] nix-daemon[854]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)6290worker2 # [ 110.825663] nix-daemon[854]: reaped child process 1529, status = succeeded6291lb # [ 110.861956] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures6292worker2 # [ 110.841119] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.662Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6293worker2 # [ 110.843200] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.663Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6294lb # [ 110.892372] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6295worker2 # [ 110.874117] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.694Z level=INFO msg="Uploading 1 narinfos"6296lb # [ 110.903314] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6297lb # [ 110.906801] niks3-server[1103]: 2026/09/23 04:43:58 INFO Signed narinfos id=47 count=16298lb # [ 110.914820] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6299lb # [ 110.925582] niks3-server[1103]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6300lb # [ 110.932974] niks3-server[1103]: 2026/09/23 04:43:58 INFO Completed upload id=476301worker2 # [ 110.903851] nix-grpc-daemon[1330]: time=2026-09-23T04:43:58.724Z level=INFO msg="Upload complete. (181ms)"6302lb # [ 110.935504] niks3-server[1103]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6303worker2 # [ 110.908057] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=7686304worker2 # [ 110.912606] nix-grpc-daemon[1269]: ts=2026-09-23T04:43:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306305worker2 # [ 110.924287] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6306worker2 # [ 110.929108] nix-daemon[854]: reaped child process 1524, status = succeeded6307worker2 # [ 110.933403] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6308client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6309client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6310client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6311client # farm-top> LOG-farm-top-on-standby6312worker2 # [ 112.368728] nix-daemon[854]: reaped child process 1533, status = succeeded6313worker2 # [ 112.448756] nix-daemon[854]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)6314worker2 # [ 112.462569] nix-daemon[854]: reaped child process 1552, status = succeeded6315lb # [ 112.496147] niks3-server[1103]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures6316worker2 # [ 112.477044] nix-grpc-daemon[1330]: time=2026-09-23T04:44:00.297Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6317worker2 # [ 112.478811] nix-grpc-daemon[1330]: time=2026-09-23T04:44:00.299Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6318lb # [ 112.539863] niks3-server[1103]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6319worker2 # [ 112.517622] nix-grpc-daemon[1330]: time=2026-09-23T04:44:00.338Z level=INFO msg="Uploading 1 narinfos"6320lb # [ 112.546448] niks3-server[1103]: 2026/09/23 04:44:00 INFO Signed narinfos id=48 count=16321lb # [ 112.553242] niks3-server[1103]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6322lb # [ 112.557445] niks3-server[1103]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6323lb # [ 112.563977] niks3-server[1103]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6324lb # [ 112.569087] niks3-server[1103]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6325lb # [ 112.572270] niks3-server[1103]: 2026/09/23 04:44:00 INFO Completed upload id=486326worker2 # [ 112.543600] nix-grpc-daemon[1330]: time=2026-09-23T04:44:00.364Z level=INFO msg="Upload complete. (199ms)"6327lb # [ 112.575575] niks3-server[1103]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6328worker2 # [ 112.550936] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16329worker2 # [ 112.564114] nix-daemon[854]: reaped child process 1531, status = succeeded6330client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.63 seconds)6331client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6332client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6333worker1 # [ 112.774597] systemd[1]: Listening on Nix worker-protocol over gRPC.6334worker1 # [ 112.781844] systemd[1]: Starting Nix worker-protocol over gRPC...6335worker1 # [ 112.873701] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6336worker1 # [ 112.875755] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6337worker1 # [ 112.878960] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16338worker1 # [ 112.883309] nix-grpc-daemon[1334]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6339worker1 # [ 112.886179] nix-grpc-daemon[1334]: E0000 00:00:1790138640.924694 1334 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6340worker1 # [ 112.892069] nix-grpc-daemon[1334]: E0000 00:00:1790138640.930126 1334 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6341worker1 # [ 112.894237] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6342worker1 # [ 112.895785] systemd[1]: Started Nix worker-protocol over gRPC.6343worker1 # [ 112.904114] nix-grpc-daemon[1334]: E0000 00:00:1790138640.945256 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6344lb: must succeed: curl -sf localhost:9901/clusters6345worker1 # [ 112.918872] nix-grpc-daemon[1334]: E0000 00:00:1790138640.960397 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6346worker1 # [ 112.922149] nix-grpc-daemon[1334]: E0000 00:00:1790138640.962740 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6347worker1 # [ 112.933125] nix-grpc-daemon[1334]: E0000 00:00:1790138640.974609 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6348worker1 # [ 112.942164] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:00Z level=info event=scheduler_connected addr=lb:500516349worker2 # [ 112.930569] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:444286350worker2 # [ 112.935143] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6352worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6353worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6354lb: must succeed: curl -sf localhost:9901/clusters6355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6356lb: must succeed: curl -sf localhost:9901/clusters6357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6358worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6359worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6360(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.22 seconds)6361subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6362client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6363client # [ 113.182155] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6364client # Running as unit: rs.service; invocation ID: 708dda199ce244b3b99158620d610bf36365client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6366client # [ 113.322368] nix[1357]: don't know how to build these paths:6367client # [ 113.323315] nix[1357]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6368worker2 # [ 113.281864] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584306369worker2 # [ 113.292215] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06370worker2 # [ 113.296246] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16371worker2 # [ 113.345206] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306372worker2 # [ 113.357357] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584306373client # [ 113.407389] nix[1357]: copying 1 paths to 'lb:50051'...6374worker2 # [ 113.369316] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584306375client # [ 113.409672] nix[1357]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6376worker2 # [ 113.379711] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6377worker2 # [ 113.479486] nix-daemon[854]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)6378worker2 # [ 113.495306] nix-daemon[854]: reaped child process 1568, status = succeeded6379lb # [ 113.528442] niks3-server[1103]: 2026/09/23 04:44:01 INFO Received uploads request method=POST path=/api/pending_closures6380worker2 # [ 113.507635] nix-grpc-daemon[1330]: time=2026-09-23T04:44:01.328Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6381worker2 # [ 113.509597] nix-grpc-daemon[1330]: time=2026-09-23T04:44:01.330Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6382lb # [ 113.557073] niks3-server[1103]: 2026/09/23 04:44:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6383worker2 # [ 113.535877] nix-grpc-daemon[1330]: time=2026-09-23T04:44:01.356Z level=INFO msg="Uploading 1 narinfos"6384lb # [ 113.565057] niks3-server[1103]: 2026/09/23 04:44:01 INFO Signed narinfos id=49 count=16385lb # [ 113.571173] niks3-server[1103]: 2026/09/23 04:44:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6386lb # [ 113.576678] niks3-server[1103]: 2026/09/23 04:44:01 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6387lb # [ 113.591623] niks3-server[1103]: 2026/09/23 04:44:01 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6388worker2 # [ 113.567561] nix-grpc-daemon[1330]: time=2026-09-23T04:44:01.388Z level=INFO msg="Upload complete. (174ms)"6389lb # [ 113.596654] niks3-server[1103]: 2026/09/23 04:44:01 INFO Completed upload id=496390worker2 # [ 113.570882] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=0 paths=1 nar_bytes_in=4166391lb # [ 113.603358] niks3-server[1103]: 2026/09/23 04:44:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6392worker2 # [ 113.577278] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584306393worker2 # [ 113.585526] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6394worker2 # [ 113.594096] nix-daemon[854]: reaped child process 1563, status = succeeded6395worker2 # [ 113.597059] nix-daemon[854]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)6396client # [ 113.636288] nix[1357]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6397lb # [ 114.669870] envoy[904]: conn peer=[2001:db8:1::1]:46080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9560 tx=4376 ms=54666398lb: must succeed: curl -sf localhost:9901/clusters6399lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6400lb: must succeed: curl -sf localhost:9901/clusters6401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6402lb: must succeed: curl -sf localhost:9901/clusters6403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6404lb: must succeed: curl -sf localhost:9901/clusters6405lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6406worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6407worker1 # [ 115.844290] systemd-journald[284]: Received client request to rotate journal, rotating.6408worker1 # [ 115.865614] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9cd1869d9ceb409999d5c81f9f98b826.6409worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6410worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6411worker2 # [ 115.762573] systemd[1]: Stopping Nix worker-protocol over gRPC...6412worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6413client: must succeed: sleep 5.06414worker2 # [ 116.698437] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:04Z level=info event=draining6415client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6416worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6417worker2 # [ 121.018863] nix-daemon[854]: reaped child process 1572, status = succeeded6418worker2 # [ 121.093987] nix-daemon[854]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6419worker2 # [ 121.107309] nix-daemon[854]: reaped child process 1616, status = succeeded6420lb # [ 121.144447] niks3-server[1103]: 2026/09/23 04:44:09 INFO Received uploads request method=POST path=/api/pending_closures6421worker2 # [ 121.124201] nix-grpc-daemon[1330]: time=2026-09-23T04:44:08.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6422worker2 # [ 121.125866] nix-grpc-daemon[1330]: time=2026-09-23T04:44:08.946Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6423lb # [ 121.183649] niks3-server[1103]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6424lb # [ 121.194356] niks3-server[1103]: 2026/09/23 04:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6425lb # [ 121.200867] niks3-server[1103]: 2026/09/23 04:44:09 INFO Signed narinfos id=50 count=16426worker2 # [ 121.176716] nix-grpc-daemon[1330]: time=2026-09-23T04:44:08.997Z level=INFO msg="Uploading 1 narinfos"6427lb # [ 121.207994] niks3-server[1103]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6428lb # [ 121.211732] niks3-server[1103]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6429lb # [ 121.222392] niks3-server[1103]: 2026/09/23 04:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6430lb # [ 121.227229] niks3-server[1103]: 2026/09/23 04:44:09 INFO Completed upload id=506431lb # [ 121.230463] niks3-server[1103]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6432worker2 # [ 121.200312] nix-grpc-daemon[1330]: time=2026-09-23T04:44:09.021Z level=INFO msg="Upload complete. (191ms)"6433worker2 # [ 121.209796] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58430 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16434worker2 # [ 121.223493] nix-daemon[854]: reaped child process 1570, status = succeeded6435client # [ 121.279152] systemd[1]: rs.service: Deactivated successfully.6436client # [ 121.282300] systemd[1]: rs.service: Consumed 128ms CPU time over 8.096s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6437worker2 # [ 121.702213] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:09Z level=info event=signal_exit6438lb # [ 122.600789] niks3-server[1103]: 2026/09/23 04:44:10 INFO lead: released remote=192.168.1.4:609846439lb # [ 122.921307] niks3-server[1103]: 2026/09/23 04:44:10 INFO lead: acquired remote=192.168.1.3:483186440worker1 # [ 122.909897] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:10Z level=info event=scheduler_take_over6441worker1 # [ 123.586236] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:11Z level=info event=scheduler_restarting addr=lb:500516442worker2 # [ 123.577343] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=scheduler_restarting6443worker2 # [ 123.580756] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=scheduler_restarting addr=lb:500516444worker2 # [ 123.584204] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:444286445worker2 # [ 123.587872] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=worker_gone addr=192.168.1.4:500516446worker2 # [ 123.591182] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:444286447worker2 # [ 123.594841] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=worker_gone addr=192.168.1.3:500516448worker2 # [ 123.611974] nix-daemon[854]: reaped child process 1322, status = succeeded6449worker2 # [ 123.758628] nix-grpc-daemon[1269]: E0000 00:00:1790138651.579306 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6450worker2 # [ 123.763637] nix-grpc-daemon[1269]: E0000 00:00:1790138651.579508 1315 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6451worker1 # [ 123.786867] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:11Z level=info event=scheduler_connected addr=lb:500516452worker1 # [ 123.794197] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:592966453worker2 # [ 123.786385] nix-grpc-daemon[1269]: ts=2026-09-23T04:44:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6454worker1 # [ 123.797779] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06455worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.18 seconds)6456client: waiting for success: ! systemctl is-active rs6457client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6458client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6459client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6460client: must fail: journalctl -u rs | grep -q 'reconnecting'6461client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6462lb: must succeed: curl -sf localhost:9901/clusters6463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6464worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6465worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6466lb # [ 124.670662] envoy[904]: conn peer=[2001:db8:1::1]:40880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1500 ms=79356467lb # [ 124.674399] envoy[904]: conn peer=[2001:db8:1::4]:51718 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=27457 tx=1910 ms=330476468worker2 # [ 125.018720] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6469worker2 # [ 125.021677] systemd[1]: Stopped Nix worker-protocol over gRPC.6470worker2 # [ 125.024496] systemd[1]: nix-grpc-daemon.service: Consumed 1.689s CPU time over 34.434s wall clock time, 45.9M memory peak, 85.2K incoming IP traffic, 187.3K outgoing IP traffic.6471worker2 # [ 125.040602] systemd[1]: Starting scheduler-token.service...6472worker2 # [ 125.087661] systemd[1]: scheduler-token.service: Deactivated successfully.6473worker2 # [ 125.090195] systemd[1]: Finished scheduler-token.service.6474worker2 # [ 125.090906] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 48ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6475worker2 # [ 125.094957] systemd[1]: Starting Nix worker-protocol over gRPC...6476worker2 # [ 125.175826] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6477worker2 # [ 125.177928] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6478worker2 # [ 125.181223] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16479worker2 # [ 125.185816] nix-grpc-daemon[1626]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6480worker2 # [ 125.187148] nix-grpc-daemon[1626]: E0000 00:00:1790138653.006592 1626 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6481worker2 # [ 125.194258] nix-grpc-daemon[1626]: E0000 00:00:1790138653.012029 1626 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6482worker2 # [ 125.197924] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6483worker2 # [ 125.199806] systemd[1]: Started Nix worker-protocol over gRPC.6484worker2 # [ 125.202067] nix-grpc-daemon[1626]: E0000 00:00:1790138653.022290 1668 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6485worker2 # [ 125.213227] nix-grpc-daemon[1626]: E0000 00:00:1790138653.034157 1668 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6486worker2 # [ 125.218054] nix-grpc-daemon[1626]: E0000 00:00:1790138653.036460 1668 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6487worker2 # [ 125.227413] nix-grpc-daemon[1626]: E0000 00:00:1790138653.048361 1668 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6488worker2 # [ 125.236870] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:13Z level=info event=scheduler_connected addr=lb:500516489worker1 # [ 125.258835] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=info event=oidc_jwks provider=mock keys=16490worker1 # [ 125.263231] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:592966491worker1 # [ 125.267183] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06492lb: must succeed: curl -sf localhost:9901/clusters6493lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6494worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6495worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6496(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.34 seconds)6497subtest: scheduler restart mid-build: build finishes, no double build6498client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6499client # [ 125.516668] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6500client # Running as unit: mid.service; invocation ID: 5086716e1c614ffb90463de3cc796dba6501client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6502worker1 # [ 125.637733] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:593086503worker1 # [ 125.649790] nix-daemon[863]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6504client # [ 125.688383] nix[1393]: don't know how to build these paths:6505client # [ 125.689350] nix[1393]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6506worker1 # [ 125.671761] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06507worker1 # [ 125.676666] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16508worker1 # [ 125.720872] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:593086509worker1 # [ 125.732273] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 1/5)6510worker1 # [ 125.816926] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 188 ms (attempt 2/5)6511worker1 # [ 126.007281] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 3/5)6512worker1 # [ 126.134104] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 4/5)6513worker1 # [ 126.257607] nix-grpc-daemon[1334]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6514worker1 # [ 126.261879] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6515worker1 # [ 126.279724] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:593086516client # [ 126.315245] nix[1393]: copying 1 paths to 'lb:50051'...6517client # [ 126.319162] nix[1393]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6518worker1 # [ 126.293758] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:593086519worker1 # [ 126.301747] nix-daemon[863]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6520worker1 # [ 126.325426] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:14Z level=info event=niks3_push_started pid=14636521worker1 # [ 126.432565] nix-grpc-daemon[1463]: time=2026-09-23T04:44:14.473Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6522worker1 # [ 126.521478] nix-daemon[863]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6523worker1 # [ 126.532804] nix-daemon[863]: reaped child process 1479, status = succeeded6524lb # [ 126.558469] niks3-server[1103]: 2026/09/23 04:44:14 INFO Received uploads request method=POST path=/api/pending_closures6525worker1 # [ 126.552446] nix-grpc-daemon[1463]: time=2026-09-23T04:44:14.593Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6526worker1 # [ 126.553987] nix-grpc-daemon[1463]: time=2026-09-23T04:44:14.595Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6527lb # [ 126.595931] niks3-server[1103]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6528lb # [ 126.602158] niks3-server[1103]: 2026/09/23 04:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6529worker1 # [ 126.594521] nix-grpc-daemon[1463]: time=2026-09-23T04:44:14.636Z level=INFO msg="Uploading 1 narinfos"6530lb # [ 126.609628] niks3-server[1103]: 2026/09/23 04:44:14 INFO Signed narinfos id=51 count=16531lb # [ 126.624766] niks3-server[1103]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6532lb # [ 126.632372] niks3-server[1103]: 2026/09/23 04:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6533lb # [ 126.638682] niks3-server[1103]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6534lb # [ 126.641990] niks3-server[1103]: 2026/09/23 04:44:14 INFO Completed upload id=516535worker1 # [ 126.626615] nix-grpc-daemon[1463]: time=2026-09-23T04:44:14.667Z level=INFO msg="Upload complete. (190ms)"6536worker1 # [ 126.631069] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59308 duration_s=0 paths=1 nar_bytes_in=4246537worker1 # [ 126.633645] nix-grpc-daemon[1334]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:593086538worker1 # [ 126.642378] nix-daemon[863]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6539worker1 # [ 126.644698] nix-daemon[863]: reaped child process 1461, status = succeeded6540worker1 # [ 126.649965] nix-daemon[863]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6541client # [ 126.675446] nix[1393]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6542lb: must succeed: curl -sf localhost:9901/clusters6543lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6544lb: must succeed: curl -sf localhost:9901/clusters6545lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6546worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6547worker1 # [ 127.976267] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1334 (nix-grpc-daemon) on client request.6548worker1 # [ 127.980534] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6549lb # [ 127.997961] niks3-server[1103]: 2026/09/23 04:44:15 INFO lead: released remote=192.168.1.3:483186550client # [ 128.013484] nix[1393]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6551worker2 # [ 127.974370] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:15Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6552worker1 # [ 127.992047] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6553client # [ 128.018246] nix[1393]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6554worker1 # [ 127.994587] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6555worker1 # [ 127.998851] systemd[1]: nix-grpc-daemon.service: Consumed 373ms CPU time over 15.209s wall clock time, 35.8M memory peak, 15.3K incoming IP traffic, 27K outgoing IP traffic.6556worker1 # [ 128.002331] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6557worker1 # [ 128.012465] systemd[1]: Starting Nix worker-protocol over gRPC...6558worker1 # [ 128.040202] nix-daemon[863]: reaped child process 1453, status = succeeded6559worker1 # [ 128.042346] nix-daemon[863]: reaped child process 1485, status = succeeded6560worker1 # [ 128.053328] nix-daemon[1487]: unexpected Nix daemon error: error: interrupted by the user6561worker1 # [ 128.059169] nix-daemon[863]: reaped child process 1487, status = failed with exit code 16562worker1 # [ 128.104072] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6563worker1 # [ 128.106177] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6564worker1 # [ 128.109279] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16565worker1 # [ 128.113565] nix-grpc-daemon[1520]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6566worker1 # [ 128.116179] nix-grpc-daemon[1520]: E0000 00:00:1790138656.155007 1520 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6567worker1 # [ 128.122514] nix-grpc-daemon[1520]: E0000 00:00:1790138656.160399 1520 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6568worker1 # [ 128.126246] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6569worker1 # [ 128.129110] systemd[1]: Started Nix worker-protocol over gRPC.6570lb # [ 128.146379] niks3-server[1103]: 2026/09/23 04:44:16 INFO lead: acquired remote=192.168.1.3:571106571worker1 # [ 128.132053] nix-grpc-daemon[1520]: E0000 00:00:1790138656.172339 1562 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6572worker1 # [ 128.135057] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=scheduler_take_over6573worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.19 seconds)6574client: must succeed: sleep 5.06575worker1 # [ 128.144879] nix-grpc-daemon[1520]: E0000 00:00:1790138656.186339 1562 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6576worker1 # [ 128.150059] nix-grpc-daemon[1520]: E0000 00:00:1790138656.188672 1562 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6577worker1 # [ 128.158921] nix-grpc-daemon[1520]: E0000 00:00:1790138656.200401 1562 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6578worker1 # [ 128.168294] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=scheduler_connected addr=lb:500516579worker1 # [ 128.174685] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:593246580worker1 # [ 128.176281] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06581worker2 # [ 128.175356] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:15Z level=info event=scheduler_connected addr=lb:500516582worker1 # [ 128.192956] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=oidc_jwks provider=mock keys=16583worker1 # [ 128.194363] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:593246584worker1 # [ 128.195965] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06585worker1 # [ 128.501688] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06586worker1 # [ 128.506832] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16587worker1 # [ 128.555627] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:593266588worker1 # [ 128.564208] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6589worker1 # [ 128.579692] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6590client # [ 128.618876] nix[1393]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6591worker1 # [ 128.594391] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6592client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6593client: waiting for success: ! systemctl is-active mid6594worker1 # [ 133.412983] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:21Z level=info event=niks3_push_started pid=16046595worker1 # [ 133.445128] nix-daemon[863]: reaped child process 1579, status = succeeded6596worker1 # [ 133.523462] nix-grpc-daemon[1604]: time=2026-09-23T04:44:21.564Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6597worker1 # [ 133.611354] nix-daemon[863]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6598worker1 # [ 133.623445] nix-daemon[863]: reaped child process 1620, status = succeeded6599lb # [ 133.647443] niks3-server[1103]: 2026/09/23 04:44:21 INFO Received uploads request method=POST path=/api/pending_closures6600worker1 # [ 133.640260] nix-grpc-daemon[1604]: time=2026-09-23T04:44:21.681Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6601worker1 # [ 133.641721] nix-grpc-daemon[1604]: time=2026-09-23T04:44:21.683Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6602lb # [ 133.700800] niks3-server[1103]: 2026/09/23 04:44:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6603lb # [ 133.706985] niks3-server[1103]: 2026/09/23 04:44:21 INFO Signed narinfos id=52 count=16604worker1 # [ 133.694345] nix-grpc-daemon[1604]: time=2026-09-23T04:44:21.735Z level=INFO msg="Uploading 1 narinfos"6605lb # [ 133.722062] niks3-server[1103]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6606lb # [ 133.733890] niks3-server[1103]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6607lb # [ 133.738078] niks3-server[1103]: 2026/09/23 04:44:21 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6608lb # [ 133.741185] niks3-server[1103]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6609lb # [ 133.743242] niks3-server[1103]: 2026/09/23 04:44:21 INFO Completed upload id=526610worker1 # [ 133.727717] nix-grpc-daemon[1604]: time=2026-09-23T04:44:21.769Z level=INFO msg="Upload complete. (200ms)"6611lb # [ 133.745719] niks3-server[1103]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6612worker1 # [ 133.735461] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59326 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16613worker1 # [ 133.750953] nix-daemon[863]: reaped child process 1577, status = succeeded6614client # [ 133.799406] systemd[1]: mid.service: Deactivated successfully.6615client # [ 133.802916] systemd[1]: mid.service: Consumed 152ms CPU time over 8.285s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6616client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6617client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6618client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6619(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.02 seconds)6620subtest: interrupting the client stops the build on the worker6621client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6622client # [ 134.511876] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6623client # Running as unit: intr.service; invocation ID: 96d483117f1d495ca11ec94a062d75706624client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6625worker1 # [ 134.615621] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:593266626client # [ 134.647856] nix[1429]: don't know how to build these paths:6627client # [ 134.648854] nix[1429]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6628worker1 # [ 134.632897] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:22Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06629worker1 # [ 134.637113] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:22Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16630lb # [ 134.670919] envoy[904]: conn peer=[2001:db8:1::3]:60490 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=837 tx=315 ms=150486631lb # [ 134.675265] envoy[904]: conn peer=[2001:db8:1::1]:47086 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1939 ms=81186632worker2 # [ 134.672711] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726633worker2 # [ 134.683870] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6634worker2 # [ 134.700635] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)6635worker2 # [ 134.797723] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)6636worker2 # [ 134.951936] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 247 ms (attempt 3/5)6637worker2 # [ 135.201436] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 228 ms (attempt 4/5)6638worker2 # [ 135.437208] nix-grpc-daemon[1626]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6639worker2 # [ 135.440524] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6640worker2 # [ 135.458802] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544726641client # [ 135.508744] nix[1429]: copying 1 paths to 'lb:50051'...6642client # [ 135.513162] nix[1429]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6643worker2 # [ 135.474194] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544726644worker2 # [ 135.483151] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6645worker2 # [ 135.508840] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:23Z level=info event=niks3_push_started pid=17156646worker2 # [ 135.613553] nix-grpc-daemon[1715]: time=2026-09-23T04:44:23.434Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6647worker2 # [ 135.717474] nix-daemon[854]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6648worker2 # [ 135.731943] nix-daemon[854]: reaped child process 1735, status = succeeded6649lb # [ 135.767292] niks3-server[1103]: 2026/09/23 04:44:23 INFO Received uploads request method=POST path=/api/pending_closures6650worker2 # [ 135.747437] nix-grpc-daemon[1715]: time=2026-09-23T04:44:23.567Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6651worker2 # [ 135.748885] nix-grpc-daemon[1715]: time=2026-09-23T04:44:23.567Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6652lb # [ 135.805448] niks3-server[1103]: 2026/09/23 04:44:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6653lb # [ 135.811722] niks3-server[1103]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6654worker2 # [ 135.794060] nix-grpc-daemon[1715]: time=2026-09-23T04:44:23.614Z level=INFO msg="Uploading 1 narinfos"6655lb # [ 135.822789] niks3-server[1103]: 2026/09/23 04:44:23 INFO Signed narinfos id=53 count=16656lb # [ 135.828242] niks3-server[1103]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6657lb # [ 135.840061] niks3-server[1103]: 2026/09/23 04:44:23 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6658lb # [ 135.850393] niks3-server[1103]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6659lb # [ 135.854021] niks3-server[1103]: 2026/09/23 04:44:23 INFO Completed upload id=536660worker2 # [ 135.825389] nix-grpc-daemon[1715]: time=2026-09-23T04:44:23.646Z level=INFO msg="Upload complete. (208ms)"6661worker2 # [ 135.832275] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=4246662worker2 # [ 135.838196] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726663worker2 # [ 135.849257] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6664worker2 # [ 135.856133] nix-daemon[854]: reaped child process 1713, status = succeeded6665worker2 # [ 135.861281] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6666client # [ 135.901192] nix[1429]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6667client: must succeed: systemctl kill -s INT intr6668client # [ 137.985587] systemd[1]: intr.service: Sent signal SIGINT to main process 1429 (nix) on client request.6669worker2 # [ 137.953306] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:25Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6670client # [ 137.992988] nix[1429]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6671client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6672client # [ 138.000684] nix[1429]: error: interrupted by the user6673client # [ 138.030132] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6674client # [ 138.033651] systemd[1]: intr.service: Failed with result 'exit-code'.6675client # [ 138.037255] systemd[1]: intr.service: Consumed 126ms CPU time over 3.521s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6676worker2 # [ 138.078742] nix-daemon[854]: reaped child process 1737, status = succeeded6677worker2 # [ 138.080404] nix-daemon[1739]: unexpected Nix daemon error: error: interrupted by the user6678worker2 # [ 138.085179] nix-daemon[854]: reaped child process 1739, status = failed with exit code 16679(finished: subtest: interrupting the client stops the build on the worker, in 4.69 seconds)6680subtest: a deploy mid-build drains: the build finishes, the new generation takes over6681client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6682client # [ 139.232247] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6683client # Running as unit: sw.service; invocation ID: 84ca4933428348acbec073d8dd0c68256684client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.07 seconds)6685client # [ 139.382297] nix[1449]: don't know how to build these paths:6686client # [ 139.383305] nix[1449]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6687worker1 # [ 139.354080] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:593266688worker1 # [ 139.367724] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:27Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06689worker1 # [ 139.372561] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:27Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16690worker1 # [ 139.412151] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:593266691worker1 # [ 139.423860] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)6692worker1 # [ 139.529905] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 2/5)6693worker1 # [ 139.681176] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5)6694worker1 # [ 139.818174] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 406 ms (attempt 4/5)6695worker1 # [ 140.227150] nix-grpc-daemon[1520]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6696worker1 # [ 140.230222] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6697worker1 # [ 140.246790] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:593266698client # [ 140.284211] nix[1449]: copying 1 paths to 'lb:50051'...6699worker1 # [ 140.259703] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:593266700client # [ 140.287650] nix[1449]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6701worker1 # [ 140.274122] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6702worker1 # [ 140.388505] nix-daemon[863]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6703worker1 # [ 140.408853] nix-daemon[863]: reaped child process 1667, status = succeeded6704lb # [ 140.428614] niks3-server[1103]: 2026/09/23 04:44:28 INFO Received uploads request method=POST path=/api/pending_closures6705worker1 # [ 140.424081] nix-grpc-daemon[1604]: time=2026-09-23T04:44:28.465Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6706worker1 # [ 140.426450] nix-grpc-daemon[1604]: time=2026-09-23T04:44:28.467Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6707lb # [ 140.466185] niks3-server[1103]: 2026/09/23 04:44:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6708lb # [ 140.469723] niks3-server[1103]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6709lb # [ 140.476111] niks3-server[1103]: 2026/09/23 04:44:28 INFO Signed narinfos id=54 count=16710worker1 # [ 140.463476] nix-grpc-daemon[1604]: time=2026-09-23T04:44:28.504Z level=INFO msg="Uploading 1 narinfos"6711lb # [ 140.482247] niks3-server[1103]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6712lb # [ 140.492550] niks3-server[1103]: 2026/09/23 04:44:28 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6713lb # [ 140.495827] niks3-server[1103]: 2026/09/23 04:44:28 INFO Completed upload id=546714lb # [ 140.498676] niks3-server[1103]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6715worker1 # [ 140.481699] nix-grpc-daemon[1604]: time=2026-09-23T04:44:28.523Z level=INFO msg="Upload complete. (196ms)"6716worker1 # [ 140.486784] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59326 duration_s=0 paths=1 nar_bytes_in=4166717worker1 # [ 140.491782] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:593266718worker1 # [ 140.500888] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6719worker1 # [ 140.506677] nix-daemon[863]: reaped child process 1658, status = succeeded6720worker1 # [ 140.510820] nix-daemon[863]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6721client # [ 140.537474] nix[1449]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6722worker1: must succeed: systemctl reload nix-grpc-daemon.service6723worker1 # [ 142.757887] systemd[1]: Reloading Nix worker-protocol over gRPC...6724worker1 # [ 142.788793] systemd[1]: Reloaded Nix worker-protocol over gRPC.6725worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6726worker1: must succeed: pgrep -f 'read -t [9]0 x'6727worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6728client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26729client # don't know how to build these paths:6730client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6731worker2 # [ 142.967827] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544726732worker2 # [ 142.972958] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544726733worker1 # [ 143.003876] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:31Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06734worker1 # [ 143.005733] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:31Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16735worker2 # [ 143.039123] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726736worker2 # [ 143.051772] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544726737client # copying 1 paths to 'lb:50051'...6738worker2 # [ 143.066173] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544726739client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6740worker2 # [ 143.094323] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6741worker1 # [ 143.125824] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:31Z level=info event=draining6742worker2 # [ 143.203593] nix-daemon[854]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6743worker2 # [ 143.218686] nix-daemon[854]: reaped child process 1797, status = succeeded6744lb # [ 143.257299] niks3-server[1103]: 2026/09/23 04:44:31 INFO Received uploads request method=POST path=/api/pending_closures6745worker2 # [ 143.239119] nix-grpc-daemon[1715]: time=2026-09-23T04:44:31.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6746worker2 # [ 143.241552] nix-grpc-daemon[1715]: time=2026-09-23T04:44:31.061Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6747lb # [ 143.289972] niks3-server[1103]: 2026/09/23 04:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6748worker2 # [ 143.272566] nix-grpc-daemon[1715]: time=2026-09-23T04:44:31.093Z level=INFO msg="Uploading 1 narinfos"6749lb # [ 143.301291] niks3-server[1103]: 2026/09/23 04:44:31 INFO Signed narinfos id=55 count=16750lb # [ 143.304235] niks3-server[1103]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6751lb # [ 143.313591] niks3-server[1103]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6752lb # [ 143.322483] niks3-server[1103]: 2026/09/23 04:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6753lb # [ 143.335441] niks3-server[1103]: 2026/09/23 04:44:31 INFO Completed upload id=556754lb # [ 143.338331] niks3-server[1103]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6755worker2 # [ 143.306643] nix-grpc-daemon[1715]: time=2026-09-23T04:44:31.127Z level=INFO msg="Upload complete. (200ms)"6756worker2 # [ 143.311755] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=5286757worker2 # [ 143.319719] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726758worker2 # [ 143.330835] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6759worker2 # [ 143.339127] nix-daemon[854]: reaped child process 1792, status = succeeded6760client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6761worker2 # [ 143.344808] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6762client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6763client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6764client # farm-a> LOG-farm-a-during-drain6765worker2 # [ 144.162197] nix-daemon[854]: reaped child process 1801, status = succeeded6766worker2 # [ 144.246213] nix-daemon[854]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6767worker2 # [ 144.261378] nix-daemon[854]: reaped child process 1820, status = succeeded6768lb # [ 144.300948] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received uploads request method=POST path=/api/pending_closures6769worker2 # [ 144.279411] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6770worker2 # [ 144.280960] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.101Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6771lb # [ 144.339864] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6772lb # [ 144.350624] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6773lb # [ 144.357749] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6774lb # [ 144.363102] niks3-server[1103]: 2026/09/23 04:44:32 INFO Signed narinfos id=56 count=16775worker2 # [ 144.336731] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.157Z level=INFO msg="Uploading 1 narinfos"6776lb # [ 144.374572] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6777lb # [ 144.377952] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6778lb # [ 144.382288] niks3-server[1103]: 2026/09/23 04:44:32 INFO Completed upload id=566779lb # [ 144.384669] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6780worker2 # [ 144.354825] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.175Z level=INFO msg="Upload complete. (203ms)"6781worker1 # [ 144.380267] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:32Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06782worker2 # [ 144.362382] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16783worker1 # [ 144.382045] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:32Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16784worker2 # [ 144.378958] nix-daemon[854]: reaped child process 1799, status = succeeded6785worker2 # [ 144.411973] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726786worker2 # [ 144.425193] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544726787worker2 # [ 144.438567] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.259Z level=INFO msg="All 1 paths already cached"6788client # copying 1 paths to 'lb:50051'...6789worker2 # [ 144.447907] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544726790client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6791worker2 # [ 144.463357] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6792worker2 # [ 144.569855] nix-daemon[854]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6793worker2 # [ 144.586755] nix-daemon[854]: reaped child process 1827, status = succeeded6794lb # [ 144.619683] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received uploads request method=POST path=/api/pending_closures6795worker2 # [ 144.601622] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.422Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6796worker2 # [ 144.603678] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.424Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6797lb # [ 144.656261] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6798worker2 # [ 144.632573] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.453Z level=INFO msg="Uploading 1 narinfos"6799lb # [ 144.661464] niks3-server[1103]: 2026/09/23 04:44:32 INFO Signed narinfos id=57 count=16800lb # [ 144.669285] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6801lb # [ 144.674881] envoy[904]: conn peer=[2001:db8:1::1]:43788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=33746802lb # [ 144.679749] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6803lb # [ 144.689594] niks3-server[1103]: 2026/09/23 04:44:32 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6804lb # [ 144.697338] niks3-server[1103]: 2026/09/23 04:44:32 INFO Completed upload id=576805worker2 # [ 144.668750] nix-grpc-daemon[1715]: time=2026-09-23T04:44:32.489Z level=INFO msg="Upload complete. (192ms)"6806lb # [ 144.699323] niks3-server[1103]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6807worker2 # [ 144.673864] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=6406808worker2 # [ 144.680475] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726809worker2 # [ 144.693161] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6810worker2 # [ 144.703061] nix-daemon[854]: reaped child process 1822, status = succeeded6811client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6812client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6813worker2 # [ 144.708125] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6814client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6815client # farm-b> LOG-farm-b-during-drain6816worker2 # [ 146.127139] nix-daemon[854]: reaped child process 1831, status = succeeded6817worker2 # [ 146.209612] nix-daemon[854]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6818worker2 # [ 146.224938] nix-daemon[854]: reaped child process 1850, status = succeeded6819lb # [ 146.260476] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received uploads request method=POST path=/api/pending_closures6820worker2 # [ 146.240527] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.061Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6821worker2 # [ 146.242079] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.063Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6822lb # [ 146.300993] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6823lb # [ 146.309979] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6824worker2 # [ 146.290422] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.111Z level=INFO msg="Uploading 1 narinfos"6825lb # [ 146.319053] niks3-server[1103]: 2026/09/23 04:44:34 INFO Signed narinfos id=58 count=16826lb # [ 146.323203] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6827lb # [ 146.328300] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6828lb # [ 146.339137] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6829lb # [ 146.341742] niks3-server[1103]: 2026/09/23 04:44:34 INFO Completed upload id=586830worker2 # [ 146.313164] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.134Z level=INFO msg="Upload complete. (196ms)"6831lb # [ 146.345990] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6832worker2 # [ 146.318792] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16833worker1 # [ 146.337472] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:34Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06834worker1 # [ 146.340299] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:34Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16835worker2 # [ 146.334539] nix-daemon[854]: reaped child process 1829, status = succeeded6836worker2 # [ 146.371545] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726837worker2 # [ 146.391281] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544726838worker2 # [ 146.404860] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.225Z level=INFO msg="All 2 paths already cached"6839client # copying 1 paths to 'lb:50051'...6840worker2 # [ 146.414719] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544726841client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6842worker2 # [ 146.434184] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6843worker2 # [ 146.543913] nix-daemon[854]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6844worker2 # [ 146.559910] nix-daemon[854]: reaped child process 1857, status = succeeded6845lb # [ 146.595473] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received uploads request method=POST path=/api/pending_closures6846worker2 # [ 146.575095] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.396Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6847worker2 # [ 146.577341] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.397Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6848lb # [ 146.627065] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6849worker2 # [ 146.604048] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.424Z level=INFO msg="Uploading 1 narinfos"6850lb # [ 146.632437] niks3-server[1103]: 2026/09/23 04:44:34 INFO Signed narinfos id=59 count=16851lb # [ 146.640543] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6852lb # [ 146.651110] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6853lb # [ 146.658816] niks3-server[1103]: 2026/09/23 04:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6854lb # [ 146.663966] niks3-server[1103]: 2026/09/23 04:44:34 INFO Completed upload id=596855worker2 # [ 146.637942] nix-grpc-daemon[1715]: time=2026-09-23T04:44:34.458Z level=INFO msg="Upload complete. (187ms)"6856lb # [ 146.669864] niks3-server[1103]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6857worker2 # [ 146.642932] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=7686858worker2 # [ 146.648051] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544726859worker2 # [ 146.658125] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6860worker2 # [ 146.666595] nix-daemon[854]: reaped child process 1852, status = succeeded6861client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6862worker2 # [ 146.670200] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6863client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6864client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6865client # farm-top> LOG-farm-top-during-drain6866worker2 # [ 148.116482] nix-daemon[854]: reaped child process 1861, status = succeeded6867worker2 # [ 148.198452] nix-daemon[854]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6868worker2 # [ 148.215775] nix-daemon[854]: reaped child process 1880, status = succeeded6869lb # [ 148.249719] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received uploads request method=POST path=/api/pending_closures6870worker2 # [ 148.229266] nix-grpc-daemon[1715]: time=2026-09-23T04:44:36.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6871worker2 # [ 148.231022] nix-grpc-daemon[1715]: time=2026-09-23T04:44:36.051Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6872lb # [ 148.287811] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6873worker2 # [ 148.263488] nix-grpc-daemon[1715]: time=2026-09-23T04:44:36.084Z level=INFO msg="Uploading 1 narinfos"6874lb # [ 148.292414] niks3-server[1103]: 2026/09/23 04:44:36 INFO Signed narinfos id=60 count=16875lb # [ 148.304735] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6876lb # [ 148.310491] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6877lb # [ 148.318090] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6878lb # [ 148.323851] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6879lb # [ 148.330698] niks3-server[1103]: 2026/09/23 04:44:36 INFO Completed upload id=606880worker2 # [ 148.302239] nix-grpc-daemon[1715]: time=2026-09-23T04:44:36.123Z level=INFO msg="Upload complete. (222ms)"6881lb # [ 148.333095] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6882worker2 # [ 148.310403] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16883worker2 # [ 148.328506] nix-daemon[854]: reaped child process 1859, status = succeeded6884client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.54 seconds)6885client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6886client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6887worker1: must succeed: pkill -f 'read -t [9]0 x'6888worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6889client: waiting for success: ! systemctl is-active sw6890worker1 # [ 148.532416] nix-daemon[863]: reaped child process 1671, status = succeeded6891worker1 # [ 148.607313] nix-daemon[863]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6892worker1 # [ 148.619639] nix-daemon[863]: reaped child process 1721, status = succeeded6893lb # [ 148.643547] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received uploads request method=POST path=/api/pending_closures6894worker1 # [ 148.636297] nix-grpc-daemon[1604]: time=2026-09-23T04:44:36.677Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6895worker1 # [ 148.637779] nix-grpc-daemon[1604]: time=2026-09-23T04:44:36.679Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6896lb # [ 148.686148] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6897lb # [ 148.690338] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6898lb # [ 148.696252] niks3-server[1103]: 2026/09/23 04:44:36 INFO Signed narinfos id=61 count=16899worker1 # [ 148.684115] nix-grpc-daemon[1604]: time=2026-09-23T04:44:36.725Z level=INFO msg="Uploading 1 narinfos"6900lb # [ 148.706046] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6901lb # [ 148.712055] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6902lb # [ 148.720890] niks3-server[1103]: 2026/09/23 04:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6903lb # [ 148.724365] niks3-server[1103]: 2026/09/23 04:44:36 INFO Completed upload id=616904worker1 # [ 148.710089] nix-grpc-daemon[1604]: time=2026-09-23T04:44:36.751Z level=INFO msg="Upload complete. (185ms)"6905lb # [ 148.729197] niks3-server[1103]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6906worker1 # [ 148.714452] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59326 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16907worker1 # [ 148.730097] nix-daemon[863]: reaped child process 1669, status = succeeded6908client # [ 148.778961] systemd[1]: sw.service: Deactivated successfully.6909client # [ 148.783748] systemd[1]: sw.service: Consumed 139ms CPU time over 9.550s wall clock time, 41.4M memory peak, 7.6K incoming IP traffic, 8.6K outgoing IP traffic.6910worker1 # [ 149.130139] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:37Z level=info event=signal_exit6911lb # [ 149.151284] niks3-server[1103]: 2026/09/23 04:44:37 INFO lead: released remote=192.168.1.3:571106912client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6913client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6914client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6915worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6916worker2 # [ 150.124244] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:37Z level=info event=scheduler_restarting addr=lb:500516917worker1 # [ 150.139659] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=scheduler_restarting6918worker1 # [ 150.146494] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:593246919worker1 # [ 150.150743] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=worker_gone addr=192.168.1.4:500516920worker1 # [ 150.155167] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=scheduler_restarting addr=lb:500516921worker1 # [ 150.159237] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:593246922worker1 # [ 150.163281] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=worker_gone addr=192.168.1.3:500516923worker1 # [ 150.180381] nix-daemon[863]: reaped child process 1575, status = succeeded6924worker2 # [ 150.201476] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_take_over6925lb # [ 150.230641] niks3-server[1103]: 2026/09/23 04:44:38 INFO lead: acquired remote=192.168.1.4:486486926worker2 # [ 150.324778] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_connected addr=lb:500516927worker2 # [ 150.328686] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6928worker1 # [ 150.355473] nix-grpc-daemon[1520]: ts=2026-09-23T04:44:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6929worker2 # [ 150.529931] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_connected addr=lb:500516930worker2 # [ 150.534122] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6931worker2 # [ 150.735241] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_connected addr=lb:500516932worker2 # [ 150.739145] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6933worker2 # [ 150.940202] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_connected addr=lb:500516934worker2 # [ 150.944155] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6935worker1 # [ 150.987348] nix-grpc-daemon[1520]: E0000 00:00:1790138679.028551 1566 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true6936worker1 # [ 150.992571] nix-grpc-daemon[1520]: E0000 00:00:1790138679.028766 1566 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6937worker2 # [ 151.145781] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=scheduler_connected addr=lb:500516938worker2 # [ 151.163140] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=oidc_jwks provider=mock keys=16939worker2 # [ 151.167409] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:507026940worker2 # [ 151.171366] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06941worker1 # [ 152.155474] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6942worker1 # [ 152.159656] systemd[1]: nix-grpc-daemon.service: Consumed 594ms CPU time over 24.167s wall clock time, 36.1M memory peak, 25.2K incoming IP traffic, 32.7K outgoing IP traffic.6943worker1 # [ 152.175460] systemd[1]: Starting Nix worker-protocol over gRPC...6944worker1 # [ 152.266849] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6945worker1 # [ 152.268914] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6946worker1 # [ 152.272554] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16947worker1 # [ 152.277065] nix-grpc-daemon[1735]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6948worker1 # [ 152.278434] nix-grpc-daemon[1735]: E0000 00:00:1790138680.317831 1735 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6949worker1 # [ 152.284130] nix-grpc-daemon[1735]: E0000 00:00:1790138680.323067 1735 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6950worker1 # [ 152.288175] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6951worker1 # [ 152.290300] systemd[1]: Started Nix worker-protocol over gRPC.6952worker1 # [ 152.293403] nix-grpc-daemon[1735]: E0000 00:00:1790138680.334932 1778 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6953worker1 # [ 152.309078] nix-grpc-daemon[1735]: E0000 00:00:1790138680.348638 1778 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6954worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.70 seconds)6955worker1 # [ 152.312283] nix-grpc-daemon[1735]: E0000 00:00:1790138680.348789 1778 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6956lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6957lb: must succeed: curl -sf localhost:9901/clusters6958worker1 # [ 152.321776] nix-grpc-daemon[1735]: E0000 00:00:1790138680.363300 1778 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6959worker1 # [ 152.334192] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:40Z level=info event=scheduler_connected addr=lb:500516960worker2 # [ 152.324392] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:507026961worker2 # [ 152.328655] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06962lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6963lb: must succeed: curl -sf localhost:9901/clusters6964lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6965lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6966lb: must succeed: curl -sf localhost:9901/clusters6967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6968worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6969worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6970(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.33 seconds)6971subtest: queue gauges show a burst while every build blocks6972lb: must succeed: curl -sf localhost:9901/clusters6973lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6974lb: must succeed: curl -sf localhost:9901/clusters6975lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6976worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6977worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6978client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6979client # [ 153.665154] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6980client # Running as unit: burst.service; invocation ID: 13605f94b1ab42ceadca32cb73000a506981client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)6982lb: must succeed: curl -sf localhost:9901/clusters6983lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6984lb: must succeed: curl -sf localhost:9901/clusters6985lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6986worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6987worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6988worker1 # [ 153.806318] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559706989worker1 # [ 153.817855] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6990client # [ 153.856368] nix[1505]: don't know how to build these paths:6991client # [ 153.857263] nix[1505]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6992client # [ 153.858223] nix[1505]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6993client # [ 153.859852] nix[1505]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6994client # [ 153.860970] nix[1505]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6995worker2 # [ 153.830803] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06996worker2 # [ 153.837316] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16997worker2 # [ 153.842071] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06998worker2 # [ 153.846568] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16999worker2 # [ 153.851169] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07000worker2 # [ 153.855381] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07001worker1 # [ 153.894756] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707002worker2 # [ 153.881764] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727003worker1 # [ 153.906098] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)7004worker2 # [ 153.895607] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727005client # [ 153.945286] nix[1505]: copying 1 paths to 'lb:50051'...7006client # [ 153.948596] nix[1505]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7007worker2 # [ 153.910299] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544727008worker2 # [ 153.921822] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7009worker1 # [ 153.943485] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)7010worker2 # [ 154.025977] nix-daemon[854]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7011worker1 # [ 154.035756] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 241 ms (attempt 3/5)7012worker2 # [ 154.039836] nix-daemon[854]: reaped child process 1898, status = succeeded7013lb # [ 154.078706] niks3-server[1103]: 2026/09/23 04:44:41 INFO Received uploads request method=POST path=/api/pending_closures7014worker2 # [ 154.058420] nix-grpc-daemon[1715]: time=2026-09-23T04:44:41.879Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7015worker2 # [ 154.060457] nix-grpc-daemon[1715]: time=2026-09-23T04:44:41.880Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7016lb # [ 154.109918] niks3-server[1103]: 2026/09/23 04:44:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7017lb # [ 154.117837] niks3-server[1103]: 2026/09/23 04:44:42 INFO Signed narinfos id=62 count=17018worker2 # [ 154.091796] nix-grpc-daemon[1715]: time=2026-09-23T04:44:41.912Z level=INFO msg="Uploading 1 narinfos"7019lb # [ 154.131704] niks3-server[1103]: 2026/09/23 04:44:42 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7020lb # [ 154.140384] niks3-server[1103]: 2026/09/23 04:44:42 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7021lb # [ 154.150778] niks3-server[1103]: 2026/09/23 04:44:42 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7022lb # [ 154.154259] niks3-server[1103]: 2026/09/23 04:44:42 INFO Completed upload id=627023worker2 # [ 154.125639] nix-grpc-daemon[1715]: time=2026-09-23T04:44:41.946Z level=INFO msg="Upload complete. (191ms)"7024lb # [ 154.158616] niks3-server[1103]: 2026/09/23 04:44:42 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7025worker2 # [ 154.131146] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=4167026worker2 # [ 154.137825] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727027worker2 # [ 154.151119] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7028worker2 # [ 154.160120] nix-daemon[854]: reaped child process 1893, status = succeeded7029worker2 # [ 154.163338] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7030client # [ 154.202418] nix[1505]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7031worker1 # [ 154.278820] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 764 ms (attempt 4/5)7032lb # [ 154.674477] envoy[904]: conn peer=[2001:db8:1::1]:53618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9624 tx=4397 ms=53607033lb # [ 154.679508] envoy[904]: conn peer=[2001:db8:1::1]:43792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1558 ms=93797034lb # [ 154.683077] envoy[904]: conn peer=[2001:db8:1::3]:41436 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=992 tx=378 ms=221807035lb: must succeed: curl -sf localhost:9901/clusters7036lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7037lb: must succeed: curl -sf localhost:9901/clusters7038lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7039worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7040worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7041worker1 # [ 155.046854] nix-grpc-daemon[1735]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7042worker1 # [ 155.051176] nix-grpc-daemon[1735]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7043worker1 # [ 155.068664] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707044client # [ 155.106281] nix[1505]: copying 1 paths to 'lb:50051'...7045worker1 # [ 155.081091] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707046client # [ 155.109502] nix[1505]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7047worker1 # [ 155.095418] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7048worker1 # [ 155.118092] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:43Z level=info event=niks3_push_started pid=18007049worker1 # [ 155.226709] nix-grpc-daemon[1800]: time=2026-09-23T04:44:43.267Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7050worker1 # [ 155.317072] nix-daemon[863]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)7051worker1 # [ 155.328720] nix-daemon[863]: reaped child process 1816, status = succeeded7052lb # [ 155.353249] niks3-server[1103]: 2026/09/23 04:44:43 INFO Received uploads request method=POST path=/api/pending_closures7053worker1 # [ 155.348109] nix-grpc-daemon[1800]: time=2026-09-23T04:44:43.389Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7054worker1 # [ 155.349590] nix-grpc-daemon[1800]: time=2026-09-23T04:44:43.391Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7055lb # [ 155.396315] niks3-server[1103]: 2026/09/23 04:44:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7056lb # [ 155.399950] niks3-server[1103]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7057lb # [ 155.404298] niks3-server[1103]: 2026/09/23 04:44:43 INFO Signed narinfos id=63 count=17058worker1 # [ 155.391213] nix-grpc-daemon[1800]: time=2026-09-23T04:44:43.432Z level=INFO msg="Uploading 1 narinfos"7059lb # [ 155.411747] niks3-server[1103]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7060lb # [ 155.417347] niks3-server[1103]: 2026/09/23 04:44:43 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7061lb # [ 155.421137] niks3-server[1103]: 2026/09/23 04:44:43 INFO Completed upload id=637062worker1 # [ 155.406930] nix-grpc-daemon[1800]: time=2026-09-23T04:44:43.448Z level=INFO msg="Upload complete. (176ms)"7063worker1 # [ 155.409106] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=4167064lb # [ 155.425656] niks3-server[1103]: 2026/09/23 04:44:43 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7065worker1 # [ 155.412389] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707066worker1 # [ 155.419590] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7067worker1 # [ 155.422126] nix-daemon[863]: reaped child process 1798, status = succeeded7068worker1 # [ 155.427425] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7069client # [ 155.452818] nix[1505]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7070worker2 # [ 156.181145] nix-daemon[854]: reaped child process 1902, status = succeeded7071worker2 # [ 156.257286] nix-daemon[854]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)7072worker2 # [ 156.269170] nix-daemon[854]: reaped child process 1934, status = succeeded7073lb # [ 156.305162] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received uploads request method=POST path=/api/pending_closures7074worker2 # [ 156.285266] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.106Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7075worker2 # [ 156.286963] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.107Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7076lb # [ 156.345598] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7077lb # [ 156.350080] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7078lb # [ 156.361917] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7079lb # [ 156.368801] niks3-server[1103]: 2026/09/23 04:44:44 INFO Signed narinfos id=64 count=17080worker2 # [ 156.342729] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.163Z level=INFO msg="Uploading 1 narinfos"7081lb # [ 156.373051] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7082lb # [ 156.384518] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7083lb # [ 156.388113] niks3-server[1103]: 2026/09/23 04:44:44 INFO Completed upload id=647084worker2 # [ 156.359348] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.180Z level=INFO msg="Upload complete. (188ms)"7085lb # [ 156.390822] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7086worker2 # [ 156.365622] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17087worker2 # [ 156.375265] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=17088worker2 # [ 156.385106] nix-daemon[854]: reaped child process 1900, status = succeeded7089worker2 # [ 156.418881] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727090worker2 # [ 156.431228] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727091client # [ 156.481812] nix[1505]: copying 1 paths to 'lb:50051'...7092worker2 # [ 156.442860] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544727093client # [ 156.484146] nix[1505]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7094worker2 # [ 156.455193] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7095worker2 # [ 156.553133] nix-daemon[854]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)7096worker2 # [ 156.565576] nix-daemon[854]: reaped child process 1941, status = succeeded7097lb # [ 156.603272] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received uploads request method=POST path=/api/pending_closures7098worker2 # [ 156.582093] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.403Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7099worker2 # [ 156.583633] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.404Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7100lb # [ 156.633237] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7101lb # [ 156.638480] niks3-server[1103]: 2026/09/23 04:44:44 INFO Signed narinfos id=65 count=17102worker2 # [ 156.612713] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.433Z level=INFO msg="Uploading 1 narinfos"7103lb # [ 156.646076] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7104lb # [ 156.660109] niks3-server[1103]: 2026/09/23 04:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7105lb # [ 156.666233] niks3-server[1103]: 2026/09/23 04:44:44 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7106lb # [ 156.670803] niks3-server[1103]: 2026/09/23 04:44:44 INFO Completed upload id=657107worker2 # [ 156.644370] nix-grpc-daemon[1715]: time=2026-09-23T04:44:44.465Z level=INFO msg="Upload complete. (178ms)"7108worker2 # [ 156.648868] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=4167109worker2 # [ 156.655426] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727110worker2 # [ 156.665142] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7111worker2 # [ 156.672860] nix-daemon[854]: reaped child process 1936, status = succeeded7112worker2 # [ 156.677848] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7113client # [ 156.716727] nix[1505]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7114worker1 # [ 157.465624] nix-daemon[863]: reaped child process 1820, status = succeeded7115worker1 # [ 157.551804] nix-daemon[863]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)7116worker1 # [ 157.564599] nix-daemon[863]: reaped child process 1848, status = succeeded7117lb # [ 157.588768] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received uploads request method=POST path=/api/pending_closures7118worker1 # [ 157.584843] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7119worker1 # [ 157.587068] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.627Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7120lb # [ 157.631207] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7121worker1 # [ 157.628797] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.670Z level=INFO msg="Uploading 1 narinfos"7122lb # [ 157.643953] niks3-server[1103]: 2026/09/23 04:44:45 INFO Signed narinfos id=66 count=17123lb # [ 157.649142] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7124lb # [ 157.655110] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7125lb # [ 157.664334] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7126lb # [ 157.668635] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7127lb # [ 157.675264] niks3-server[1103]: 2026/09/23 04:44:45 INFO Completed upload id=667128worker1 # [ 157.659955] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.701Z level=INFO msg="Upload complete. (223ms)"7129lb # [ 157.677835] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7130worker2 # [ 157.654570] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:45Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=17131worker1 # [ 157.667748] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17132worker1 # [ 157.681578] nix-daemon[863]: reaped child process 1818, status = succeeded7133worker1 # [ 157.711190] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707134worker1 # [ 157.723959] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707135client # [ 157.760881] nix[1505]: copying 1 paths to 'lb:50051'...7136worker1 # [ 157.735741] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707137client # [ 157.763507] nix[1505]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7138worker1 # [ 157.747178] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7139worker1 # [ 157.845290] nix-daemon[863]: accepted connection from pid 1852, user nix-grpc-daemon (trusted)7140worker1 # [ 157.857781] nix-daemon[863]: reaped child process 1855, status = succeeded7141lb # [ 157.881754] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received uploads request method=POST path=/api/pending_closures7142worker1 # [ 157.873791] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7143worker1 # [ 157.875348] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.916Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7144lb # [ 157.910494] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7145lb # [ 157.915234] niks3-server[1103]: 2026/09/23 04:44:45 INFO Signed narinfos id=67 count=17146worker1 # [ 157.902687] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.943Z level=INFO msg="Uploading 1 narinfos"7147lb # [ 157.925125] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7148lb # [ 157.939259] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7149lb # [ 157.942199] niks3-server[1103]: 2026/09/23 04:44:45 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7150lb # [ 157.949638] niks3-server[1103]: 2026/09/23 04:44:45 INFO Completed upload id=677151worker1 # [ 157.934091] nix-grpc-daemon[1800]: time=2026-09-23T04:44:45.975Z level=INFO msg="Upload complete. (174ms)"7152worker1 # [ 157.938518] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=4167153lb # [ 157.955168] niks3-server[1103]: 2026/09/23 04:44:45 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7154worker1 # [ 157.944415] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707155worker1 # [ 157.955104] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7156worker1 # [ 157.958473] nix-daemon[863]: reaped child process 1850, status = succeeded7157worker1 # [ 157.963756] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7158client # [ 157.989481] nix[1505]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7159worker2 # [ 158.495451] nix-daemon[854]: reaped child process 1945, status = succeeded7160worker2 # [ 158.569822] nix-daemon[854]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)7161worker2 # [ 158.582763] nix-daemon[854]: reaped child process 1973, status = succeeded7162lb # [ 158.619573] niks3-server[1103]: 2026/09/23 04:44:46 INFO Received uploads request method=POST path=/api/pending_closures7163worker2 # [ 158.599438] nix-grpc-daemon[1715]: time=2026-09-23T04:44:46.420Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7164worker2 # [ 158.602059] nix-grpc-daemon[1715]: time=2026-09-23T04:44:46.421Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7165lb # [ 158.658920] niks3-server[1103]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7166lb # [ 158.669486] niks3-server[1103]: 2026/09/23 04:44:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7167worker2 # [ 158.646039] nix-grpc-daemon[1715]: time=2026-09-23T04:44:46.466Z level=INFO msg="Uploading 1 narinfos"7168lb # [ 158.675280] niks3-server[1103]: 2026/09/23 04:44:46 INFO Signed narinfos id=68 count=17169lb # [ 158.682431] niks3-server[1103]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7170lb # [ 158.688055] niks3-server[1103]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7171lb # [ 158.698875] niks3-server[1103]: 2026/09/23 04:44:46 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7172lb # [ 158.702162] niks3-server[1103]: 2026/09/23 04:44:46 INFO Completed upload id=687173worker2 # [ 158.673491] nix-grpc-daemon[1715]: time=2026-09-23T04:44:46.494Z level=INFO msg="Upload complete. (187ms)"7174lb # [ 158.705089] niks3-server[1103]: 2026/09/23 04:44:46 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7175worker2 # [ 158.679285] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17176worker2 # [ 158.691477] nix-daemon[854]: reaped child process 1943, status = succeeded7177worker1 # [ 159.581349] nix-daemon[863]: reaped child process 1859, status = succeeded7178worker1 # [ 159.657959] nix-daemon[863]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)7179worker1 # [ 159.671190] nix-daemon[863]: reaped child process 1887, status = succeeded7180lb # [ 159.694438] niks3-server[1103]: 2026/09/23 04:44:47 INFO Received uploads request method=POST path=/api/pending_closures7181worker1 # [ 159.687981] nix-grpc-daemon[1800]: time=2026-09-23T04:44:47.728Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7182worker1 # [ 159.689445] nix-grpc-daemon[1800]: time=2026-09-23T04:44:47.728Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7183lb # [ 159.735136] niks3-server[1103]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7184lb # [ 159.740653] niks3-server[1103]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7185lb # [ 159.749776] niks3-server[1103]: 2026/09/23 04:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7186lb # [ 159.753495] niks3-server[1103]: 2026/09/23 04:44:47 INFO Signed narinfos id=69 count=17187worker1 # [ 159.740416] nix-grpc-daemon[1800]: time=2026-09-23T04:44:47.781Z level=INFO msg="Uploading 1 narinfos"7188lb # [ 159.759446] niks3-server[1103]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7189lb # [ 159.769987] niks3-server[1103]: 2026/09/23 04:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7190lb # [ 159.774228] niks3-server[1103]: 2026/09/23 04:44:47 INFO Completed upload id=697191worker1 # [ 159.758792] nix-grpc-daemon[1800]: time=2026-09-23T04:44:47.800Z level=INFO msg="Upload complete. (186ms)"7192lb # [ 159.776146] niks3-server[1103]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7193worker1 # [ 159.764728] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17194worker1 # [ 159.778834] nix-daemon[863]: reaped child process 1857, status = succeeded7195client # [ 159.824420] systemd[1]: burst.service: Deactivated successfully.7196client # [ 159.827991] systemd[1]: burst.service: Consumed 193ms CPU time over 6.163s wall clock time, 52M memory peak, 14.4K incoming IP traffic, 19.4K outgoing IP traffic.7197client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7198client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.05 seconds)7199lb: must succeed: curl -sf localhost:9901/clusters7200lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7201lb: must succeed: curl -sf localhost:9901/clusters7202lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7203worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7204worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7205(finished: subtest: queue gauges show a burst while every build blocks, in 7.40 seconds)7206subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7207client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17208worker1 # [ 161.037132] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559707209worker1 # [ 161.043220] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707210worker2 # [ 161.070788] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:48Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07211worker2 # [ 161.075568] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:48Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17212worker2 # [ 161.080149] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:48Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07213worker2 # [ 161.084108] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:48Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07214worker1 # [ 161.132483] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707215worker1 # [ 161.144854] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707216worker1 # [ 161.156901] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707217worker1 # [ 161.168177] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7218worker1 # [ 161.272886] nix-daemon[863]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7219worker1 # [ 161.285984] nix-daemon[863]: reaped child process 1898, status = succeeded7220lb # [ 161.309575] niks3-server[1103]: 2026/09/23 04:44:49 INFO Received uploads request method=POST path=/api/pending_closures7221worker1 # [ 161.303340] nix-grpc-daemon[1800]: time=2026-09-23T04:44:49.344Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7222worker1 # [ 161.305444] nix-grpc-daemon[1800]: time=2026-09-23T04:44:49.346Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7223lb # [ 161.341442] niks3-server[1103]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7224lb # [ 161.349648] niks3-server[1103]: 2026/09/23 04:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7225lb # [ 161.355360] niks3-server[1103]: 2026/09/23 04:44:49 INFO Signed narinfos id=70 count=17226worker1 # [ 161.344060] nix-grpc-daemon[1800]: time=2026-09-23T04:44:49.385Z level=INFO msg="Uploading 1 narinfos"7227lb # [ 161.365065] niks3-server[1103]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7228lb # [ 161.374056] niks3-server[1103]: 2026/09/23 04:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7229lb # [ 161.378700] niks3-server[1103]: 2026/09/23 04:44:49 INFO Completed upload id=707230worker1 # [ 161.363123] nix-grpc-daemon[1800]: time=2026-09-23T04:44:49.404Z level=INFO msg="Upload complete. (179ms)"7231lb # [ 161.380987] niks3-server[1103]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7232worker1 # [ 161.367198] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=5047233worker1 # [ 161.371985] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707234worker1 # [ 161.380865] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7235worker1 # [ 161.389290] nix-daemon[863]: reaped child process 1893, status = succeeded7236worker1 # [ 161.391967] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7237worker1 # [ 162.231517] nix-daemon[863]: reaped child process 1902, status = succeeded7238worker1 # [ 162.309830] nix-daemon[863]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)7239worker1 # [ 162.321124] nix-daemon[863]: reaped child process 1921, status = succeeded7240lb # [ 162.345486] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures7241worker1 # [ 162.337664] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.378Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7242worker1 # [ 162.339275] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.380Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7243lb # [ 162.386809] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7244lb # [ 162.391221] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7245lb # [ 162.398589] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7246lb # [ 162.405607] niks3-server[1103]: 2026/09/23 04:44:50 INFO Signed narinfos id=71 count=17247worker1 # [ 162.392579] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.433Z level=INFO msg="Uploading 1 narinfos"7248lb # [ 162.412515] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7249lb # [ 162.420872] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7250lb # [ 162.425109] niks3-server[1103]: 2026/09/23 04:44:50 INFO Completed upload id=717251worker1 # [ 162.409717] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.451Z level=INFO msg="Upload complete. (211ms)"7252lb # [ 162.427426] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7253worker1 # [ 162.416870] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17254worker2 # [ 162.404469] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:50Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17255worker1 # [ 162.425748] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707256worker2 # [ 162.413969] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:50Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07257worker1 # [ 162.437544] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707258worker1 # [ 162.442937] nix-daemon[863]: reaped child process 1900, status = succeeded7259worker1 # [ 162.448331] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707260worker1 # [ 162.454369] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7261worker1 # [ 162.541976] nix-daemon[863]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)7262worker1 # [ 162.554842] nix-daemon[863]: reaped child process 1928, status = succeeded7263lb # [ 162.576349] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures7264worker1 # [ 162.568905] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7265worker1 # [ 162.570411] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.611Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7266lb # [ 162.609302] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7267lb # [ 162.613797] niks3-server[1103]: 2026/09/23 04:44:50 INFO Signed narinfos id=72 count=17268worker1 # [ 162.601436] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.642Z level=INFO msg="Uploading 1 narinfos"7269lb # [ 162.621254] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7270lb # [ 162.633658] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7271lb # [ 162.638358] niks3-server[1103]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7272lb # [ 162.648858] niks3-server[1103]: 2026/09/23 04:44:50 INFO Completed upload id=727273worker1 # [ 162.633122] nix-grpc-daemon[1800]: time=2026-09-23T04:44:50.674Z level=INFO msg="Upload complete. (173ms)"7274lb # [ 162.652106] niks3-server[1103]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7275worker1 # [ 162.637468] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=5047276worker1 # [ 162.642593] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707277worker1 # [ 162.652138] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7278worker1 # [ 162.657691] nix-daemon[863]: reaped child process 1923, status = succeeded7279worker1 # [ 162.661973] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7280worker1 # [ 163.499379] nix-daemon[863]: reaped child process 1932, status = succeeded7281worker1 # [ 163.582201] nix-daemon[863]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)7282worker1 # [ 163.594465] nix-daemon[863]: reaped child process 1951, status = succeeded7283lb # [ 163.618286] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received uploads request method=POST path=/api/pending_closures7284worker1 # [ 163.611594] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.652Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7285worker1 # [ 163.613190] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.654Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7286lb # [ 163.657150] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7287lb # [ 163.669582] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7288worker1 # [ 163.664446] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.705Z level=INFO msg="Uploading 1 narinfos"7289lb # [ 163.680742] niks3-server[1103]: 2026/09/23 04:44:51 INFO Signed narinfos id=73 count=17290lb # [ 163.683850] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7291lb # [ 163.687197] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7292lb # [ 163.697884] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7293lb # [ 163.705134] niks3-server[1103]: 2026/09/23 04:44:51 INFO Completed upload id=737294worker1 # [ 163.690893] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.732Z level=INFO msg="Upload complete. (222ms)"7295lb # [ 163.708120] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7296worker2 # [ 163.685250] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:51Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17297worker1 # [ 163.696995] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17298worker1 # [ 163.712730] nix-daemon[863]: reaped child process 1930, status = succeeded7299worker1 # [ 163.741879] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707300worker2 # [ 163.731348] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:51Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07301worker1 # [ 163.752611] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707302worker1 # [ 163.762072] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707303worker1 # [ 163.772357] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7304worker1 # [ 163.865241] nix-daemon[863]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)7305worker1 # [ 163.877348] nix-daemon[863]: reaped child process 1958, status = succeeded7306lb # [ 163.900453] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received uploads request method=POST path=/api/pending_closures7307worker1 # [ 163.889429] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7308worker1 # [ 163.890879] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.932Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7309lb # [ 163.917061] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7310lb # [ 163.920119] niks3-server[1103]: 2026/09/23 04:44:51 INFO Signed narinfos id=74 count=17311worker1 # [ 163.904747] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.946Z level=INFO msg="Uploading 1 narinfos"7312lb # [ 163.922693] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7313lb # [ 163.925376] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7314lb # [ 163.930865] niks3-server[1103]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7315lb # [ 163.934944] niks3-server[1103]: 2026/09/23 04:44:51 INFO Completed upload id=747316worker1 # [ 163.919281] nix-grpc-daemon[1800]: time=2026-09-23T04:44:51.960Z level=INFO msg="Upload complete. (136ms)"7317lb # [ 163.937181] niks3-server[1103]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7318worker1 # [ 163.921523] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=5047319worker1 # [ 163.925961] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707320worker1 # [ 163.931269] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7321worker1 # [ 163.933498] nix-daemon[863]: reaped child process 1953, status = succeeded7322worker1 # [ 163.938380] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7323lb # [ 164.673730] envoy[904]: conn peer=[2001:db8:1::1]:41612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9746 tx=5239 ms=59777324worker1 # [ 165.187186] nix-daemon[863]: reaped child process 1962, status = succeeded7325worker1 # [ 165.270443] nix-daemon[863]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)7326worker1 # [ 165.281792] nix-daemon[863]: reaped child process 1981, status = succeeded7327lb # [ 165.306868] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received uploads request method=POST path=/api/pending_closures7328worker1 # [ 165.299097] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.340Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7329worker1 # [ 165.300884] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.342Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7330lb # [ 165.353432] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7331lb # [ 165.360106] niks3-server[1103]: 2026/09/23 04:44:53 INFO Signed narinfos id=75 count=17332worker1 # [ 165.347566] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.388Z level=INFO msg="Uploading 1 narinfos"7333lb # [ 165.367748] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7334lb # [ 165.372771] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7335lb # [ 165.377993] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7336lb # [ 165.387638] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7337lb # [ 165.391669] niks3-server[1103]: 2026/09/23 04:44:53 INFO Completed upload id=757338worker1 # [ 165.376393] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.417Z level=INFO msg="Upload complete. (229ms)"7339lb # [ 165.394738] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7340worker2 # [ 165.372664] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:53Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17341worker1 # [ 165.385240] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17342worker1 # [ 165.392606] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707343worker2 # [ 165.380560] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:53Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07344worker1 # [ 165.407529] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707345worker1 # [ 165.412549] nix-daemon[863]: reaped child process 1960, status = succeeded7346worker1 # [ 165.418500] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.459Z level=INFO msg="All 1 paths already cached"7347worker1 # [ 165.424922] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707348worker1 # [ 165.433207] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7349worker1 # [ 165.516067] nix-daemon[863]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7350worker1 # [ 165.528559] nix-daemon[863]: reaped child process 1988, status = succeeded7351lb # [ 165.551516] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received uploads request method=POST path=/api/pending_closures7352worker1 # [ 165.544039] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.585Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7353worker1 # [ 165.545944] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.587Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7354lb # [ 165.581424] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7355lb # [ 165.588239] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7356lb # [ 165.593365] niks3-server[1103]: 2026/09/23 04:44:53 INFO Signed narinfos id=76 count=17357worker1 # [ 165.581045] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.622Z level=INFO msg="Uploading 1 narinfos"7358lb # [ 165.607877] niks3-server[1103]: 2026/09/23 04:44:53 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7359worker1 # [ 165.598743] nix-grpc-daemon[1800]: time=2026-09-23T04:44:53.640Z level=INFO msg="Upload complete. (160ms)"7360lb # [ 165.614459] niks3-server[1103]: 2026/09/23 04:44:53 INFO Completed upload id=767361lb # [ 165.616776] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7362worker1 # [ 165.601419] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=6247363worker1 # [ 165.607743] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707364lb # [ 165.627285] niks3-server[1103]: 2026/09/23 04:44:53 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7365worker1 # [ 165.619060] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7366worker1 # [ 165.620661] nix-daemon[863]: reaped child process 1983, status = succeeded7367worker1 # [ 165.625829] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7368worker1 # [ 166.867398] nix-daemon[863]: reaped child process 1992, status = succeeded7369worker1 # [ 166.944467] nix-daemon[863]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)7370worker1 # [ 166.956695] nix-daemon[863]: reaped child process 2011, status = succeeded7371lb # [ 166.980969] niks3-server[1103]: 2026/09/23 04:44:54 INFO Received uploads request method=POST path=/api/pending_closures7372worker1 # [ 166.973580] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7373worker1 # [ 166.975140] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.016Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7374lb # [ 167.025113] niks3-server[1103]: 2026/09/23 04:44:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7375lb # [ 167.033091] niks3-server[1103]: 2026/09/23 04:44:54 INFO Signed narinfos id=77 count=17376worker1 # [ 167.017775] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.059Z level=INFO msg="Uploading 1 narinfos"7377lb # [ 167.037564] niks3-server[1103]: 2026/09/23 04:44:54 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7378lb # [ 167.044242] niks3-server[1103]: 2026/09/23 04:44:54 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7379lb # [ 167.047547] niks3-server[1103]: 2026/09/23 04:44:54 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7380lb # [ 167.057747] niks3-server[1103]: 2026/09/23 04:44:54 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7381lb # [ 167.061567] niks3-server[1103]: 2026/09/23 04:44:54 INFO Completed upload id=777382worker1 # [ 167.046226] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.087Z level=INFO msg="Upload complete. (212ms)"7383lb # [ 167.063314] niks3-server[1103]: 2026/09/23 04:44:54 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7384worker1 # [ 167.052524] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17385worker2 # [ 167.040411] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:54Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17386worker2 # [ 167.051368] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:54Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07387worker1 # [ 167.067374] nix-daemon[863]: reaped child process 1990, status = succeeded7388worker1 # [ 167.103856] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707389worker1 # [ 167.116559] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707390worker1 # [ 167.131147] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.172Z level=INFO msg="All 1 paths already cached"7391worker1 # [ 167.138740] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707392worker1 # [ 167.146917] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7393worker1 # [ 167.245344] nix-daemon[863]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)7394worker1 # [ 167.257711] nix-daemon[863]: reaped child process 2018, status = succeeded7395lb # [ 167.280731] niks3-server[1103]: 2026/09/23 04:44:55 INFO Received uploads request method=POST path=/api/pending_closures7396worker1 # [ 167.273586] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.315Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7397worker1 # [ 167.275395] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.316Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7398lb # [ 167.311683] niks3-server[1103]: 2026/09/23 04:44:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7399worker1 # [ 167.303717] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.345Z level=INFO msg="Uploading 1 narinfos"7400lb # [ 167.320327] niks3-server[1103]: 2026/09/23 04:44:55 INFO Signed narinfos id=78 count=17401lb # [ 167.328106] niks3-server[1103]: 2026/09/23 04:44:55 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7402lb # [ 167.335913] niks3-server[1103]: 2026/09/23 04:44:55 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7403lb # [ 167.345114] niks3-server[1103]: 2026/09/23 04:44:55 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7404lb # [ 167.350404] niks3-server[1103]: 2026/09/23 04:44:55 INFO Completed upload id=787405worker1 # [ 167.337147] nix-grpc-daemon[1800]: time=2026-09-23T04:44:55.378Z level=INFO msg="Upload complete. (177ms)"7406lb # [ 167.357353] niks3-server[1103]: 2026/09/23 04:44:55 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7407worker1 # [ 167.340858] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=6247408worker1 # [ 167.345893] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707409worker1 # [ 167.357113] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7410worker1 # [ 167.363061] nix-daemon[863]: reaped child process 2013, status = succeeded7411worker1 # [ 167.365825] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7412worker1 # [ 168.409274] nix-daemon[863]: reaped child process 2022, status = succeeded7413worker1 # [ 168.485509] nix-daemon[863]: accepted connection from pid 2038, user nix-grpc-daemon (trusted)7414worker1 # [ 168.496691] nix-daemon[863]: reaped child process 2041, status = succeeded7415lb # [ 168.520741] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received uploads request method=POST path=/api/pending_closures7416worker1 # [ 168.513439] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.554Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7417worker1 # [ 168.515097] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.556Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7418lb # [ 168.561093] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7419lb # [ 168.573234] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7420worker1 # [ 168.566358] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.607Z level=INFO msg="Uploading 1 narinfos"7421lb # [ 168.582341] niks3-server[1103]: 2026/09/23 04:44:56 INFO Signed narinfos id=79 count=17422lb # [ 168.587693] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7423lb # [ 168.593108] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7424lb # [ 168.601976] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7425lb # [ 168.606274] niks3-server[1103]: 2026/09/23 04:44:56 INFO Completed upload id=797426lb # [ 168.607559] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7427worker1 # [ 168.590811] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.632Z level=INFO msg="Upload complete. (217ms)"7428worker1 # [ 168.596990] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17429worker2 # [ 168.587316] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:56Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17430worker2 # [ 168.595340] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:56Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07431worker1 # [ 168.612573] nix-daemon[863]: reaped child process 2020, status = succeeded7432worker1 # [ 168.644269] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707433worker1 # [ 168.656518] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707434worker1 # [ 168.669067] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.710Z level=INFO msg="All 1 paths already cached"7435worker1 # [ 168.677356] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707436worker1 # [ 168.684754] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7437worker1 # [ 168.783037] nix-daemon[863]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)7438worker1 # [ 168.797733] nix-daemon[863]: reaped child process 2048, status = succeeded7439lb # [ 168.818730] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received uploads request method=POST path=/api/pending_closures7440worker1 # [ 168.810860] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.852Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7441worker1 # [ 168.812357] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.853Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7442lb # [ 168.851439] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7443lb # [ 168.857357] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7444lb # [ 168.864498] niks3-server[1103]: 2026/09/23 04:44:56 INFO Signed narinfos id=80 count=17445worker1 # [ 168.851973] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.893Z level=INFO msg="Uploading 1 narinfos"7446lb # [ 168.872623] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7447lb # [ 168.885763] niks3-server[1103]: 2026/09/23 04:44:56 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7448worker1 # [ 168.876321] nix-grpc-daemon[1800]: time=2026-09-23T04:44:56.917Z level=INFO msg="Upload complete. (178ms)"7449lb # [ 168.892227] niks3-server[1103]: 2026/09/23 04:44:56 INFO Completed upload id=807450lb # [ 168.896082] niks3-server[1103]: 2026/09/23 04:44:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7451worker1 # [ 168.879108] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=6247452worker1 # [ 168.884201] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707453worker1 # [ 168.896057] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7454worker1 # [ 168.902106] nix-daemon[863]: reaped child process 2043, status = succeeded7455worker1 # [ 168.905360] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7456worker1 # [ 170.146566] nix-daemon[863]: reaped child process 2052, status = succeeded7457worker1 # [ 170.227388] nix-daemon[863]: accepted connection from pid 2068, user nix-grpc-daemon (trusted)7458worker1 # [ 170.240623] nix-daemon[863]: reaped child process 2071, status = succeeded7459lb # [ 170.260789] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received uploads request method=POST path=/api/pending_closures7460worker1 # [ 170.253727] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.295Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7461worker1 # [ 170.255585] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.296Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7462lb # [ 170.302798] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7463lb # [ 170.311912] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7464lb # [ 170.317877] niks3-server[1103]: 2026/09/23 04:44:58 INFO Signed narinfos id=81 count=17465worker1 # [ 170.302553] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.343Z level=INFO msg="Uploading 1 narinfos"7466lb # [ 170.325650] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7467lb # [ 170.333813] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7468lb # [ 170.341166] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7469lb # [ 170.345727] niks3-server[1103]: 2026/09/23 04:44:58 INFO Completed upload id=817470lb # [ 170.347826] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7471worker1 # [ 170.331879] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.373Z level=INFO msg="Upload complete. (218ms)"7472worker1 # [ 170.338728] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17473worker2 # [ 170.328620] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:58Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17474worker2 # [ 170.335821] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:58Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07475worker1 # [ 170.354267] nix-daemon[863]: reaped child process 2050, status = succeeded7476worker1 # [ 170.385419] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707477worker1 # [ 170.398242] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707478worker1 # [ 170.411730] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.453Z level=INFO msg="All 2 paths already cached"7479worker1 # [ 170.419750] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707480worker1 # [ 170.430248] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7481worker1 # [ 170.526090] nix-daemon[863]: accepted connection from pid 2075, user nix-grpc-daemon (trusted)7482worker1 # [ 170.540042] nix-daemon[863]: reaped child process 2078, status = succeeded7483lb # [ 170.560775] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received uploads request method=POST path=/api/pending_closures7484worker1 # [ 170.553679] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.595Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7485worker1 # [ 170.555248] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.596Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7486lb # [ 170.592862] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7487lb # [ 170.600262] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7488lb # [ 170.605801] niks3-server[1103]: 2026/09/23 04:44:58 INFO Signed narinfos id=82 count=17489worker1 # [ 170.592991] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.634Z level=INFO msg="Uploading 1 narinfos"7490lb # [ 170.619267] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7491lb # [ 170.625656] niks3-server[1103]: 2026/09/23 04:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7492lb # [ 170.630474] niks3-server[1103]: 2026/09/23 04:44:58 INFO Completed upload id=827493worker1 # [ 170.617198] nix-grpc-daemon[1800]: time=2026-09-23T04:44:58.658Z level=INFO msg="Upload complete. (175ms)"7494lb # [ 170.635774] niks3-server[1103]: 2026/09/23 04:44:58 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7495worker1 # [ 170.621097] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=7527496worker1 # [ 170.626406] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707497worker1 # [ 170.636204] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7498worker1 # [ 170.643762] nix-daemon[863]: reaped child process 2073, status = succeeded7499worker1 # [ 170.648588] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7500worker1 # [ 171.291316] nix-daemon[863]: reaped child process 2082, status = succeeded7501worker1 # [ 171.367770] nix-daemon[863]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)7502worker1 # [ 171.380980] nix-daemon[863]: reaped child process 2101, status = succeeded7503lb # [ 171.404196] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received uploads request method=POST path=/api/pending_closures7504worker1 # [ 171.396927] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7505worker1 # [ 171.399871] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.440Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7506lb # [ 171.442249] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7507worker1 # [ 171.435172] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.476Z level=INFO msg="Uploading 1 narinfos"7508lb # [ 171.450920] niks3-server[1103]: 2026/09/23 04:44:59 INFO Signed narinfos id=83 count=17509lb # [ 171.459146] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7510lb # [ 171.465470] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7511lb # [ 171.470725] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7512lb # [ 171.478363] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7513lb # [ 171.485258] niks3-server[1103]: 2026/09/23 04:44:59 INFO Completed upload id=837514worker1 # [ 171.469558] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.510Z level=INFO msg="Upload complete. (214ms)"7515lb # [ 171.488641] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7516worker2 # [ 171.463940] nix-grpc-daemon[1626]: ts=2026-09-23T04:44:59Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17517worker1 # [ 171.475928] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17518worker1 # [ 171.483863] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707519worker1 # [ 171.495267] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707520worker1 # [ 171.502602] nix-daemon[863]: reaped child process 2080, status = succeeded7521worker1 # [ 171.509278] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.550Z level=INFO msg="All 2 paths already cached"7522worker1 # [ 171.516273] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707523worker1 # [ 171.522763] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7524worker1 # [ 171.609442] nix-daemon[863]: accepted connection from pid 2105, user nix-grpc-daemon (trusted)7525worker1 # [ 171.622547] nix-daemon[863]: reaped child process 2108, status = succeeded7526lb # [ 171.644979] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received uploads request method=POST path=/api/pending_closures7527worker1 # [ 171.638910] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.680Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7528worker1 # [ 171.640841] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.681Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7529lb # [ 171.674733] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7530worker1 # [ 171.666499] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.707Z level=INFO msg="Uploading 1 narinfos"7531lb # [ 171.682218] niks3-server[1103]: 2026/09/23 04:44:59 INFO Signed narinfos id=84 count=17532lb # [ 171.689165] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7533lb # [ 171.698817] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7534lb # [ 171.708112] niks3-server[1103]: 2026/09/23 04:44:59 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7535lb # [ 171.714461] niks3-server[1103]: 2026/09/23 04:44:59 INFO Completed upload id=847536worker1 # [ 171.698925] nix-grpc-daemon[1800]: time=2026-09-23T04:44:59.740Z level=INFO msg="Upload complete. (167ms)"7537worker1 # [ 171.702862] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=7527538lb # [ 171.720191] niks3-server[1103]: 2026/09/23 04:44:59 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7539worker1 # [ 171.709176] nix-grpc-daemon[1735]: ts=2026-09-23T04:44:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707540worker1 # [ 171.720108] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7541worker1 # [ 171.726146] nix-daemon[863]: reaped child process 2103, status = succeeded7542worker1 # [ 171.729296] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7543worker1 # [ 172.966258] nix-daemon[863]: reaped child process 2112, status = succeeded7544worker1 # [ 173.041966] nix-daemon[863]: accepted connection from pid 2128, user nix-grpc-daemon (trusted)7545worker1 # [ 173.054471] nix-daemon[863]: reaped child process 2131, status = succeeded7546lb # [ 173.078462] niks3-server[1103]: 2026/09/23 04:45:00 INFO Received uploads request method=POST path=/api/pending_closures7547worker1 # [ 173.073276] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.114Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7548worker1 # [ 173.074710] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.116Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7549lb # [ 173.120070] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7550lb # [ 173.124232] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7551lb # [ 173.131632] niks3-server[1103]: 2026/09/23 04:45:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7552lb # [ 173.135786] niks3-server[1103]: 2026/09/23 04:45:01 INFO Signed narinfos id=85 count=17553worker1 # [ 173.122651] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.164Z level=INFO msg="Uploading 1 narinfos"7554lb # [ 173.142458] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7555lb # [ 173.146804] niks3-server[1103]: 2026/09/23 04:45:01 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7556lb # [ 173.150364] niks3-server[1103]: 2026/09/23 04:45:01 INFO Completed upload id=857557worker1 # [ 173.134925] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.176Z level=INFO msg="Upload complete. (197ms)"7558lb # [ 173.155417] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7559worker1 # [ 173.139899] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17560worker2 # [ 173.127919] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:00Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17561worker1 # [ 173.149526] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707562worker1 # [ 173.158648] nix-daemon[863]: reaped child process 2110, status = succeeded7563worker1 # [ 173.160247] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707564worker1 # [ 173.166149] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.207Z level=INFO msg="All 2 paths already cached"7565worker1 # [ 173.170413] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559707566worker1 # [ 173.175445] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7567worker1 # [ 173.258525] nix-daemon[863]: accepted connection from pid 2135, user nix-grpc-daemon (trusted)7568worker1 # [ 173.271901] nix-daemon[863]: reaped child process 2138, status = succeeded7569lb # [ 173.293450] niks3-server[1103]: 2026/09/23 04:45:01 INFO Received uploads request method=POST path=/api/pending_closures7570worker1 # [ 173.286538] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.328Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7571worker1 # [ 173.288595] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.329Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7572lb # [ 173.324728] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7573lb # [ 173.331543] niks3-server[1103]: 2026/09/23 04:45:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7574lb # [ 173.336496] niks3-server[1103]: 2026/09/23 04:45:01 INFO Signed narinfos id=86 count=17575worker1 # [ 173.323571] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.364Z level=INFO msg="Uploading 1 narinfos"7576lb # [ 173.342486] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7577lb # [ 173.349873] niks3-server[1103]: 2026/09/23 04:45:01 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7578lb # [ 173.353451] niks3-server[1103]: 2026/09/23 04:45:01 INFO Completed upload id=867579worker1 # [ 173.339163] nix-grpc-daemon[1800]: time=2026-09-23T04:45:01.380Z level=INFO msg="Upload complete. (158ms)"7580lb # [ 173.356652] niks3-server[1103]: 2026/09/23 04:45:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7581worker1 # [ 173.341429] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=0 paths=1 nar_bytes_in=7527582worker1 # [ 173.345386] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559707583worker1 # [ 173.351639] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7584worker1 # [ 173.353959] nix-daemon[863]: reaped child process 2133, status = succeeded7585worker1 # [ 173.359235] nix-daemon[863]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)7586worker1 # [ 174.398408] nix-daemon[863]: reaped child process 2142, status = succeeded7587worker1 # [ 174.481877] nix-daemon[863]: accepted connection from pid 2158, user nix-grpc-daemon (trusted)7588worker1 # [ 174.493826] nix-daemon[863]: reaped child process 2161, status = succeeded7589lb # [ 174.517276] niks3-server[1103]: 2026/09/23 04:45:02 INFO Received uploads request method=POST path=/api/pending_closures7590worker1 # [ 174.510586] nix-grpc-daemon[1800]: time=2026-09-23T04:45:02.551Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7591worker1 # [ 174.514066] nix-grpc-daemon[1800]: time=2026-09-23T04:45:02.553Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7592lb # [ 174.556174] niks3-server[1103]: 2026/09/23 04:45:02 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7593lb # [ 174.569879] niks3-server[1103]: 2026/09/23 04:45:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7594lb # [ 174.576704] niks3-server[1103]: 2026/09/23 04:45:02 INFO Signed narinfos id=87 count=17595worker1 # [ 174.563651] nix-grpc-daemon[1800]: time=2026-09-23T04:45:02.605Z level=INFO msg="Uploading 1 narinfos"7596lb # [ 174.584123] niks3-server[1103]: 2026/09/23 04:45:02 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7597lb # [ 174.589769] niks3-server[1103]: 2026/09/23 04:45:02 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7598lb # [ 174.598065] niks3-server[1103]: 2026/09/23 04:45:02 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7599lb # [ 174.601620] niks3-server[1103]: 2026/09/23 04:45:02 INFO Completed upload id=877600worker1 # [ 174.586107] nix-grpc-daemon[1800]: time=2026-09-23T04:45:02.627Z level=INFO msg="Upload complete. (219ms)"7601lb # [ 174.605830] niks3-server[1103]: 2026/09/23 04:45:02 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7602worker1 # [ 174.592994] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55970 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17603worker1 # [ 174.607994] nix-daemon[863]: reaped child process 2140, status = succeeded7604client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.76 seconds)7605worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7606lb # [ 174.674907] envoy[904]: conn peer=[2001:db8:1::1]:41614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27051 tx=12291 ms=135767607worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7608client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7609client # [ 174.752865] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7610client # Running as unit: gpu.service; invocation ID: 5dde8cfa39714c03af7f96fb97a15d167611client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7612lb: must succeed: curl -sf localhost:9901/clusters7613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7614lb: must succeed: curl -sf localhost:9901/clusters7615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7616worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7617worker1 # [ 174.867860] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559707618worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7619client # [ 174.900136] nix[1579]: don't know how to build these paths:7620client # [ 174.901263] nix[1579]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7621worker2 # [ 174.863349] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727622worker2 # [ 174.883445] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:02Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07623client # [ 174.928322] nix[1579]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7624lb: must succeed: curl -sf localhost:9901/clusters7625lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7626lb: must succeed: curl -sf localhost:9901/clusters7627lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7628worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7629worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7630client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7631client # [ 204.929755] nix[1579]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7632client # [ 204.939660] nix[1579]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7633client # [ 204.974558] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7634client # [ 204.977907] systemd[1]: gpu.service: Failed with result 'exit-code'.7635client # [ 204.982710] systemd[1]: gpu.service: Consumed 147ms CPU time over 30.228s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7636client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.45 seconds)7637client: must succeed: journalctl -u gpu -o cat7638client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7639lb: must succeed: curl -sf localhost:9901/clusters7640lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7641lb: must succeed: curl -sf localhost:9901/clusters7642lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7643worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7644worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7645(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.70 seconds)7646subtest: low disk drains a worker and builds go to the other7647worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7648worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.01 seconds)7649worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7650worker1 # [ 207.305974] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:35Z level=info event=unhealthy reason=min_free7651worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)7652client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27653client # don't know how to build these paths:7654client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7655worker2 # [ 207.812709] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544727656worker1 # [ 207.829455] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559707657worker2 # [ 207.837787] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07658worker2 # [ 207.842752] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17659worker2 # [ 207.882461] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727660worker2 # [ 207.895513] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727661client # copying 1 paths to 'lb:50051'...7662client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7663worker2 # [ 207.908272] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544727664worker2 # [ 207.919625] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7665worker2 # [ 208.023186] nix-daemon[854]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)7666worker2 # [ 208.038213] nix-daemon[854]: reaped child process 2018, status = succeeded7667lb # [ 208.073554] niks3-server[1103]: 2026/09/23 04:45:35 INFO Received uploads request method=POST path=/api/pending_closures7668worker2 # [ 208.053910] nix-grpc-daemon[1715]: time=2026-09-23T04:45:35.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7669worker2 # [ 208.055418] nix-grpc-daemon[1715]: time=2026-09-23T04:45:35.874Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7670lb # [ 208.111406] niks3-server[1103]: 2026/09/23 04:45:36 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7671lb # [ 208.120956] niks3-server[1103]: 2026/09/23 04:45:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7672worker2 # [ 208.099102] nix-grpc-daemon[1715]: time=2026-09-23T04:45:35.919Z level=INFO msg="Uploading 1 narinfos"7673lb # [ 208.127769] niks3-server[1103]: 2026/09/23 04:45:36 INFO Signed narinfos id=88 count=17674lb # [ 208.139719] niks3-server[1103]: 2026/09/23 04:45:36 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7675lb # [ 208.144610] niks3-server[1103]: 2026/09/23 04:45:36 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7676lb # [ 208.149363] niks3-server[1103]: 2026/09/23 04:45:36 INFO Completed upload id=887677lb # [ 208.150709] niks3-server[1103]: 2026/09/23 04:45:36 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7678worker2 # [ 208.120665] nix-grpc-daemon[1715]: time=2026-09-23T04:45:35.941Z level=INFO msg="Upload complete. (189ms)"7679worker2 # [ 208.125636] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=5127680worker2 # [ 208.131676] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727681worker2 # [ 208.142482] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7682worker2 # [ 208.152637] nix-daemon[854]: reaped child process 2013, status = succeeded7683worker2 # [ 208.156834] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7684client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7685client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7686client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7687client # farm-a> LOG-farm-a-drain7688worker2 # [ 209.195524] nix-daemon[854]: reaped child process 2022, status = succeeded7689worker2 # [ 209.280088] nix-daemon[854]: accepted connection from pid 2038, user nix-grpc-daemon (trusted)7690worker2 # [ 209.292591] nix-daemon[854]: reaped child process 2041, status = succeeded7691lb # [ 209.327509] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received uploads request method=POST path=/api/pending_closures7692worker2 # [ 209.308406] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7693worker2 # [ 209.310611] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.130Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7694lb # [ 209.366496] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7695lb # [ 209.371889] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7696lb # [ 209.376692] niks3-server[1103]: 2026/09/23 04:45:37 INFO Signed narinfos id=89 count=17697worker2 # [ 209.350602] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.171Z level=INFO msg="Uploading 1 narinfos"7698lb # [ 209.382664] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7699lb # [ 209.385812] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7700lb # [ 209.391603] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7701lb # [ 209.394476] niks3-server[1103]: 2026/09/23 04:45:37 INFO Completed upload id=897702worker2 # [ 209.365588] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.186Z level=INFO msg="Upload complete. (201ms)"7703lb # [ 209.396310] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7704worker2 # [ 209.370821] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17705worker2 # [ 209.382828] nix-daemon[854]: reaped child process 2020, status = succeeded7706worker2 # [ 209.417683] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07707worker2 # [ 209.423417] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17708worker2 # [ 209.461961] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727709worker2 # [ 209.474956] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727710worker2 # [ 209.488909] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.309Z level=INFO msg="All 1 paths already cached"7711client # copying 1 paths to 'lb:50051'...7712client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7713worker2 # [ 209.497593] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544727714worker2 # [ 209.507351] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7715worker2 # [ 209.606334] nix-daemon[854]: accepted connection from pid 2047, user nix-grpc-daemon (trusted)7716worker2 # [ 209.619731] nix-daemon[854]: reaped child process 2050, status = succeeded7717lb # [ 209.655197] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received uploads request method=POST path=/api/pending_closures7718worker2 # [ 209.635062] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.456Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7719worker2 # [ 209.636619] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.457Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7720lb # [ 209.683529] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7721lb # [ 209.689699] niks3-server[1103]: 2026/09/23 04:45:37 INFO Signed narinfos id=90 count=17722worker2 # [ 209.663937] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.484Z level=INFO msg="Uploading 1 narinfos"7723lb # [ 209.704206] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7724lb # [ 209.708552] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7725lb # [ 209.715956] niks3-server[1103]: 2026/09/23 04:45:37 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7726lb # [ 209.718869] niks3-server[1103]: 2026/09/23 04:45:37 INFO Completed upload id=907727worker2 # [ 209.690089] nix-grpc-daemon[1715]: time=2026-09-23T04:45:37.511Z level=INFO msg="Upload complete. (170ms)"7728lb # [ 209.721765] niks3-server[1103]: 2026/09/23 04:45:37 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7729worker2 # [ 209.693857] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=6247730worker2 # [ 209.698788] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727731worker2 # [ 209.709542] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7732worker2 # [ 209.715120] nix-daemon[854]: reaped child process 2045, status = succeeded7733worker2 # [ 209.717846] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7734client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7735client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7736client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7737client # farm-b> LOG-farm-b-drain7738worker2 # [ 211.152627] nix-daemon[854]: reaped child process 2054, status = succeeded7739worker2 # [ 211.231249] nix-daemon[854]: accepted connection from pid 2070, user nix-grpc-daemon (trusted)7740worker2 # [ 211.243876] nix-daemon[854]: reaped child process 2073, status = succeeded7741lb # [ 211.281654] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received uploads request method=POST path=/api/pending_closures7742worker2 # [ 211.263836] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7743worker2 # [ 211.265415] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.086Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7744lb # [ 211.328167] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7745lb # [ 211.335562] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7746lb # [ 211.345240] niks3-server[1103]: 2026/09/23 04:45:39 INFO Signed narinfos id=91 count=17747worker2 # [ 211.319174] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.139Z level=INFO msg="Uploading 1 narinfos"7748lb # [ 211.350148] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7749lb # [ 211.355468] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7750lb # [ 211.363487] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7751lb # [ 211.367370] niks3-server[1103]: 2026/09/23 04:45:39 INFO Completed upload id=917752lb # [ 211.370497] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7753worker2 # [ 211.340186] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.161Z level=INFO msg="Upload complete. (212ms)"7754worker2 # [ 211.346812] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17755worker2 # [ 211.358445] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07756worker2 # [ 211.363189] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17757worker2 # [ 211.372313] nix-daemon[854]: reaped child process 2052, status = succeeded7758worker2 # [ 211.402825] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727759worker2 # [ 211.415450] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544727760worker2 # [ 211.429084] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.249Z level=INFO msg="All 2 paths already cached"7761client # copying 1 paths to 'lb:50051'...7762client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7763worker2 # [ 211.439304] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544727764worker2 # [ 211.447756] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7765worker2 # [ 211.543383] nix-daemon[854]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)7766worker2 # [ 211.557993] nix-daemon[854]: reaped child process 2082, status = succeeded7767lb # [ 211.592727] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received uploads request method=POST path=/api/pending_closures7768worker2 # [ 211.572994] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7769worker2 # [ 211.574566] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.395Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7770lb # [ 211.627142] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7771lb # [ 211.632976] niks3-server[1103]: 2026/09/23 04:45:39 INFO Signed narinfos id=92 count=17772worker2 # [ 211.607593] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.428Z level=INFO msg="Uploading 1 narinfos"7773lb # [ 211.639207] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7774lb # [ 211.652048] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7775lb # [ 211.657575] niks3-server[1103]: 2026/09/23 04:45:39 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7776lb # [ 211.662973] niks3-server[1103]: 2026/09/23 04:45:39 INFO Completed upload id=927777worker2 # [ 211.634245] nix-grpc-daemon[1715]: time=2026-09-23T04:45:39.455Z level=INFO msg="Upload complete. (174ms)"7778lb # [ 211.668561] niks3-server[1103]: 2026/09/23 04:45:39 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7779worker2 # [ 211.638212] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=0 paths=1 nar_bytes_in=7527780worker2 # [ 211.644453] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544727781worker2 # [ 211.656116] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7782worker2 # [ 211.660287] nix-daemon[854]: reaped child process 2077, status = succeeded7783worker2 # [ 211.665195] nix-daemon[854]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)7784client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7785client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7786client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7787client # farm-top> LOG-farm-top-drain7788worker2 # [ 212.905146] nix-daemon[854]: reaped child process 2086, status = succeeded7789worker2 # [ 212.987297] nix-daemon[854]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)7790worker2 # [ 212.999989] nix-daemon[854]: reaped child process 2105, status = succeeded7791lb # [ 213.037729] niks3-server[1103]: 2026/09/23 04:45:40 INFO Received uploads request method=POST path=/api/pending_closures7792worker2 # [ 213.018128] nix-grpc-daemon[1715]: time=2026-09-23T04:45:40.838Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7793worker2 # [ 213.021057] nix-grpc-daemon[1715]: time=2026-09-23T04:45:40.840Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7794lb # [ 213.078750] niks3-server[1103]: 2026/09/23 04:45:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7795lb # [ 213.084352] niks3-server[1103]: 2026/09/23 04:45:41 INFO Signed narinfos id=93 count=17796worker2 # [ 213.058598] nix-grpc-daemon[1715]: time=2026-09-23T04:45:40.879Z level=INFO msg="Uploading 1 narinfos"7797lb # [ 213.092953] niks3-server[1103]: 2026/09/23 04:45:41 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7798lb # [ 213.103104] niks3-server[1103]: 2026/09/23 04:45:41 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7799lb # [ 213.107298] niks3-server[1103]: 2026/09/23 04:45:41 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7800lb # [ 213.117774] niks3-server[1103]: 2026/09/23 04:45:41 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7801lb # [ 213.121404] niks3-server[1103]: 2026/09/23 04:45:41 INFO Completed upload id=937802worker2 # [ 213.092823] nix-grpc-daemon[1715]: time=2026-09-23T04:45:40.913Z level=INFO msg="Upload complete. (219ms)"7803lb # [ 213.123378] niks3-server[1103]: 2026/09/23 04:45:41 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7804worker2 # [ 213.099613] nix-grpc-daemon[1626]: ts=2026-09-23T04:45:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54472 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=17805worker2 # [ 213.114280] nix-daemon[854]: reaped child process 2084, status = succeeded7806client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.48 seconds)7807client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7808client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7809worker1: must succeed: rm /nix/.rw-store/fill7810worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7811worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7812worker1 # [ 214.308904] nix-grpc-daemon[1735]: ts=2026-09-23T04:45:42Z level=info event=healthy reason=min_free7813worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7814(finished: subtest: low disk drains a worker and builds go to the other, in 8.93 seconds)7815(finished: run the VM test script, in 215.43 seconds)7816lb # [ 214.676535] envoy[904]: conn peer=[2001:db8:1::1]:53358 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300577817lb # [ 214.680522] envoy[904]: conn peer=[2001:db8:1::1]:46544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4374 ms=53057818test script finished in 215.61s7819cleanup7820kill QemuMachine (pid 49)7821client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7822client # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7823client # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7824client # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7825kill QemuMachine (pid 52)7826lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7827lb # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7828lb # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7829lb # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7830kill QemuMachine (pid 50)7831worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7832worker1 # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7833worker1 # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7834worker1 # [2026-09-23T04:45:43Z INFO virtiofsd] Client disconnected, shutting down7835kill QemuMachine (pid 51)7836worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7837worker2 # [2026-09-23T04:45:44Z INFO virtiofsd] Client disconnected, shutting down7838worker2 # [2026-09-23T04:45:44Z INFO virtiofsd] Client disconnected, shutting down7839worker2 # [2026-09-23T04:45:44Z INFO virtiofsd] Client disconnected, shutting down7840(finished: cleanup, in 1.15 seconds)7841additionally exposed symbols:7842 client, lb, worker1, worker2,7843 vlan1,7844 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh