vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #548
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1: QEMU running (pid 52)13worker1 # Disk image does not exist, creating the virtualisation disk image...14worker1 # Formatting '/build/vm-state-worker1/tmp.91TojKDAU9', fmt=raw size=107374182415worker2 # Disk image does not exist, creating the virtualisation disk image...16worker1 # mke2fs 1.47.4 (6-Mar-2025)17worker1 # Discarding device blocks: 0/262144 done18worker2 # Formatting '/build/vm-state-worker2/tmp.uiyAb6cUP4', fmt=raw size=107374182419worker2: QEMU running (pid 51)20worker2 # mke2fs 1.47.4 (6-Mar-2025)21worker2 # Discarding device blocks: 0/262144 done22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23client # Disk image does not exist, creating the virtualisation disk image...24worker1 # Filesystem UUID: 8851ecf9-41c2-47fe-beec-3e4751d7677225client: QEMU running (pid 49)26worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes27client # Formatting '/build/vm-state-client/tmp.xJcHzaOym9', fmt=raw size=107374182428worker2 # Filesystem UUID: ed1e66d5-357a-4014-b002-15e4458b76d329client # mke2fs 1.47.4 (6-Mar-2025)30worker2 # Superblock backups stored on blocks:31client # Discarding device blocks: 0/262144 done32worker2 # 32768, 98304, 163840, 22937633client # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # 35client # Filesystem UUID: 76799aa6-4ca2-4066-b0be-376a60231b6436worker2 # Allocating group tables: 0/8 done37client # Superblock backups stored on blocks:38worker2 # Writing inode tables: 0/8 done39client # 32768, 98304, 163840, 22937640worker2 # Creating journal (8192 blocks): done41client # 42worker2 # Writing superblocks and filesystem accounting information: 0/8 done43client # Allocating group tables: 0/8 done44worker2 # 45client # Writing inode tables: 0/8 done46worker2 # Virtualisation disk image created.47client # Creating journal (8192 blocks): done48worker2 # Starting virtiofs daemons...49client # Writing superblocks and filesystem accounting information: 0/8 done50worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)51client # 52client # Virtualisation disk image created.53worker1 # Superblock backups stored on blocks:54client # Starting virtiofs daemons...55worker1 # 32768, 98304, 163840, 22937656client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)57worker1 # 58client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether59worker1 # Allocating group tables: 0/8 done60client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...61worker1 # Writing inode tables: 0/8 done62client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)63worker1 # Creating journal (8192 blocks): done64client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65worker1 # Writing superblocks and filesystem accounting information: 0/8 done66client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...67worker1 # 68client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69lb # Disk image does not exist, creating the virtualisation disk image...70worker1 # Virtualisation disk image created.71client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72lb # Formatting '/build/vm-state-lb/tmp.wDsZraxWWG', fmt=raw size=107374182473worker1 # Starting virtiofs daemons...74client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...75lb # mke2fs 1.47.4 (6-Mar-2025)76worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests78lb # Discarding device blocks: 0/262144 done79worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests81lb # Creating filesystem with 262144 4k blocks and 65536 inodes82worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...83client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests84lb # Filesystem UUID: e136e456-ff5d-48ae-b492-c371e2be883585worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87lb: QEMU running (pid 50)88lb # Superblock backups stored on blocks:89worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90lb # 32768, 98304, 163840, 22937691worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...93lb # 94(finished: start all VMs, in 0.22 seconds)95worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97lb # Allocating group tables: 0/8 done98client: waiting for unit network-addresses-eth1.service99worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...100worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101lb # Writing inode tables: 0/8 done102client: waiting for the VM to finish booting103worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...104worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)105worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests106worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests108worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...109worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests110worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...111lb # Creating journal (8192 blocks): done112worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests113lb # Writing superblocks and filesystem accounting information: 0/8 done114worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests115lb # 116worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137worker2 # 138worker2 # 139client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142client # 143client # 144client # 145client # 146worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147worker2 # 148worker2 # 149worker2 # 150worker2 # 151client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00152worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00153client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...154client # 155client # 156worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157worker2 # 158worker2 # 159worker1 # 160worker1 # 161worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00162lb # 163lb # 164worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...165worker1 # 166worker1 # 167worker1 # 168worker1 # 169lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00170worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00171lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172lb # 173lb # 174lb # 175lb # 176worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...177worker1 # 178worker1 # 179lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180client # Booting from ROM...181worker2 # Booting from ROM...182lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...183lb # 184lb # 185worker1 # Booting from ROM...186lb # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test/init regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration console=ttyS0,115200n8 console=tty0190client # [ 0.000000] BIOS-provided physical RAM map:191client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable192worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wdxq8bvyk71y06pp5xincwaxz514413f-nixos-system-worker2-test/init regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration console=ttyS0,115200n8 console=tty0193worker2 # [ 0.000000] BIOS-provided physical RAM map:194client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved195worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable196client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved197worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable199worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved200client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved201worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable202client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved203worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved204client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved205worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved206client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved207worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved208client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved209worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved210client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved211client # [ 0.000000] NX (Execute Disable) protection: active212worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved213client # [ 0.000000] APIC: Static calls initialized214client # [ 0.000000] SMBIOS 2.8 present.215worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved216worker2 # [ 0.000000] NX (Execute Disable) protection: active217worker2 # [ 0.000000] APIC: Static calls initialized218client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014219worker2 # [ 0.000000] SMBIOS 2.8 present.220client # [ 0.000000] DMI: Memory slots populated: 1/1221client # [ 0.000000] Hypervisor detected: KVM222worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014223client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000224worker2 # [ 0.000000] DMI: Memory slots populated: 1/1225client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00226worker2 # [ 0.000000] Hypervisor detected: KVM227client # [ 0.000000] kvm-clock: using sched offset of 446635667 cycles228worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000229worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00230client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns231worker2 # [ 0.000000] kvm-clock: using sched offset of 471439904 cycles232client # [ 0.000005] tsc: Detected 2400.012 MHz processor233client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000234worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns235worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor236client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs237worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000238client # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT239worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026240client # [ 0.002776] found SMP MP-table at [mem 0x000f5450-0x000f545f]241worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs242client # [ 0.002787] Using GB pages for direct mapping243worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT244client # [ 0.002913] RAMDISK: [mem 0x5e370000-0x5ffcffff]245worker2 # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f]246client # [ 0.002923] ACPI: Early table checksum verification disabled247worker2 # [ 0.002754] Using GB pages for direct mapping248client # [ 0.002927] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )249worker2 # [ 0.002848] RAMDISK: [mem 0x5e36f000-0x5ffcffff]250worker2 # [ 0.002859] ACPI: Early table checksum verification disabled251client # [ 0.002930] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)252worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mjibqls45h3zals0927jhbwnzswp7x1m-nixos-system-worker1-test/init regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration console=ttyS0,115200n8 console=tty0253worker1 # [ 0.000000] BIOS-provided physical RAM map:254worker2 # [ 0.002862] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )255lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026256client # [ 0.002934] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)257worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable258worker2 # [ 0.002866] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)259client # [ 0.002941] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)260worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved261client # [ 0.002943] ACPI: FACS 0x000000005FFE0000 000040262worker2 # [ 0.002870] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)263worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved264worker2 # [ 0.002878] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.002945] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)266worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable267worker2 # [ 0.002880] ACPI: FACS 0x000000005FFE0000 000040268lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0269client # [ 0.002947] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)270worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved271lb # [ 0.000000] BIOS-provided physical RAM map:272worker2 # [ 0.002882] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved274client # [ 0.002948] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)275lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable276worker2 # [ 0.002883] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)277lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved278worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved279client # [ 0.002950] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)280worker2 # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)281worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved282lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved283client # [ 0.002951] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]284worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved285lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable286worker2 # [ 0.002886] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)287client # [ 0.002952] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]288lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved289worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved290worker2 # [ 0.002888] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]291client # [ 0.002953] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]292worker1 # [ 0.000000] NX (Execute Disable) protection: active293lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved294worker2 # [ 0.002889] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]295worker1 # [ 0.000000] APIC: Static calls initialized296client # [ 0.002953] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]297worker1 # [ 0.000000] SMBIOS 2.8 present.298lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved299worker2 # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]300client # [ 0.002953] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]301lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved302worker2 # [ 0.002890] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]303client # [ 0.002954] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]304worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014305worker1 # [ 0.000000] DMI: Memory slots populated: 1/1306lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved307worker2 # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]308client # [ 0.002954] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]309worker1 # [ 0.000000] Hypervisor detected: KVM310client # [ 0.003186] No NUMA configuration found311lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved312worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000313worker2 # [ 0.002891] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]314lb # [ 0.000000] NX (Execute Disable) protection: active315client # [ 0.003186] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]316worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00317lb # [ 0.000000] APIC: Static calls initialized318worker2 # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]319client # [ 0.003189] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]320lb # [ 0.000000] SMBIOS 2.8 present.321worker2 # [ 0.003127] No NUMA configuration found322worker1 # [ 0.000001] kvm-clock: using sched offset of 482185618 cycles323client # [ 0.005851] Zone ranges:324worker2 # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]325client # [ 0.005852] DMA [mem 0x0000000000001000-0x0000000000ffffff]326lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014327worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns328worker2 # [ 0.003130] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]329client # [ 0.005854] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]330worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor331worker2 # [ 0.005854] Zone ranges:332client # [ 0.005855] Normal empty333lb # [ 0.000000] DMI: Memory slots populated: 1/1334client # [ 0.005856] Device empty335lb # [ 0.000000] Hypervisor detected: KVM336worker2 # [ 0.005855] DMA [mem 0x0000000000001000-0x0000000000ffffff]337lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000338worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000339client # [ 0.005857] Movable zone start for each node340worker2 # [ 0.005857] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]341client # [ 0.005858] Early memory node ranges342worker1 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs343lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00344worker2 # [ 0.005858] Normal empty345worker2 # [ 0.005859] Device empty346lb # [ 0.000000] kvm-clock: using sched offset of 475440794 cycles347client # [ 0.005858] node 0: [mem 0x0000000000001000-0x000000000009efff]348worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT349worker2 # [ 0.005859] Movable zone start for each node350client # [ 0.005859] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]351worker2 # [ 0.005860] Early memory node ranges352worker1 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f]353lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns354worker1 # [ 0.002739] Using GB pages for direct mapping355worker2 # [ 0.005861] node 0: [mem 0x0000000000001000-0x000000000009efff]356client # [ 0.005860] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]357lb # [ 0.000004] tsc: Detected 2400.012 MHz processor358worker1 # [ 0.002899] RAMDISK: [mem 0x5e370000-0x5ffcffff]359client # [ 0.005882] On node 0, zone DMA: 1 pages in unavailable ranges360worker2 # [ 0.005862] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]361lb # [ 0.000803] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000362worker1 # [ 0.002906] ACPI: Early table checksum verification disabled363client # [ 0.006159] On node 0, zone DMA: 97 pages in unavailable ranges364worker2 # [ 0.005863] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]365worker1 # [ 0.002909] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )366lb # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs367client # [ 0.033190] On node 0, zone DMA32: 40 pages in unavailable ranges368worker2 # [ 0.005883] On node 0, zone DMA: 1 pages in unavailable ranges369lb # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT370client # [ 0.033659] ACPI: PM-Timer IO Port: 0x608371worker1 # [ 0.002913] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)372worker2 # [ 0.006140] On node 0, zone DMA: 97 pages in unavailable ranges373lb # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]374client # [ 0.033671] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])375worker2 # [ 0.032380] On node 0, zone DMA32: 40 pages in unavailable ranges376worker1 # [ 0.002917] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)377lb # [ 0.002733] Using GB pages for direct mapping378client # [ 0.033699] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23379worker2 # [ 0.032842] ACPI: PM-Timer IO Port: 0x608380lb # [ 0.002855] RAMDISK: [mem 0x5e36e000-0x5ffcffff]381worker1 # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)382client # [ 0.033701] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)383worker2 # [ 0.032854] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])384lb # [ 0.002862] ACPI: Early table checksum verification disabled385worker1 # [ 0.002925] ACPI: FACS 0x000000005FFE0000 000040386client # [ 0.033703] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)387lb # [ 0.002864] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )388worker2 # [ 0.032883] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23389worker1 # [ 0.002927] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)390client # [ 0.033704] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)391worker2 # [ 0.032886] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)392lb # [ 0.002868] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)393client # [ 0.033705] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)394worker2 # [ 0.032887] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)395worker1 # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)396lb # [ 0.002872] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)397client # [ 0.033706] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)398worker2 # [ 0.032888] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)399worker1 # [ 0.002930] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)400lb # [ 0.002878] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)401client # [ 0.033709] ACPI: Using ACPI (MADT) for SMP configuration information402worker2 # [ 0.032889] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)403lb # [ 0.002880] ACPI: FACS 0x000000005FFE0000 000040404client # [ 0.033709] ACPI: HPET id: 0x8086a201 base: 0xfed00000405worker1 # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)406worker2 # [ 0.032890] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)407client # [ 0.033713] TSC deadline timer available408lb # [ 0.002881] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)409worker1 # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]410client # [ 0.033717] CPU topo: Max. logical packages: 1411worker2 # [ 0.032893] ACPI: Using ACPI (MADT) for SMP configuration information412worker2 # [ 0.032894] ACPI: HPET id: 0x8086a201 base: 0xfed00000413client # [ 0.033718] CPU topo: Max. logical dies: 1414worker1 # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]415lb # [ 0.002883] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)416worker2 # [ 0.032897] TSC deadline timer available417client # [ 0.033718] CPU topo: Max. dies per package: 1418worker1 # [ 0.002934] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]419worker2 # [ 0.032901] CPU topo: Max. logical packages: 1420client # [ 0.033722] CPU topo: Max. threads per core: 1421lb # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)422worker2 # [ 0.032902] CPU topo: Max. logical dies: 1423client # [ 0.033722] CPU topo: Num. cores per package: 1424worker1 # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]425worker2 # [ 0.032902] CPU topo: Max. dies per package: 1426client # [ 0.033723] CPU topo: Num. threads per package: 1427lb # [ 0.002886] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)428worker2 # [ 0.032905] CPU topo: Max. threads per core: 1429worker1 # [ 0.002935] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]430client # [ 0.033723] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs431lb # [ 0.002887] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]432worker2 # [ 0.032906] CPU topo: Num. cores per package: 1433worker1 # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]434client # [ 0.033742] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()435worker2 # [ 0.032906] CPU topo: Num. threads per package: 1436lb # [ 0.002888] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]437worker1 # [ 0.002936] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]438worker2 # [ 0.032906] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs439client # [ 0.033773] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]440worker1 # [ 0.003155] No NUMA configuration found441lb # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]442worker2 # [ 0.032924] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()443client # [ 0.033775] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]444lb # [ 0.002889] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]445worker1 # [ 0.003156] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]446client # [ 0.033776] [mem 0x60000000-0xafffffff] available for PCI devices447worker2 # [ 0.032953] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]448worker1 # [ 0.003158] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]449lb # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]450worker1 # [ 0.005368] Zone ranges:451client # [ 0.033778] Booting paravirtualized kernel on KVM452worker2 # [ 0.032955] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]453lb # [ 0.002890] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]454worker1 # [ 0.005369] DMA [mem 0x0000000000001000-0x0000000000ffffff]455worker2 # [ 0.032956] [mem 0x60000000-0xafffffff] available for PCI devices456worker1 # [ 0.005370] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]457lb # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]458client # [ 0.033781] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns459worker1 # [ 0.005371] Normal empty460worker2 # [ 0.032958] Booting paravirtualized kernel on KVM461lb # [ 0.003120] No NUMA configuration found462worker1 # [ 0.005372] Device empty463client # [ 0.038251] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1464worker1 # [ 0.005372] Movable zone start for each node465lb # [ 0.003121] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]466worker2 # [ 0.032961] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns467client # [ 0.040393] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152468worker1 # [ 0.005373] Early memory node ranges469lb # [ 0.003124] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]470lb # [ 0.005268] Zone ranges:471client # [ 0.040448] kvm-guest: PV spinlocks disabled, single CPU472worker1 # [ 0.005373] node 0: [mem 0x0000000000001000-0x000000000009efff]473worker2 # [ 0.037441] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1474lb # [ 0.005269] DMA [mem 0x0000000000001000-0x0000000000ffffff]475worker1 # [ 0.005374] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]476worker2 # [ 0.039593] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152477lb # [ 0.005270] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]478worker2 # [ 0.039638] kvm-guest: PV spinlocks disabled, single CPU479worker1 # [ 0.005376] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]480lb # [ 0.005271] Normal empty481lb # [ 0.005272] Device empty482worker1 # [ 0.005397] On node 0, zone DMA: 1 pages in unavailable ranges483lb # [ 0.005272] Movable zone start for each node484worker1 # [ 0.005652] On node 0, zone DMA: 97 pages in unavailable ranges485lb # [ 0.005273] Early memory node ranges486client # [ 0.040449] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test/init regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration console=ttyS0,115200n8 console=tty0487worker1 # [ 0.030835] On node 0, zone DMA32: 40 pages in unavailable ranges488lb # [ 0.005273] node 0: [mem 0x0000000000001000-0x000000000009efff]489worker1 # [ 0.031282] ACPI: PM-Timer IO Port: 0x608490lb # [ 0.005274] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]491worker1 # [ 0.031292] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])492worker2 # [ 0.039640] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wdxq8bvyk71y06pp5xincwaxz514413f-nixos-system-worker2-test/init regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration console=ttyS0,115200n8 console=tty0493client # [ 0.040544] Unknown kernel command line parameters "regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration", will be passed to user space.494lb # [ 0.005275] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]495client # [ 0.040556] random: crng init done496worker1 # [ 0.031319] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23497lb # [ 0.005293] On node 0, zone DMA: 1 pages in unavailable ranges498worker1 # [ 0.031321] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)499client # [ 0.040557] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes500lb # [ 0.005543] On node 0, zone DMA: 97 pages in unavailable ranges501worker2 # [ 0.039733] Unknown kernel command line parameters "regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration", will be passed to user space.502worker1 # [ 0.031322] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)503worker2 # [ 0.039745] random: crng init done504lb # [ 0.030610] On node 0, zone DMA32: 40 pages in unavailable ranges505client # [ 0.042650] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)506lb # [ 0.031069] ACPI: PM-Timer IO Port: 0x608507worker1 # [ 0.031324] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)508worker2 # [ 0.039746] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes509client # [ 0.043700] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)510lb # [ 0.031079] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])511client # [ 0.043730] Fallback order for Node 0: 0512worker1 # [ 0.031324] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)513worker2 # [ 0.041863] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)514lb # [ 0.031106] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23515client # [ 0.043733] Built 1 zonelists, mobility grouping on. Total pages: 393078516worker1 # [ 0.031325] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)517client # [ 0.043733] Policy zone: DMA32518worker2 # [ 0.042930] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)519lb # [ 0.031108] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)520worker1 # [ 0.031327] ACPI: Using ACPI (MADT) for SMP configuration information521worker2 # [ 0.042963] Fallback order for Node 0: 0522client # [ 0.046220] mem auto-init: stack:all(zero), heap alloc:on, heap free:off523lb # [ 0.031110] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)524worker1 # [ 0.031328] ACPI: HPET id: 0x8086a201 base: 0xfed00000525worker2 # [ 0.042966] Built 1 zonelists, mobility grouping on. Total pages: 393078526worker1 # [ 0.031332] TSC deadline timer available527client # [ 0.049581] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1528lb # [ 0.031111] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)529worker2 # [ 0.042967] Policy zone: DMA32530worker1 # [ 0.031336] CPU topo: Max. logical packages: 1531client # [ 0.052848] allocated 3145728 bytes of page_ext532lb # [ 0.031112] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)533worker2 # [ 0.045637] mem auto-init: stack:all(zero), heap alloc:on, heap free:off534worker1 # [ 0.031336] CPU topo: Max. logical dies: 1535client # [ 0.062707] ftrace: allocating 48787 entries in 192 pages536worker1 # [ 0.031337] CPU topo: Max. dies per package: 1537worker2 # [ 0.048757] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1538client # [ 0.062710] ftrace: allocated 192 pages with 2 groups539lb # [ 0.031113] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)540worker1 # [ 0.031340] CPU topo: Max. threads per core: 1541client # [ 0.063531] Dynamic Preempt: lazy542worker2 # [ 0.052080] allocated 3145728 bytes of page_ext543lb # [ 0.031115] ACPI: Using ACPI (MADT) for SMP configuration information544worker1 # [ 0.031341] CPU topo: Num. cores per package: 1545client # [ 0.063679] rcu: Preemptible hierarchical RCU implementation.546worker2 # [ 0.061933] ftrace: allocating 48787 entries in 192 pages547lb # [ 0.031116] ACPI: HPET id: 0x8086a201 base: 0xfed00000548worker1 # [ 0.031341] CPU topo: Num. threads per package: 1549client # [ 0.063680] rcu: RCU event tracing is enabled.550worker2 # [ 0.061935] ftrace: allocated 192 pages with 2 groups551lb # [ 0.031119] TSC deadline timer available552worker2 # [ 0.062774] Dynamic Preempt: lazy553worker1 # [ 0.031341] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs554lb # [ 0.031123] CPU topo: Max. logical packages: 1555client # [ 0.063680] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.556worker2 # [ 0.062950] rcu: Preemptible hierarchical RCU implementation.557lb # [ 0.031124] CPU topo: Max. logical dies: 1558client # [ 0.063682] Trampoline variant of Tasks RCU enabled.559worker1 # [ 0.031358] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()560worker2 # [ 0.062951] rcu: RCU event tracing is enabled.561lb # [ 0.031124] CPU topo: Max. dies per package: 1562client # [ 0.063682] Rude variant of Tasks RCU enabled.563worker1 # [ 0.031386] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]564lb # [ 0.031128] CPU topo: Max. threads per core: 1565worker2 # [ 0.062951] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.566client # [ 0.063682] Tracing variant of Tasks RCU enabled.567lb # [ 0.031128] CPU topo: Num. cores per package: 1568worker2 # [ 0.062953] Trampoline variant of Tasks RCU enabled.569worker1 # [ 0.031388] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]570client # [ 0.063683] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.571lb # [ 0.031129] CPU topo: Num. threads per package: 1572worker2 # [ 0.062953] Rude variant of Tasks RCU enabled.573worker1 # [ 0.031389] [mem 0x60000000-0xafffffff] available for PCI devices574client # [ 0.063684] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1575lb # [ 0.031129] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs576worker2 # [ 0.062954] Tracing variant of Tasks RCU enabled.577worker1 # [ 0.031390] Booting paravirtualized kernel on KVM578worker2 # [ 0.062954] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.579lb # [ 0.031145] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()580client # [ 0.063705] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.581worker1 # [ 0.031392] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns582worker2 # [ 0.062955] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1583lb # [ 0.031174] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]584client # [ 0.063706] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.585worker1 # [ 0.035865] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1586lb # [ 0.031175] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]587worker2 # [ 0.062964] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588client # [ 0.063707] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589worker1 # [ 0.037905] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152590lb # [ 0.031177] [mem 0x60000000-0xafffffff] available for PCI devices591worker2 # [ 0.062966] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.592client # [ 0.068057] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16593lb # [ 0.031178] Booting paravirtualized kernel on KVM594worker1 # [ 0.037948] kvm-guest: PV spinlocks disabled, single CPU595client # [ 0.068328] rcu: srcu_init: Setting srcu_struct sizes based on contention.596worker2 # [ 0.062967] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.597lb # [ 0.031181] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns598worker2 # [ 0.067346] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16599client # [ 0.068336] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns600lb # [ 0.035637] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1601worker2 # [ 0.067624] rcu: srcu_init: Setting srcu_struct sizes based on contention.602lb # [ 0.037706] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152603client # [ 0.068436] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)604lb # [ 0.037748] kvm-guest: PV spinlocks disabled, single CPU605client # [ 0.071969] Console: colour VGA+ 80x25606worker2 # [ 0.067632] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns607worker1 # [ 0.037949] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mjibqls45h3zals0927jhbwnzswp7x1m-nixos-system-worker1-test/init regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration console=ttyS0,115200n8 console=tty0608client # [ 0.071971] printk: legacy console [tty0] enabled609client # [ 0.101567] printk: legacy console [ttyS0] enabled610worker2 # [ 0.067736] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)611worker2 # [ 0.071271] Console: colour VGA+ 80x25612client # [ 0.199286] ACPI: Core revision 20250807613worker1 # [ 0.038038] Unknown kernel command line parameters "regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration", will be passed to user space.614worker2 # [ 0.071274] printk: legacy console [tty0] enabled615worker1 # [ 0.038051] random: crng init done616worker2 # [ 0.100827] printk: legacy console [ttyS0] enabled617worker1 # [ 0.038052] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes618client # [ 0.200231] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns619lb # [ 0.037749] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0620worker2 # [ 0.198699] ACPI: Core revision 20250807621worker1 # [ 0.040211] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)622client # [ 0.201760] APIC: Switch to symmetric I/O mode setup623worker1 # [ 0.041220] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)624worker2 # [ 0.199586] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns625client # [ 0.202739] x2apic enabled626lb # [ 0.037839] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.627worker1 # [ 0.041249] Fallback order for Node 0: 0628lb # [ 0.037852] random: crng init done629worker2 # [ 0.201111] APIC: Switch to symmetric I/O mode setup630client # [ 0.203523] APIC: Switched APIC routing to: physical x2apic631worker1 # [ 0.041252] Built 1 zonelists, mobility grouping on. Total pages: 393078632lb # [ 0.037852] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes633worker1 # [ 0.041253] Policy zone: DMA32634worker2 # [ 0.202095] x2apic enabled635lb # [ 0.040040] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)636worker1 # [ 0.043639] mem auto-init: stack:all(zero), heap alloc:on, heap free:off637worker2 # [ 0.202884] APIC: Switched APIC routing to: physical x2apic638client # [ 0.205314] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1639worker1 # [ 0.046509] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1640lb # [ 0.041054] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)641worker1 # [ 0.049654] allocated 3145728 bytes of page_ext642lb # [ 0.041083] Fallback order for Node 0: 0643worker1 # [ 0.058976] ftrace: allocating 48787 entries in 192 pages644lb # [ 0.041086] Built 1 zonelists, mobility grouping on. Total pages: 393078645worker2 # [ 0.204684] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1646client # [ 0.206282] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns647lb # [ 0.041087] Policy zone: DMA32648worker1 # [ 0.058977] ftrace: allocated 192 pages with 2 groups649worker1 # [ 0.059969] Dynamic Preempt: lazy650lb # [ 0.043437] mem auto-init: stack:all(zero), heap alloc:on, heap free:off651worker1 # [ 0.060134] rcu: Preemptible hierarchical RCU implementation.652client # [ 0.207926] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)653worker2 # [ 0.205649] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns654lb # [ 0.046349] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1655worker1 # [ 0.060135] rcu: RCU event tracing is enabled.656lb # [ 0.049479] allocated 3145728 bytes of page_ext657client # [ 0.209012] x86/cpu: User Mode Instruction Prevention (UMIP) activated658worker1 # [ 0.060136] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.659lb # [ 0.058866] ftrace: allocating 48787 entries in 192 pages660worker2 # [ 0.207292] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)661worker1 # [ 0.060137] Trampoline variant of Tasks RCU enabled.662lb # [ 0.058869] ftrace: allocated 192 pages with 2 groups663client # [ 0.210072] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127664worker1 # [ 0.060137] Rude variant of Tasks RCU enabled.665lb # [ 0.059842] Dynamic Preempt: lazy666worker2 # [ 0.208378] x86/cpu: User Mode Instruction Prevention (UMIP) activated667worker1 # [ 0.060137] Tracing variant of Tasks RCU enabled.668lb # [ 0.060001] rcu: Preemptible hierarchical RCU implementation.669client # [ 0.210923] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0670lb # [ 0.060002] rcu: RCU event tracing is enabled.671worker2 # [ 0.209418] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127672worker1 # [ 0.060138] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.673lb # [ 0.060003] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.674worker1 # [ 0.060139] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1675worker2 # [ 0.210289] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0676client # [ 0.211927] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto677lb # [ 0.060004] Trampoline variant of Tasks RCU enabled.678lb # [ 0.060004] Rude variant of Tasks RCU enabled.679worker1 # [ 0.060158] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.680client # [ 0.213923] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl681lb # [ 0.060005] Tracing variant of Tasks RCU enabled.682worker2 # [ 0.211293] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto683worker1 # [ 0.060160] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.684lb # [ 0.060005] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.685client # [ 0.214923] Transient Scheduler Attacks: Vulnerable: No microcode686worker2 # [ 0.213289] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl687worker1 # [ 0.060161] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.688lb # [ 0.060006] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1689client # [ 0.215923] Spectre V2 : Mitigation: Enhanced / Automatic IBRS690worker1 # [ 0.064469] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16691worker2 # [ 0.214290] Transient Scheduler Attacks: Vulnerable: No microcode692lb # [ 0.060022] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.693client # [ 0.216923] Speculative Return Stack Overflow: Mitigation: Safe RET694worker1 # [ 0.064744] rcu: srcu_init: Setting srcu_struct sizes based on contention.695worker2 # [ 0.215289] Spectre V2 : Mitigation: Enhanced / Automatic IBRS696lb # [ 0.060024] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.697worker1 # [ 0.064751] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns698worker2 # [ 0.216289] Speculative Return Stack Overflow: Mitigation: Safe RET699lb # [ 0.060024] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.700worker1 # [ 0.064850] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)701lb # [ 0.064313] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16702worker1 # [ 0.068343] Console: colour VGA+ 80x25703lb # [ 0.064589] rcu: srcu_init: Setting srcu_struct sizes based on contention.704worker1 # [ 0.068346] printk: legacy console [tty0] enabled705client # [ 0.217923] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization706worker1 # [ 0.099947] printk: legacy console [ttyS0] enabled707lb # [ 0.064595] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns708client # [ 0.219931] Spectre V2 : Enabling IBPB for BPF709worker1 # [ 0.196550] ACPI: Core revision 20250807710worker2 # [ 0.217289] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization711lb # [ 0.064693] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)712client # [ 0.220924] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier713lb # [ 0.068368] Console: colour VGA+ 80x25714worker2 # [ 0.219297] Spectre V2 : Enabling IBPB for BPF715worker1 # [ 0.197412] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns716lb # [ 0.068370] printk: legacy console [tty0] enabled717client # [ 0.221924] active return thunk: srso_alias_return_thunk718lb # [ 0.098765] printk: legacy console [ttyS0] enabled719worker1 # [ 0.198904] APIC: Switch to symmetric I/O mode setup720worker2 # [ 0.220290] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier721lb # [ 0.194176] ACPI: Core revision 20250807722worker1 # [ 0.199887] x2apic enabled723client # [ 0.222944] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'724worker2 # [ 0.221290] active return thunk: srso_alias_return_thunk725client # [ 0.223923] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'726worker1 # [ 0.200664] APIC: Switched APIC routing to: physical x2apic727lb # [ 0.195074] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns728worker2 # [ 0.222312] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'729client # [ 0.224923] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'730lb # [ 0.196575] APIC: Switch to symmetric I/O mode setup731worker2 # [ 0.223289] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'732worker1 # [ 0.202523] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1733client # [ 0.225923] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'734lb # [ 0.197552] x2apic enabled735worker2 # [ 0.225026] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'736client # [ 0.226923] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'737lb # [ 0.198339] APIC: Switched APIC routing to: physical x2apic738worker2 # [ 0.226029] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'739worker1 # [ 0.203502] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns740client # [ 0.227923] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'741worker2 # [ 0.227058] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'742lb # [ 0.200145] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1743worker1 # [ 0.205197] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)744client # [ 0.228923] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'745worker2 # [ 0.228056] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'746client # [ 0.230808] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256747worker1 # [ 0.206280] x86/cpu: User Mode Instruction Prevention (UMIP) activated748lb # [ 0.201105] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns749worker2 # [ 0.229072] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'750client # [ 0.231625] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64751worker1 # [ 0.208108] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127752worker2 # [ 0.230290] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256753client # [ 0.232623] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512754lb # [ 0.202738] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)755worker1 # [ 0.208882] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0756worker2 # [ 0.231289] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64757client # [ 0.233627] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024758lb # [ 0.203822] x86/cpu: User Mode Instruction Prevention (UMIP) activated759worker2 # [ 0.232231] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512760worker1 # [ 0.209947] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto761client # [ 0.233923] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8762lb # [ 0.204861] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127763worker2 # [ 0.232992] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024764lb # [ 0.205736] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0765worker1 # [ 0.211194] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl766client # [ 0.234923] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.767worker2 # [ 0.233998] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8768worker1 # [ 0.212195] Transient Scheduler Attacks: Vulnerable: No microcode769lb # [ 0.206739] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto770worker2 # [ 0.235013] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.771worker1 # [ 0.213926] Spectre V2 : Mitigation: Enhanced / Automatic IBRS772worker1 # [ 0.214891] Speculative Return Stack Overflow: Mitigation: Safe RET773lb # [ 0.208736] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl774lb # [ 0.209736] Transient Scheduler Attacks: Vulnerable: No microcode775lb # [ 0.210735] Spectre V2 : Mitigation: Enhanced / Automatic IBRS776lb # [ 0.212492] Speculative Return Stack Overflow: Mitigation: Safe RET777worker1 # [ 0.215194] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization778worker1 # [ 0.217201] Spectre V2 : Enabling IBPB for BPF779worker1 # [ 0.218752] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier780worker1 # [ 0.219195] active return thunk: srso_alias_return_thunk781lb # [ 0.213523] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization782lb # [ 0.214743] Spectre V2 : Enabling IBPB for BPF783worker1 # [ 0.221097] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'784lb # [ 0.216292] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier785worker1 # [ 0.222194] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'786lb # [ 0.216736] active return thunk: srso_alias_return_thunk787worker1 # [ 0.223194] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'788lb # [ 0.218617] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'789worker1 # [ 0.224194] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'790lb # [ 0.219735] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'791worker1 # [ 0.225193] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'792lb # [ 0.220735] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'793worker1 # [ 0.226194] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'794lb # [ 0.221736] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'795worker1 # [ 0.227194] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'796lb # [ 0.222735] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'797worker1 # [ 0.228195] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256798lb # [ 0.223735] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'799worker1 # [ 0.229948] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64800worker1 # [ 0.230954] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512801lb # [ 0.224735] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'802worker1 # [ 0.231948] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024803lb # [ 0.225735] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256804worker1 # [ 0.232955] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8805lb # [ 0.226735] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64806lb # [ 0.227735] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512807worker1 # [ 0.233949] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.808lb # [ 0.228735] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809lb # [ 0.229735] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810lb # [ 0.230735] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.269760] Freeing SMP alternatives memory: 44K812client # [ 0.270501] pid_max: default: 32768 minimum: 301813worker2 # [ 0.269444] Freeing SMP alternatives memory: 44K814client # [ 0.271032] LSM: initializing lsm=capability,landlock,yama,bpf,ima815client # [ 0.272025] landlock: Up and running.816worker2 # [ 0.270243] pid_max: default: 32768 minimum: 301817client # [ 0.272924] Yama: becoming mindful.818worker2 # [ 0.270945] LSM: initializing lsm=capability,landlock,yama,bpf,ima819client # [ 0.274015] LSM support for eBPF active820worker2 # [ 0.272186] landlock: Up and running.821worker2 # [ 0.272755] Yama: becoming mindful.822client # [ 0.274775] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)823worker2 # [ 0.273503] LSM support for eBPF active824client # [ 0.275965] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)825worker2 # [ 0.274411] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)826worker2 # [ 0.275331] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)827client # [ 0.278337] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)828client # [ 0.279511] Performance Events: Fam17h+ core perfctr, AMD PMU driver.829client # [ 0.279932] ... version: 2830worker2 # [ 0.278526] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)831client # [ 0.280655] ... bit width: 48832client # [ 0.280926] ... generic counters: 6833worker2 # [ 0.279947] Performance Events: Fam17h+ core perfctr, AMD PMU driver.834client # [ 0.281633] ... generic bitmap: 000000000000003f835worker2 # [ 0.280298] ... version: 2836client # [ 0.281925] ... fixed-purpose counters: 0837worker2 # [ 0.281039] ... bit width: 48838client # [ 0.282653] ... fixed-purpose bitmap: 0000000000000000839worker2 # [ 0.281292] ... generic counters: 6840client # [ 0.282925] ... value mask: 0000ffffffffffff841worker2 # [ 0.282054] ... generic bitmap: 000000000000003f842worker2 # [ 0.282292] ... fixed-purpose counters: 0843client # [ 0.283836] ... max period: 00007fffffffffff844worker2 # [ 0.283038] ... fixed-purpose bitmap: 0000000000000000845client # [ 0.284637] ... global_ctrl mask: 000000000000003f846worker2 # [ 0.283292] ... value mask: 0000ffffffffffff847client # [ 0.285037] signal: max sigframe size: 3376848client # [ 0.285826] rcu: Hierarchical SRCU implementation.849worker2 # [ 0.284237] ... max period: 00007fffffffffff850client # [ 0.286530] rcu: Max phase no-delay instances is 400.851worker2 # [ 0.285027] ... global_ctrl mask: 000000000000003f852worker2 # [ 0.285405] signal: max sigframe size: 3376853worker2 # [ 0.286254] rcu: Hierarchical SRCU implementation.854worker2 # [ 0.286916] rcu: Max phase no-delay instances is 400.855worker1 # [ 0.268176] Freeing SMP alternatives memory: 44K856worker1 # [ 0.268785] pid_max: default: 32768 minimum: 301857worker1 # [ 0.269897] LSM: initializing lsm=capability,landlock,yama,bpf,ima858client # [ 0.291434] smp: Bringing up secondary CPUs ...859client # [ 0.291940] smp: Brought up 1 node, 1 CPU860worker1 # [ 0.270276] landlock: Up and running.861lb # [ 0.265352] Freeing SMP alternatives memory: 44K862worker1 # [ 0.271618] Yama: becoming mindful.863lb # [ 0.265738] pid_max: default: 32768 minimum: 301864client # [ 0.292587] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)865worker1 # [ 0.272405] LSM support for eBPF active866lb # [ 0.266831] LSM: initializing lsm=capability,landlock,yama,bpf,ima867worker2 # [ 0.291836] smp: Bringing up secondary CPUs ...868lb # [ 0.267833] landlock: Up and running.869worker2 # [ 0.292308] smp: Brought up 1 node, 1 CPU870worker1 # [ 0.273298] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)871client # [ 0.293070] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)872lb # [ 0.268423] Yama: becoming mindful.873worker2 # [ 0.292943] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)874client # [ 0.294188] devtmpfs: initialized875lb # [ 0.269353] LSM support for eBPF active876worker1 # [ 0.274231] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)877client # [ 0.294936] x86/mm: Memory block size: 128MB878lb # [ 0.269848] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)879worker2 # [ 0.293444] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)880worker1 # [ 0.276817] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)881lb # [ 0.270772] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)882worker2 # [ 0.294556] devtmpfs: initialized883client # [ 0.296685] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)884worker2 # [ 0.295386] x86/mm: Memory block size: 128MB885worker1 # [ 0.277697] Performance Events: Fam17h+ core perfctr, AMD PMU driver.886client # [ 0.297811] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).887worker1 # [ 0.278200] ... version: 2888lb # [ 0.273671] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)889client # [ 0.299005] pinctrl core: initialized pinctrl subsystem890worker1 # [ 0.278921] ... bit width: 48891worker2 # [ 0.297208] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)892worker1 # [ 0.279197] ... generic counters: 6893lb # [ 0.274222] Performance Events: Fam17h+ core perfctr, AMD PMU driver.894client # [ 0.300191] PM: RTC time: 16:12:08, date: 2026-09-23895lb # [ 0.274740] ... version: 2896worker1 # [ 0.279881] ... generic bitmap: 000000000000003f897worker2 # [ 0.298198] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).898lb # [ 0.275417] ... bit width: 48899worker1 # [ 0.280246] ... fixed-purpose counters: 0900worker2 # [ 0.299371] pinctrl core: initialized pinctrl subsystem901lb # [ 0.275738] ... generic counters: 6902worker1 # [ 0.280990] ... fixed-purpose bitmap: 0000000000000000903lb # [ 0.276444] ... generic bitmap: 000000000000003f904worker2 # [ 0.300485] PM: RTC time: 16:12:08, date: 2026-09-23905worker1 # [ 0.281893] ... value mask: 0000ffffffffffff906lb # [ 0.276781] ... fixed-purpose counters: 0907client # [ 0.303829] NET: Registered PF_NETLINK/PF_ROUTE protocol family908worker1 # [ 0.282197] ... max period: 00007fffffffffff909lb # [ 0.277468] ... fixed-purpose bitmap: 0000000000000000910worker1 # [ 0.283089] ... global_ctrl mask: 000000000000003f911lb # [ 0.277738] ... value mask: 0000ffffffffffff912client # [ 0.305158] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations913worker1 # [ 0.284033] signal: max sigframe size: 3376914lb # [ 0.278590] ... max period: 00007fffffffffff915worker2 # [ 0.304045] NET: Registered PF_NETLINK/PF_ROUTE protocol family916worker1 # [ 0.284830] rcu: Hierarchical SRCU implementation.917client # [ 0.305947] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations918lb # [ 0.279387] ... global_ctrl mask: 000000000000003f919worker1 # [ 0.285200] rcu: Max phase no-delay instances is 400.920lb # [ 0.279844] signal: max sigframe size: 3376921worker2 # [ 0.304788] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations922client # [ 0.307197] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations923lb # [ 0.280574] rcu: Hierarchical SRCU implementation.924client # [ 0.307936] audit: initializing netlink subsys (disabled)925lb # [ 0.281307] rcu: Max phase no-delay instances is 400.926worker2 # [ 0.305317] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations927client # [ 0.309057] thermal_sys: Registered thermal governor 'fair_share'928worker2 # [ 0.306571] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations929client # [ 0.309059] thermal_sys: Registered thermal governor 'bang_bang'930worker2 # [ 0.307302] audit: initializing netlink subsys (disabled)931client # [ 0.309927] thermal_sys: Registered thermal governor 'step_wise'932worker1 # [ 0.290504] smp: Bringing up secondary CPUs ...933worker2 # [ 0.308526] thermal_sys: Registered thermal governor 'fair_share'934client # [ 0.310890] thermal_sys: Registered thermal governor 'user_space'935worker1 # [ 0.291211] smp: Brought up 1 node, 1 CPU936worker2 # [ 0.308528] thermal_sys: Registered thermal governor 'bang_bang'937worker1 # [ 0.291867] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)938lb # [ 0.286185] smp: Bringing up secondary CPUs ...939client # [ 0.311673] audit: type=2000 audit(1790179929.076:1): state=initialized audit_enabled=0 res=1940lb # [ 0.286751] smp: Brought up 1 node, 1 CPU941worker2 # [ 0.309294] thermal_sys: Registered thermal governor 'step_wise'942client # [ 0.312928] thermal_sys: Registered thermal governor 'power_allocator'943lb # [ 0.287407] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)944worker2 # [ 0.310258] thermal_sys: Registered thermal governor 'user_space'945client # [ 0.312941] cpuidle: using governor menu946worker1 # [ 0.292323] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)947worker1 # [ 0.293398] devtmpfs: initialized948worker2 # [ 0.311015] audit: type=2000 audit(1790179929.078:1): state=initialized audit_enabled=0 res=1949client # [ 0.315134] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5950lb # [ 0.287932] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)951worker1 # [ 0.294222] x86/mm: Memory block size: 128MB952worker2 # [ 0.312294] thermal_sys: Registered thermal governor 'power_allocator'953lb # [ 0.288970] devtmpfs: initialized954worker2 # [ 0.312308] cpuidle: using governor menu955client # [ 0.316153] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]956lb # [ 0.289780] x86/mm: Memory block size: 128MB957worker1 # [ 0.295980] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)958client # [ 0.316929] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry959worker2 # [ 0.314517] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5960client # [ 0.317938] PCI: Using configuration type 1 for base access961worker1 # [ 0.297092] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).962lb # [ 0.291525] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)963worker2 # [ 0.315516] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]964worker1 # [ 0.298270] pinctrl core: initialized pinctrl subsystem965client # [ 0.319077] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.966worker2 # [ 0.316295] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry967lb # [ 0.292611] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).968worker1 # [ 0.299416] PM: RTC time: 16:12:08, date: 2026-09-23969worker2 # [ 0.317303] PCI: Using configuration type 1 for base access970lb # [ 0.293804] pinctrl core: initialized pinctrl subsystem971lb # [ 0.294941] PM: RTC time: 16:12:08, date: 2026-09-23972worker2 # [ 0.318469] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.973worker1 # [ 0.302821] NET: Registered PF_NETLINK/PF_ROUTE protocol family974client # [ 0.324044] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages975lb # [ 0.298368] NET: Registered PF_NETLINK/PF_ROUTE protocol family976worker1 # [ 0.303671] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations977client # [ 0.324927] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page978client # [ 0.325919] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages979worker1 # [ 0.304217] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations980lb # [ 0.299226] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations981worker2 # [ 0.323517] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages982lb # [ 0.299757] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations983worker1 # [ 0.305467] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations984worker2 # [ 0.324293] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page985worker1 # [ 0.306206] audit: initializing netlink subsys (disabled)986lb # [ 0.301004] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations987lb # [ 0.301747] audit: initializing netlink subsys (disabled)988worker1 # [ 0.307198] audit: type=2000 audit(1790179929.099:1): state=initialized audit_enabled=0 res=1989worker1 # [ 0.308364] thermal_sys: Registered thermal governor 'fair_share'990lb # [ 0.302740] audit: type=2000 audit(1790179929.103:1): state=initialized audit_enabled=0 res=1991worker1 # [ 0.308366] thermal_sys: Registered thermal governor 'bang_bang'992lb # [ 0.303832] thermal_sys: Registered thermal governor 'fair_share'993client # [ 0.332928] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page994worker1 # [ 0.309198] thermal_sys: Registered thermal governor 'step_wise'995worker2 # [ 0.329293] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages996lb # [ 0.303834] thermal_sys: Registered thermal governor 'bang_bang'997worker1 # [ 0.310177] thermal_sys: Registered thermal governor 'user_space'998worker2 # [ 0.330292] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page999lb # [ 0.304738] thermal_sys: Registered thermal governor 'step_wise'1000worker1 # [ 0.310915] thermal_sys: Registered thermal governor 'power_allocator'1001lb # [ 0.305634] thermal_sys: Registered thermal governor 'user_space'1002worker1 # [ 0.311211] cpuidle: using governor menu1003lb # [ 0.306434] thermal_sys: Registered thermal governor 'power_allocator'1004lb # [ 0.306751] cpuidle: using governor menu1005worker1 # [ 0.313443] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51006lb # [ 0.308949] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51007worker1 # [ 0.314433] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1008lb # [ 0.309964] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1009worker1 # [ 0.315199] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1010client # [ 0.341352] ACPI: Added _OSI(Module Device)1011worker1 # [ 0.316208] PCI: Using configuration type 1 for base access1012lb # [ 0.310740] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1013client # [ 0.341928] ACPI: Added _OSI(Processor Device)1014lb # [ 0.311750] PCI: Using configuration type 1 for base access1015client # [ 0.342627] ACPI: Added _OSI(Processor Aggregator Device)1016worker1 # [ 0.317332] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1017lb # [ 0.312834] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1018worker2 # [ 0.341632] ACPI: Added _OSI(Module Device)1019worker2 # [ 0.342294] ACPI: Added _OSI(Processor Device)1020worker2 # [ 0.343050] ACPI: Added _OSI(Processor Aggregator Device)1021worker1 # [ 0.324219] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1022lb # [ 0.319760] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1023worker1 # [ 0.325197] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1024lb # [ 0.320738] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1025client # [ 0.351469] ACPI: 1 ACPI AML tables successfully acquired and loaded1026worker2 # [ 0.350886] ACPI: 1 ACPI AML tables successfully acquired and loaded1027worker1 # [ 0.330198] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1028client # [ 0.355384] ACPI: Interpreter enabled1029lb # [ 0.325739] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1030worker1 # [ 0.331197] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1031client # [ 0.355941] ACPI: PM: (supports S0 S3 S4 S5)1032lb # [ 0.326738] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1033client # [ 0.356670] ACPI: Using IOAPIC for interrupt routing1034client # [ 0.356986] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1035worker2 # [ 0.354543] ACPI: Interpreter enabled1036worker2 # [ 0.355244] ACPI: PM: (supports S0 S3 S4 S5)1037client # [ 0.357927] PCI: Using E820 reservations for host bridge windows1038worker2 # [ 0.355840] ACPI: Using IOAPIC for interrupt routing1039worker2 # [ 0.356337] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1040client # [ 0.361064] ACPI: Enabled 2 GPEs in block 00 to 3F1041worker2 # [ 0.357293] PCI: Using E820 reservations for host bridge windows1042worker2 # [ 0.360430] ACPI: Enabled 2 GPEs in block 00 to 3F1043worker1 # [ 0.342421] ACPI: Added _OSI(Module Device)1044lb # [ 0.337957] ACPI: Added _OSI(Module Device)1045worker1 # [ 0.343198] ACPI: Added _OSI(Processor Device)1046lb # [ 0.338662] ACPI: Added _OSI(Processor Device)1047worker1 # [ 0.343920] ACPI: Added _OSI(Processor Aggregator Device)1048lb # [ 0.339266] ACPI: Added _OSI(Processor Aggregator Device)1049client # [ 0.369740] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1050client # [ 0.370748] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1051client # [ 0.372299] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1052worker2 # [ 0.369144] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1053worker1 # [ 0.349576] ACPI: 1 ACPI AML tables successfully acquired and loaded1054lb # [ 0.344968] ACPI: 1 ACPI AML tables successfully acquired and loaded1055client # [ 0.373049] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1056worker2 # [ 0.370098] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1057client # [ 0.374264] PCI host bridge to bus 0000:001058client # [ 0.374930] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1059worker2 # [ 0.371820] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1060client # [ 0.375926] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1061worker1 # [ 0.353599] ACPI: Interpreter enabled1062worker2 # [ 0.372415] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1063lb # [ 0.349139] ACPI: Interpreter enabled1064worker1 # [ 0.354213] ACPI: PM: (supports S0 S3 S4 S5)1065worker2 # [ 0.373715] PCI host bridge to bus 0000:001066lb # [ 0.349752] ACPI: PM: (supports S0 S3 S4 S5)1067client # [ 0.376926] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1068worker1 # [ 0.354891] ACPI: Using IOAPIC for interrupt routing1069lb # [ 0.350421] ACPI: Using IOAPIC for interrupt routing1070worker2 # [ 0.374297] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1071client # [ 0.377927] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1072worker2 # [ 0.375292] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1073client # [ 0.378926] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1074worker2 # [ 0.376292] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1075client # [ 0.379931] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1076worker1 # [ 0.357249] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1077lb # [ 0.352804] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1078client # [ 0.380955] pci_bus 0000:00: root bus resource [bus 00-ff]1079worker2 # [ 0.377292] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1080worker1 # [ 0.358197] PCI: Using E820 reservations for host bridge windows1081lb # [ 0.353738] PCI: Using E820 reservations for host bridge windows1082worker2 # [ 0.378292] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1083client # [ 0.381884] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1084worker2 # [ 0.379293] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1085worker1 # [ 0.361265] ACPI: Enabled 2 GPEs in block 00 to 3F1086client # [ 0.383374] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1087worker2 # [ 0.380292] pci_bus 0000:00: root bus resource [bus 00-ff]1088lb # [ 0.356795] ACPI: Enabled 2 GPEs in block 00 to 3F1089worker2 # [ 0.381322] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1090worker2 # [ 0.382729] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1091worker1 # [ 0.368648] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1092lb # [ 0.363975] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1093client # [ 0.386070] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1094worker1 # [ 0.369203] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1095lb # [ 0.364744] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1096client # [ 0.386938] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1097client # [ 0.387881] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1098lb # [ 0.365814] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1099worker1 # [ 0.370277] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1100lb # [ 0.366859] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1101worker1 # [ 0.371320] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1102client # [ 0.388847] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1103worker2 # [ 0.385448] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1104lb # [ 0.368096] PCI host bridge to bus 0000:001105worker1 # [ 0.372534] PCI host bridge to bus 0000:001106worker2 # [ 0.386304] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1107lb # [ 0.368742] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1108worker1 # [ 0.373201] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1109client # [ 0.390666] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110worker2 # [ 0.387381] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1111lb # [ 0.369738] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1112worker1 # [ 0.374197] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1113lb # [ 0.370738] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1114worker1 # [ 0.375197] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1115worker2 # [ 0.388655] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1116lb # [ 0.371738] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1117client # [ 0.392964] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1118worker1 # [ 0.376197] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1119client # [ 0.393832] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1120lb # [ 0.372738] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1121worker2 # [ 0.391014] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1122worker1 # [ 0.377197] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1123client # [ 0.394648] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1124lb # [ 0.373738] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1125worker1 # [ 0.378197] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1126client # [ 0.394932] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1127lb # [ 0.374739] pci_bus 0000:00: root bus resource [bus 00-ff]1128worker1 # [ 0.379197] pci_bus 0000:00: root bus resource [bus 00-ff]1129worker2 # [ 0.394331] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1130lb # [ 0.375724] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1131worker1 # [ 0.380181] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1132client # [ 0.396481] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133worker2 # [ 0.395199] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1134lb # [ 0.377107] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1135worker2 # [ 0.396066] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1136worker1 # [ 0.381624] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1137worker2 # [ 0.397131] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1138client # [ 0.397944] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1139client # [ 0.398791] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1140worker2 # [ 0.399140] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1141client # [ 0.399719] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1142client # [ 0.401343] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1143worker2 # [ 0.402176] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1144lb # [ 0.379777] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1145worker2 # [ 0.403146] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1146worker1 # [ 0.384293] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1147lb # [ 0.380751] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1148worker2 # [ 0.404039] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1149worker1 # [ 0.385288] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1150lb # [ 0.381685] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1151client # [ 0.403976] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1152worker1 # [ 0.386266] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1153worker2 # [ 0.404891] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1154client # [ 0.405549] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1155lb # [ 0.382642] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1156worker1 # [ 0.387504] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1157client # [ 0.405948] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1158lb # [ 0.384627] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1159client # [ 0.407504] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1160worker1 # [ 0.388926] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1161worker2 # [ 0.407300] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1162worker2 # [ 0.409012] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1163lb # [ 0.387745] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1164client # [ 0.408958] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1165worker1 # [ 0.391237] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1166lb # [ 0.388554] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1167worker2 # [ 0.409314] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1168client # [ 0.409851] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1169worker1 # [ 0.392098] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1170lb # [ 0.389431] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1171client # [ 0.410660] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1172worker1 # [ 0.392942] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1173worker2 # [ 0.410857] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1174lb # [ 0.389744] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1175worker1 # [ 0.393203] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1176client # [ 0.410932] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1177lb # [ 0.391207] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1178worker1 # [ 0.394748] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1179worker2 # [ 0.413271] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1180client # [ 0.412480] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1181worker2 # [ 0.413946] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1182worker2 # [ 0.414321] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1183worker1 # [ 0.396204] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1184client # [ 0.413960] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1185lb # [ 0.393746] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1186worker1 # [ 0.397165] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1187client # [ 0.414948] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1188worker2 # [ 0.415298] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1189lb # [ 0.394553] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1190worker1 # [ 0.397905] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1191lb # [ 0.395427] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1192client # [ 0.416502] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1193worker2 # [ 0.416867] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1194worker1 # [ 0.398763] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1195lb # [ 0.396288] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196client # [ 0.417969] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1197worker2 # [ 0.418337] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1198worker2 # [ 0.419314] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1199client # [ 0.418948] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1200lb # [ 0.398746] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1201worker1 # [ 0.401246] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1202lb # [ 0.399570] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1203worker1 # [ 0.402204] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1204worker2 # [ 0.420842] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205client # [ 0.420504] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1206worker1 # [ 0.403143] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1207lb # [ 0.400445] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1208client # [ 0.421969] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1209worker2 # [ 0.422304] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1210worker1 # [ 0.404690] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1211client # [ 0.422948] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212lb # [ 0.401282] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1213worker2 # [ 0.423315] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1214worker1 # [ 0.406233] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1215client # [ 0.424464] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker2 # [ 0.424867] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1217lb # [ 0.403578] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1218worker1 # [ 0.407074] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1219lb # [ 0.404341] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1220worker1 # [ 0.407898] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1221client # [ 0.425971] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1222lb # [ 0.404760] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1223worker2 # [ 0.426305] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1224worker1 # [ 0.408203] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1225lb # [ 0.405744] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1226client # [ 0.426941] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1227worker2 # [ 0.427265] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1228worker1 # [ 0.409942] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229lb # [ 0.407233] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1230client # [ 0.428461] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1231worker2 # [ 0.428694] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1232worker1 # [ 0.412008] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1233worker1 # [ 0.412976] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1234client # [ 0.429977] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1235worker2 # [ 0.430305] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1236lb # [ 0.409750] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1237client # [ 0.430859] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1238worker1 # [ 0.414709] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239lb # [ 0.410690] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1240worker2 # [ 0.431314] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1241client # [ 0.431659] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1242worker2 # [ 0.432860] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1243lb # [ 0.412094] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1244worker1 # [ 0.416240] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1245client # [ 0.432498] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1246worker1 # [ 0.417151] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1247lb # [ 0.413750] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1248worker2 # [ 0.435121] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1249lb # [ 0.414743] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1250worker1 # [ 0.418797] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1251worker2 # [ 0.435963] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1252client # [ 0.433934] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1253client # [ 0.435567] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1254worker2 # [ 0.436313] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1255lb # [ 0.416308] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1256worker1 # [ 0.420062] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1257client # [ 0.435949] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1258worker1 # [ 0.420910] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1259worker2 # [ 0.437827] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1260client # [ 0.437518] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1261worker1 # [ 0.421743] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1262lb # [ 0.418750] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1263lb # [ 0.419678] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1264client # [ 0.438664] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1265worker2 # [ 0.439988] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1266worker1 # [ 0.423182] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1267client # [ 0.439123] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1268worker2 # [ 0.440299] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1269lb # [ 0.421141] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1270worker1 # [ 0.423950] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1271worker2 # [ 0.441237] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1272client # [ 0.440600] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1273lb # [ 0.422750] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1274worker1 # [ 0.425536] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1275worker2 # [ 0.442775] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1276lb # [ 0.423701] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1277client # [ 0.441132] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1278worker1 # [ 0.427249] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1279worker2 # [ 0.444045] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1280lb # [ 0.425113] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1281client # [ 0.442647] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1282worker1 # [ 0.428123] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1283worker2 # [ 0.444500] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1284worker1 # [ 0.428952] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1285worker1 # [ 0.429749] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1286lb # [ 0.427745] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1287worker2 # [ 0.446709] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1288lb # [ 0.428590] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1289worker2 # [ 0.447486] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1290lb # [ 0.429493] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1291worker1 # [ 0.431233] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1292worker2 # [ 0.449032] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1293lb # [ 0.430278] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1294worker1 # [ 0.432203] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1295worker1 # [ 0.433158] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1296lb # [ 0.431747] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1297worker1 # [ 0.434668] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1298lb # [ 0.432612] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1299worker1 # [ 0.435764] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1300client # [ 0.443151] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1301lb # [ 0.433486] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1302worker1 # [ 0.436424] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1303client # [ 0.445568] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1304lb # [ 0.434301] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1305worker1 # [ 0.437755] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1306client # [ 0.446212] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1307lb # [ 0.435555] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1308worker1 # [ 0.438398] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1309client # [ 0.447217] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1310lb # [ 0.436555] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1311worker1 # [ 0.439711] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1312client # [ 0.448212] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1313lb # [ 0.438446] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1314worker2 # [ 0.449513] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1315lb # [ 0.438930] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1316client # [ 0.449954] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1317worker2 # [ 0.452051] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1318client # [ 0.450847] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1319lb # [ 0.440453] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1320worker2 # [ 0.452570] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1321client # [ 0.452070] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1322worker2 # [ 0.453580] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1323client # [ 0.453570] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1324worker2 # [ 0.454545] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1325worker2 # [ 0.456320] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1326worker2 # [ 0.457253] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1327client # [ 0.457057] ACPI: PCI: Interrupt link LNKA configured for IRQ 101328client # [ 0.458033] ACPI: PCI: Interrupt link LNKB configured for IRQ 101329worker1 # [ 0.440460] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1330worker2 # [ 0.458348] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1331client # [ 0.459023] ACPI: PCI: Interrupt link LNKC configured for IRQ 111332worker1 # [ 0.442758] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1333client # [ 0.460034] ACPI: PCI: Interrupt link LNKD configured for IRQ 111334worker2 # [ 0.460007] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1335client # [ 0.461024] ACPI: PCI: Interrupt link LNKE configured for IRQ 101336worker1 # [ 0.443448] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1337client # [ 0.462021] ACPI: PCI: Interrupt link LNKF configured for IRQ 101338worker1 # [ 0.444517] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1339lb # [ 0.441703] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1340client # [ 0.463023] ACPI: PCI: Interrupt link LNKG configured for IRQ 111341client # [ 0.464043] ACPI: PCI: Interrupt link LNKH configured for IRQ 111342worker1 # [ 0.445446] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1343worker2 # [ 0.463400] ACPI: PCI: Interrupt link LNKA configured for IRQ 101344client # [ 0.464991] ACPI: PCI: Interrupt link GSIA configured for IRQ 161345worker2 # [ 0.464430] ACPI: PCI: Interrupt link LNKB configured for IRQ 101346worker1 # [ 0.447001] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1347lb # [ 0.444385] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1348client # [ 0.465941] ACPI: PCI: Interrupt link GSIB configured for IRQ 171349worker2 # [ 0.465399] ACPI: PCI: Interrupt link LNKC configured for IRQ 111350worker1 # [ 0.447813] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1351client # [ 0.466941] ACPI: PCI: Interrupt link GSIC configured for IRQ 181352lb # [ 0.445021] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1353worker2 # [ 0.466398] ACPI: PCI: Interrupt link LNKD configured for IRQ 111354client # [ 0.467942] ACPI: PCI: Interrupt link GSID configured for IRQ 191355worker1 # [ 0.448506] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1356worker2 # [ 0.467389] ACPI: PCI: Interrupt link LNKE configured for IRQ 101357lb # [ 0.446025] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1358client # [ 0.468914] ACPI: PCI: Interrupt link GSIE configured for IRQ 201359worker2 # [ 0.468389] ACPI: PCI: Interrupt link LNKF configured for IRQ 101360worker1 # [ 0.449810] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1361client # [ 0.469663] ACPI: PCI: Interrupt link GSIF configured for IRQ 211362lb # [ 0.446983] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1363worker2 # [ 0.469393] ACPI: PCI: Interrupt link LNKG configured for IRQ 111364client # [ 0.469941] ACPI: PCI: Interrupt link GSIG configured for IRQ 221365client # [ 0.470910] ACPI: PCI: Interrupt link GSIH configured for IRQ 231366worker2 # [ 0.470401] ACPI: PCI: Interrupt link LNKH configured for IRQ 111367worker2 # [ 0.471337] ACPI: PCI: Interrupt link GSIA configured for IRQ 161368lb # [ 0.448767] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1369worker1 # [ 0.453268] ACPI: PCI: Interrupt link LNKA configured for IRQ 101370client # [ 0.472846] iommu: Default domain type: Translated1371worker2 # [ 0.472308] ACPI: PCI: Interrupt link GSIB configured for IRQ 171372lb # [ 0.449579] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1373client # [ 0.473531] iommu: DMA domain TLB invalidation policy: lazy mode1374worker1 # [ 0.454330] ACPI: PCI: Interrupt link LNKB configured for IRQ 101375worker2 # [ 0.473222] ACPI: PCI: Interrupt link GSIC configured for IRQ 181376client # [ 0.474188] ACPI: bus type USB registered1377worker2 # [ 0.474013] ACPI: PCI: Interrupt link GSID configured for IRQ 191378worker1 # [ 0.455298] ACPI: PCI: Interrupt link LNKC configured for IRQ 111379lb # [ 0.451131] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1380client # [ 0.474944] usbcore: registered new interface driver usbfs1381worker2 # [ 0.474304] ACPI: PCI: Interrupt link GSIE configured for IRQ 201382worker1 # [ 0.456294] ACPI: PCI: Interrupt link LNKD configured for IRQ 111383client # [ 0.475842] usbcore: registered new interface driver hub1384lb # [ 0.452556] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1385worker2 # [ 0.475273] ACPI: PCI: Interrupt link GSIF configured for IRQ 211386worker1 # [ 0.457296] ACPI: PCI: Interrupt link LNKE configured for IRQ 101387client # [ 0.476556] usbcore: registered new device driver usb1388worker2 # [ 0.476042] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389worker1 # [ 0.458296] ACPI: PCI: Interrupt link LNKF configured for IRQ 101390client # [ 0.477766] NetLabel: Initializing1391worker2 # [ 0.476304] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392worker1 # [ 0.459297] ACPI: PCI: Interrupt link LNKG configured for IRQ 111393client # [ 0.478367] NetLabel: domain hash size = 1281394worker1 # [ 0.460294] ACPI: PCI: Interrupt link LNKH configured for IRQ 111395client # [ 0.478926] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1396worker2 # [ 0.478340] iommu: Default domain type: Translated1397lb # [ 0.456297] ACPI: PCI: Interrupt link LNKA configured for IRQ 101398worker1 # [ 0.461232] ACPI: PCI: Interrupt link GSIA configured for IRQ 161399client # [ 0.479842] NetLabel: unlabeled traffic allowed by default1400worker2 # [ 0.479141] iommu: DMA domain TLB invalidation policy: lazy mode1401lb # [ 0.456845] ACPI: PCI: Interrupt link LNKB configured for IRQ 101402client # [ 0.480569] PCI: Using ACPI for IRQ routing1403worker1 # [ 0.462185] ACPI: PCI: Interrupt link GSIB configured for IRQ 171404worker2 # [ 0.480280] ACPI: bus type USB registered1405lb # [ 0.457836] ACPI: PCI: Interrupt link LNKC configured for IRQ 111406worker1 # [ 0.463214] ACPI: PCI: Interrupt link GSIC configured for IRQ 181407worker2 # [ 0.480894] usbcore: registered new interface driver usbfs1408lb # [ 0.458833] ACPI: PCI: Interrupt link LNKD configured for IRQ 111409worker2 # [ 0.481307] usbcore: registered new interface driver hub1410worker1 # [ 0.464213] ACPI: PCI: Interrupt link GSID configured for IRQ 191411lb # [ 0.459833] ACPI: PCI: Interrupt link LNKE configured for IRQ 101412worker2 # [ 0.482206] usbcore: registered new device driver usb1413worker1 # [ 0.465192] ACPI: PCI: Interrupt link GSIE configured for IRQ 201414lb # [ 0.460837] ACPI: PCI: Interrupt link LNKF configured for IRQ 101415worker1 # [ 0.466157] ACPI: PCI: Interrupt link GSIF configured for IRQ 211416worker2 # [ 0.483865] NetLabel: Initializing1417lb # [ 0.461837] ACPI: PCI: Interrupt link LNKG configured for IRQ 111418worker1 # [ 0.466997] ACPI: PCI: Interrupt link GSIG configured for IRQ 221419worker2 # [ 0.484293] NetLabel: domain hash size = 1281420lb # [ 0.462836] ACPI: PCI: Interrupt link LNKH configured for IRQ 111421worker1 # [ 0.467951] ACPI: PCI: Interrupt link GSIH configured for IRQ 231422worker2 # [ 0.485055] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1423lb # [ 0.463774] ACPI: PCI: Interrupt link GSIA configured for IRQ 161424worker2 # [ 0.485324] NetLabel: unlabeled traffic allowed by default1425lb # [ 0.464741] ACPI: PCI: Interrupt link GSIB configured for IRQ 171426worker2 # [ 0.486251] PCI: Using ACPI for IRQ routing1427worker1 # [ 0.469176] iommu: Default domain type: Translated1428lb # [ 0.465681] ACPI: PCI: Interrupt link GSIC configured for IRQ 181429worker1 # [ 0.469795] iommu: DMA domain TLB invalidation policy: lazy mode1430lb # [ 0.466474] ACPI: PCI: Interrupt link GSID configured for IRQ 191431worker1 # [ 0.470462] ACPI: bus type USB registered1432lb # [ 0.466749] ACPI: PCI: Interrupt link GSIE configured for IRQ 201433worker1 # [ 0.471255] usbcore: registered new interface driver usbfs1434lb # [ 0.467744] ACPI: PCI: Interrupt link GSIF configured for IRQ 211435worker1 # [ 0.472188] usbcore: registered new interface driver hub1436worker1 # [ 0.472886] usbcore: registered new device driver usb1437lb # [ 0.468715] ACPI: PCI: Interrupt link GSIG configured for IRQ 221438lb # [ 0.469680] ACPI: PCI: Interrupt link GSIH configured for IRQ 231439worker1 # [ 0.473997] NetLabel: Initializing1440worker1 # [ 0.474203] NetLabel: domain hash size = 1281441lb # [ 0.471507] iommu: Default domain type: Translated1442worker1 # [ 0.474979] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1443lb # [ 0.471745] iommu: DMA domain TLB invalidation policy: lazy mode1444worker1 # [ 0.475959] NetLabel: unlabeled traffic allowed by default1445lb # [ 0.472980] ACPI: bus type USB registered1446worker1 # [ 0.476209] PCI: Using ACPI for IRQ routing1447lb # [ 0.473717] usbcore: registered new interface driver usbfs1448lb # [ 0.474460] usbcore: registered new interface driver hub1449lb # [ 0.474747] usbcore: registered new device driver usb1450lb # [ 0.476464] NetLabel: Initializing1451lb # [ 0.476744] NetLabel: domain hash size = 1281452lb # [ 0.477615] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453lb # [ 0.478576] NetLabel: unlabeled traffic allowed by default1454lb # [ 0.479462] PCI: Using ACPI for IRQ routing1455client # [ 0.524745] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.524922] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.524922] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.524931] vgaarb: loaded1459client # [ 0.525674] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.525928] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.530927] pci 0000:00:01.0: vgaarb: setting as boot VGA device1462worker2 # [ 0.531288] pci 0000:00:01.0: vgaarb: bridge control possible1463client # [ 0.532008] clocksource: Switched to clocksource kvm-clock1464worker2 # [ 0.531288] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1465worker2 # [ 0.531297] vgaarb: loaded1466worker2 # [ 0.532068] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01467client # [ 0.535532] VFS: Disk quotas dquot_6.6.01468worker2 # [ 0.532294] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1469client # [ 0.536283] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1470client # [ 0.537707] pnp: PnP ACPI init1471client # [ 0.538518] ACPI: IRQ 4 override to edge(!), high(!)1472worker1 # [ 0.520900] pci 0000:00:01.0: vgaarb: setting as boot VGA device1473worker1 # [ 0.521193] pci 0000:00:01.0: vgaarb: bridge control possible1474client # [ 0.539507] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1475worker1 # [ 0.521193] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1476client # [ 0.541170] pnp: PnP ACPI: found 5 devices1477worker2 # [ 0.538380] clocksource: Switched to clocksource kvm-clock1478worker1 # [ 0.521201] vgaarb: loaded1479worker1 # [ 0.521962] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01480worker1 # [ 0.522197] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1481worker2 # [ 0.541945] VFS: Disk quotas dquot_6.6.01482lb # [ 0.522091] pci 0000:00:01.0: vgaarb: setting as boot VGA device1483worker2 # [ 0.542729] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1484lb # [ 0.522734] pci 0000:00:01.0: vgaarb: bridge control possible1485worker2 # [ 0.544301] pnp: PnP ACPI init1486lb # [ 0.522734] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1487worker2 # [ 0.545106] ACPI: IRQ 4 override to edge(!), high(!)1488lb # [ 0.522743] vgaarb: loaded1489worker1 # [ 0.527287] clocksource: Switched to clocksource kvm-clock1490client # [ 0.548851] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1491lb # [ 0.523455] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01492worker2 # [ 0.546093] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1493client # [ 0.550248] clocksource: Switched to clocksource acpi_pm1494lb # [ 0.523738] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495worker2 # [ 0.547582] pnp: PnP ACPI: found 5 devices1496client # [ 0.551209] NET: Registered PF_INET protocol family1497worker1 # [ 0.530726] VFS: Disk quotas dquot_6.6.01498client # [ 0.552430] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1499worker1 # [ 0.531428] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1500worker1 # [ 0.532750] pnp: PnP ACPI init1501worker1 # [ 0.533564] ACPI: IRQ 4 override to edge(!), high(!)1502lb # [ 0.527828] clocksource: Switched to clocksource kvm-clock1503worker1 # [ 0.534529] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1504worker1 # [ 0.536083] pnp: PnP ACPI: found 5 devices1505lb # [ 0.531303] VFS: Disk quotas dquot_6.6.01506worker2 # [ 0.555138] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1507lb # [ 0.531993] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1508worker2 # [ 0.556674] clocksource: Switched to clocksource acpi_pm1509lb # [ 0.533275] pnp: PnP ACPI init1510worker2 # [ 0.557649] NET: Registered PF_INET protocol family1511lb # [ 0.534038] ACPI: IRQ 4 override to edge(!), high(!)1512worker2 # [ 0.558902] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1513lb # [ 0.535004] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1514lb # [ 0.536547] pnp: PnP ACPI: found 5 devices1515worker1 # [ 0.543681] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1516worker1 # [ 0.545191] clocksource: Switched to clocksource acpi_pm1517worker1 # [ 0.546217] NET: Registered PF_INET protocol family1518worker1 # [ 0.547482] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1519client # [ 0.568897] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1520client # [ 0.570403] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521client # [ 0.571767] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522lb # [ 0.544183] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523lb # [ 0.545784] clocksource: Switched to clocksource acpi_pm1524client # [ 0.573154] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1525lb # [ 0.546729] NET: Registered PF_INET protocol family1526client # [ 0.574427] TCP: Hash tables configured (established 16384 bind 16384)1527lb # [ 0.547865] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1528client # [ 0.575576] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1529client # [ 0.576877] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1530client # [ 0.578059] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1531worker2 # [ 0.575354] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1532client # [ 0.579314] NET: Registered PF_UNIX/PF_LOCAL protocol family1533client # [ 0.580310] NET: Registered PF_XDP protocol family1534worker2 # [ 0.577003] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1535client # [ 0.581192] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1536worker2 # [ 0.578389] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1537client # [ 0.582244] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1538worker2 # [ 0.579783] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1539client # [ 0.583305] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1540worker2 # [ 0.581069] TCP: Hash tables configured (established 16384 bind 16384)1541client # [ 0.584414] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1542worker2 # [ 0.582216] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1543client # [ 0.585546] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544worker1 # [ 0.563990] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1545worker2 # [ 0.583469] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1546client # [ 0.586644] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547worker1 # [ 0.565526] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1548worker2 # [ 0.584553] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1549client # [ 0.588357] ACPI: \_SB_.GSIA: Enabled at IRQ 161550worker1 # [ 0.566810] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1551worker2 # [ 0.585808] NET: Registered PF_UNIX/PF_LOCAL protocol family1552worker2 # [ 0.586805] NET: Registered PF_XDP protocol family1553worker1 # [ 0.568195] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1554client # [ 0.590661] ACPI: \_SB_.GSIB: Enabled at IRQ 171555worker2 # [ 0.587630] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1556worker1 # [ 0.569465] TCP: Hash tables configured (established 16384 bind 16384)1557worker2 # [ 0.588634] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1558lb # [ 0.563953] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1559worker1 # [ 0.570539] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1560client # [ 0.592842] ACPI: \_SB_.GSIC: Enabled at IRQ 181561worker2 # [ 0.589652] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1562lb # [ 0.565425] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1563worker1 # [ 0.571777] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1564worker2 # [ 0.590743] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1565worker1 # [ 0.572856] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1566lb # [ 0.566672] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1567client # [ 0.594816] ACPI: \_SB_.GSID: Enabled at IRQ 191568worker2 # [ 0.591862] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker1 # [ 0.574050] NET: Registered PF_UNIX/PF_LOCAL protocol family1570lb # [ 0.567957] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1571client # [ 0.596447] PCI: CLS 0 bytes, default 641572worker1 # [ 0.575027] NET: Registered PF_XDP protocol family1573worker2 # [ 0.592969] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1574lb # [ 0.569175] TCP: Hash tables configured (established 16384 bind 16384)1575worker1 # [ 0.575861] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1576worker2 # [ 0.594624] ACPI: \_SB_.GSIA: Enabled at IRQ 161577client # [ 0.597409] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578lb # [ 0.570248] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1579worker1 # [ 0.576873] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1580client # [ 0.599128] Trying to unpack rootfs image as initramfs...1581lb # [ 0.571421] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1582worker1 # [ 0.577854] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1583lb # [ 0.572460] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1584worker2 # [ 0.597296] ACPI: \_SB_.GSIB: Enabled at IRQ 171585worker1 # [ 0.578937] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1586lb # [ 0.573650] NET: Registered PF_UNIX/PF_LOCAL protocol family1587worker1 # [ 0.580011] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1588lb # [ 0.574609] NET: Registered PF_XDP protocol family1589worker2 # [ 0.599198] ACPI: \_SB_.GSIC: Enabled at IRQ 181590worker1 # [ 0.581145] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1591lb # [ 0.575429] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1592worker1 # [ 0.582841] ACPI: \_SB_.GSIA: Enabled at IRQ 161593lb # [ 0.576431] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1594worker2 # [ 0.601087] ACPI: \_SB_.GSID: Enabled at IRQ 191595lb # [ 0.577416] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1596worker2 # [ 0.602794] PCI: CLS 0 bytes, default 641597worker1 # [ 0.584977] ACPI: \_SB_.GSIB: Enabled at IRQ 171598lb # [ 0.578532] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1599worker2 # [ 0.603780] Trying to unpack rootfs image as initramfs...1600lb # [ 0.579612] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1601worker1 # [ 0.587105] ACPI: \_SB_.GSIC: Enabled at IRQ 181602lb # [ 0.580684] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1603lb # [ 0.582362] ACPI: \_SB_.GSIA: Enabled at IRQ 161604worker1 # [ 0.589038] ACPI: \_SB_.GSID: Enabled at IRQ 191605worker2 # [ 0.606119] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1606worker1 # [ 0.590728] PCI: CLS 0 bytes, default 641607lb # [ 0.584559] ACPI: \_SB_.GSIB: Enabled at IRQ 171608worker1 # [ 0.591629] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1609lb # [ 0.586684] ACPI: \_SB_.GSIC: Enabled at IRQ 181610worker1 # [ 0.593391] Trying to unpack rootfs image as initramfs...1611lb # [ 0.588561] ACPI: \_SB_.GSID: Enabled at IRQ 191612lb # [ 0.590585] PCI: CLS 0 bytes, default 641613lb # [ 0.591467] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614lb # [ 0.593279] Trying to unpack rootfs image as initramfs...1615client # [ 0.643544] Initialise system trusted keyrings1616client # [ 0.644532] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker2 # [ 0.649087] Initialise system trusted keyrings1618worker2 # [ 0.650110] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker1 # [ 0.635864] Initialise system trusted keyrings1620worker1 # [ 0.639519] workingset: timestamp_bits=40 max_order=19 bucket_order=01621lb # [ 0.636547] Initialise system trusted keyrings1622lb # [ 0.637490] workingset: timestamp_bits=40 max_order=19 bucket_order=01623client # [ 0.668748] Key type asymmetric registered1624client # [ 0.669456] Asymmetric key parser 'x509' registered1625client # [ 0.670368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1626client # [ 0.673731] io scheduler mq-deadline registered1627client # [ 0.674545] io scheduler kyber registered1628worker2 # [ 0.673813] Key type asymmetric registered1629worker2 # [ 0.674569] Asymmetric key parser 'x509' registered1630client # [ 0.678821] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1631worker2 # [ 0.675543] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1632client # [ 0.680068] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1633worker1 # [ 0.660636] Key type asymmetric registered1634worker1 # [ 0.661367] Asymmetric key parser 'x509' registered1635worker2 # [ 0.680819] io scheduler mq-deadline registered1636worker1 # [ 0.662262] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1637worker2 # [ 0.681611] io scheduler kyber registered1638client # [ 0.685405] Linux agpgart interface v0.1031639client # [ 0.686204] ACPI: bus type drm_connector registered1640worker2 # [ 0.684219] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1641lb # [ 0.661255] Key type asymmetric registered1642worker1 # [ 0.667563] io scheduler mq-deadline registered1643worker2 # [ 0.685513] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1644client # [ 0.689052] usbcore: registered new interface driver usbserial_generic1645lb # [ 0.661927] Asymmetric key parser 'x509' registered1646worker1 # [ 0.668280] io scheduler kyber registered1647client # [ 0.690108] usbserial: USB Serial support registered for generic1648lb # [ 0.662737] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1649worker1 # [ 0.671050] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1650lb # [ 0.666297] io scheduler mq-deadline registered1651worker2 # [ 0.691099] Linux agpgart interface v0.1031652worker1 # [ 0.672379] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1653lb # [ 0.667044] io scheduler kyber registered1654worker2 # [ 0.691878] ACPI: bus type drm_connector registered1655client # [ 0.694630] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1656client # [ 0.694630] Please enable it if your BIOS has the CPPC option.1657client # [ 0.696906] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658lb # [ 0.669714] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1659worker2 # [ 0.695120] usbcore: registered new interface driver usbserial_generic1660lb # [ 0.670928] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1661worker1 # [ 0.677864] Linux agpgart interface v0.1031662worker2 # [ 0.696213] usbserial: USB Serial support registered for generic1663client # [ 0.699762] drop_monitor: Initializing network drop monitor service1664worker1 # [ 0.678615] ACPI: bus type drm_connector registered1665client # [ 0.700875] NET: Registered PF_INET6 protocol family1666worker2 # [ 0.698697] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1667worker2 # [ 0.698697] Please enable it if your BIOS has the CPPC option.1668worker1 # [ 0.681888] usbcore: registered new interface driver usbserial_generic1669client # [ 0.704037] Segment Routing with IPv61670lb # [ 0.676434] Linux agpgart interface v0.1031671worker2 # [ 0.700990] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1672worker1 # [ 0.682988] usbserial: USB Serial support registered for generic1673lb # [ 0.677179] ACPI: bus type drm_connector registered1674client # [ 0.706653] In-situ OAM (IOAM) with IPv61675worker2 # [ 0.703805] drop_monitor: Initializing network drop monitor service1676client # [ 0.707767] IPI shorthand broadcast: enabled1677worker2 # [ 0.705044] NET: Registered PF_INET6 protocol family1678lb # [ 0.680386] usbcore: registered new interface driver usbserial_generic1679worker1 # [ 0.686485] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1680lb # [ 0.681420] usbserial: USB Serial support registered for generic1681worker1 # [ 0.686485] Please enable it if your BIOS has the CPPC option.1682worker1 # [ 0.688719] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1683lb # [ 0.684176] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1684worker2 # [ 0.710132] Segment Routing with IPv61685lb # [ 0.684176] Please enable it if your BIOS has the CPPC option.1686worker1 # [ 0.691590] drop_monitor: Initializing network drop monitor service1687worker2 # [ 0.710834] In-situ OAM (IOAM) with IPv61688worker1 # [ 0.692762] NET: Registered PF_INET6 protocol family1689lb # [ 0.686361] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1690worker2 # [ 0.712956] IPI shorthand broadcast: enabled1691client # [ 0.715884] sched_clock: Marking stable (580020248, 135585922)->(790455437, -74849267)1692lb # [ 0.689320] drop_monitor: Initializing network drop monitor service1693worker1 # [ 0.696047] Segment Routing with IPv61694lb # [ 0.690466] NET: Registered PF_INET6 protocol family1695client # [ 0.719868] registered taskstats version 11696worker1 # [ 0.698509] In-situ OAM (IOAM) with IPv61697client # [ 0.720833] Loading compiled-in X.509 certificates1698worker1 # [ 0.699534] IPI shorthand broadcast: enabled1699lb # [ 0.693595] Segment Routing with IPv61700lb # [ 0.696215] In-situ OAM (IOAM) with IPv61701lb # [ 0.697275] IPI shorthand broadcast: enabled1702worker2 # [ 0.721316] sched_clock: Marking stable (585014818, 135656381)->(796467777, -75796578)1703worker1 # [ 0.707724] sched_clock: Marking stable (571014967, 136442578)->(777735591, -70278046)1704worker2 # [ 0.726896] registered taskstats version 11705worker2 # [ 0.727892] Loading compiled-in X.509 certificates1706lb # [ 0.704527] sched_clock: Marking stable (570014122, 134139379)->(775138804, -70985303)1707worker1 # [ 0.711696] registered taskstats version 11708worker1 # [ 0.712668] Loading compiled-in X.509 certificates1709lb # [ 0.710340] registered taskstats version 11710client # [ 0.738625] Demotion targets for Node 0: null1711lb # [ 0.711328] Loading compiled-in X.509 certificates1712client # [ 0.741652] Key type .fscrypt registered1713client # [ 0.742372] Key type fscrypt-provisioning registered1714client # [ 0.743369] ima: No TPM chip found, activating TPM-bypass!1715worker2 # [ 0.744302] Demotion targets for Node 0: null1716client # [ 0.747622] ima: Allocated hash algorithm: sha11717client # [ 0.748416] ima: No architecture policies found1718client # [ 0.749418] PM: Magic number: 10:612:2371719worker2 # [ 0.747720] Key type .fscrypt registered1720worker2 # [ 0.748419] Key type fscrypt-provisioning registered1721worker2 # [ 0.749429] ima: No TPM chip found, activating TPM-bypass!1722worker1 # [ 0.731472] Demotion targets for Node 0: null1723client # [ 0.753119] RAS: Correctable Errors collector initialized.1724worker1 # [ 0.732308] Key type .fscrypt registered1725worker2 # [ 0.751683] ima: Allocated hash algorithm: sha11726worker2 # [ 0.752526] ima: No architecture policies found1727worker1 # [ 0.735461] Key type fscrypt-provisioning registered1728lb # [ 0.730167] Demotion targets for Node 0: null1729worker1 # [ 0.736388] ima: No TPM chip found, activating TPM-bypass!1730lb # [ 0.731029] Key type .fscrypt registered1731worker1 # [ 0.737351] ima: Allocated hash algorithm: sha11732worker1 # [ 0.738190] ima: No architecture policies found1733worker2 # [ 0.756693] PM: Magic number: 10:612:2371734worker2 # [ 0.758349] RAS: Correctable Errors collector initialized.1735worker1 # [ 0.740468] PM: Magic number: 10:612:2371736lb # [ 0.734158] Key type fscrypt-provisioning registered1737client # [ 0.762777] clk: Disabling unused clocks1738lb # [ 0.735040] ima: No TPM chip found, activating TPM-bypass!1739client # [ 0.763453] PM: genpd: Disabling unused power domains1740worker1 # [ 0.741974] RAS: Correctable Errors collector initialized.1741lb # [ 0.735934] ima: Allocated hash algorithm: sha11742lb # [ 0.736756] ima: No architecture policies found1743lb # [ 0.739166] PM: Magic number: 10:612:2371744lb # [ 0.740720] RAS: Correctable Errors collector initialized.1745worker2 # [ 0.766368] clk: Disabling unused clocks1746worker1 # [ 0.751160] clk: Disabling unused clocks1747worker2 # [ 0.769720] PM: genpd: Disabling unused power domains1748worker1 # [ 0.755484] PM: genpd: Disabling unused power domains1749lb # [ 0.750822] clk: Disabling unused clocks1750lb # [ 0.753187] PM: genpd: Disabling unused power domains1751client # [ 0.924497] Freeing initrd memory: 29056K1752client # [ 0.927774] Freeing unused decrypted memory: 2028K1753client # [ 0.930423] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.931540] Write protecting the kernel read-only data: 32768k1755client # [ 0.933441] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker2 # [ 0.931310] Freeing initrd memory: 29060K1757worker1 # [ 0.913516] Freeing initrd memory: 29056K1758client # [ 0.934989] Freeing unused kernel image (rodata/data gap) memory: 720K1759worker2 # [ 0.934497] Freeing unused decrypted memory: 2028K1760worker1 # [ 0.916599] Freeing unused decrypted memory: 2028K1761worker2 # [ 0.937144] Freeing unused kernel image (initmem) memory: 3652K1762worker1 # [ 0.919122] Freeing unused kernel image (initmem) memory: 3652K1763lb # [ 0.913603] Freeing initrd memory: 29064K1764worker2 # [ 0.938216] Write protecting the kernel read-only data: 32768k1765worker1 # [ 0.920225] Write protecting the kernel read-only data: 32768k1766worker2 # [ 0.940026] Freeing unused kernel image (text/rodata gap) memory: 1184K1767worker1 # [ 0.922082] Freeing unused kernel image (text/rodata gap) memory: 1184K1768lb # [ 0.916655] Freeing unused decrypted memory: 2028K1769worker2 # [ 0.941547] Freeing unused kernel image (rodata/data gap) memory: 720K1770worker1 # [ 0.923627] Freeing unused kernel image (rodata/data gap) memory: 720K1771lb # [ 0.919177] Freeing unused kernel image (initmem) memory: 3652K1772lb # [ 0.920262] Write protecting the kernel read-only data: 32768k1773lb # [ 0.922065] Freeing unused kernel image (text/rodata gap) memory: 1184K1774lb # [ 0.923566] Freeing unused kernel image (rodata/data gap) memory: 720K1775client # [ 0.986911] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776client # [ 0.988052] Run /init as init process1777worker2 # [ 0.992458] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778worker1 # [ 0.974527] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker2 # [ 0.993568] Run /init as init process1780worker1 # [ 0.975550] Run /init as init process1781client # [ 0.999138] systemd[1]: Inserted module 'autofs4'1782lb # [ 0.975533] x86/mm: Checked W+X mappings: passed, no W+X pages found.1783lb # [ 0.976612] Run /init as init process1784worker1 # [ 0.985983] systemd[1]: Inserted module 'autofs4'1785worker2 # [ 1.004453] systemd[1]: Inserted module 'autofs4'1786lb # [ 0.986992] systemd[1]: Inserted module 'autofs4'1787client # [ 1.018999] fuse: init (API version 7.45)1788client # [ 1.026376] ACPI: \_SB_.GSIG: Enabled at IRQ 221789worker1 # [ 1.005751] fuse: init (API version 7.45)1790worker2 # [ 1.024369] fuse: init (API version 7.45)1791client # [ 1.028823] ACPI: \_SB_.GSIH: Enabled at IRQ 231792client # [ 1.031881] ACPI: \_SB_.GSIE: Enabled at IRQ 201793client # [ 1.033863] ACPI: \_SB_.GSIF: Enabled at IRQ 211794lb # [ 1.006605] fuse: init (API version 7.45)1795worker1 # [ 1.013139] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker2 # [ 1.031767] ACPI: \_SB_.GSIG: Enabled at IRQ 221797worker1 # [ 1.015485] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker2 # [ 1.034402] ACPI: \_SB_.GSIH: Enabled at IRQ 231799client # [ 1.039211] virtiofs virtio5: discovered new tag: nix-store1800worker1 # [ 1.018356] ACPI: \_SB_.GSIE: Enabled at IRQ 201801worker2 # [ 1.037762] ACPI: \_SB_.GSIE: Enabled at IRQ 201802client # [ 1.040875] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1803lb # [ 1.014016] ACPI: \_SB_.GSIG: Enabled at IRQ 221804worker1 # [ 1.020368] ACPI: \_SB_.GSIF: Enabled at IRQ 211805worker2 # [ 1.039928] ACPI: \_SB_.GSIF: Enabled at IRQ 211806lb # [ 1.016351] ACPI: \_SB_.GSIH: Enabled at IRQ 231807worker1 # [ 1.025380] virtiofs virtio5: discovered new tag: nix-store1808lb # [ 1.019408] ACPI: \_SB_.GSIE: Enabled at IRQ 201809client # [ 1.048014] virtiofs virtio6: discovered new tag: shared1810worker1 # [ 1.026932] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1811worker2 # [ 1.045551] virtiofs virtio5: discovered new tag: nix-store1812lb # [ 1.021291] ACPI: \_SB_.GSIF: Enabled at IRQ 211813client # [ 1.049715] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1814worker2 # [ 1.047073] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1815client # [ 1.052840] virtiofs virtio7: discovered new tag: xchg1816lb # [ 1.025848] virtiofs virtio5: discovered new tag: nix-store1817client # [ 1.054276] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1818worker1 # [ 1.033681] virtiofs virtio6: discovered new tag: shared1819lb # [ 1.027892] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1820worker1 # [ 1.035196] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1821worker2 # [ 1.054569] virtiofs virtio6: discovered new tag: shared1822worker2 # [ 1.056077] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823worker1 # [ 1.038423] virtiofs virtio7: discovered new tag: xchg1824worker1 # [ 1.039922] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1825lb # [ 1.034466] virtiofs virtio6: discovered new tag: shared1826worker2 # [ 1.060507] virtiofs virtio7: discovered new tag: xchg1827lb # [ 1.035969] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1828worker2 # [ 1.062403] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1829lb # [ 1.039786] virtiofs virtio7: discovered new tag: xchg1830lb # [ 1.041319] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1831client # [ 1.073402] systemd[1]: Successfully made /usr/ read-only.1832worker1 # [ 1.061619] systemd[1]: Successfully made /usr/ read-only.1833worker2 # [ 1.084476] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.063864] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.408889] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.414021] systemd[1]: Detected virtualization kvm.1837client # [ 1.414909] systemd[1]: Detected architecture x86-64.1838client # [ 1.415790] systemd[1]: Running in initrd.1839client # [ 1.416855] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.418071] systemd[1]: Hostname set to <client>.1841worker1 # [ 1.397232] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker1 # [ 1.402412] systemd[1]: Detected virtualization kvm.1843worker1 # [ 1.403307] systemd[1]: Detected architecture x86-64.1844worker1 # [ 1.404194] systemd[1]: Running in initrd.1845worker2 # [ 1.419960] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1846worker1 # [ 1.405351] systemd[1]: Initializing machine ID from random generator.1847worker2 # [ 1.424703] systemd[1]: Detected virtualization kvm.1848worker1 # [ 1.406695] systemd[1]: Hostname set to <worker1>.1849worker2 # [ 1.425515] systemd[1]: Detected architecture x86-64.1850worker2 # [ 1.426343] systemd[1]: Running in initrd.1851lb # [ 1.398902] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1852worker2 # [ 1.427354] systemd[1]: Initializing machine ID from random generator.1853lb # [ 1.403778] systemd[1]: Detected virtualization kvm.1854worker2 # [ 1.428544] systemd[1]: Hostname set to <worker2>.1855lb # [ 1.404618] systemd[1]: Detected architecture x86-64.1856lb # [ 1.405490] systemd[1]: Running in initrd.1857lb # [ 1.406547] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.407736] systemd[1]: Hostname set to <lb>.1859client # [ 1.606958] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker1 # [ 1.596118] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861lb # [ 1.607380] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.635924] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.644436] systemd[1]: Queued start job for default target Initrd Default Target.1864client # [ 1.648883] systemd[1]: Created slice Slice /system/modprobe.1865client # [ 1.650096] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866client # [ 1.651560] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867client # [ 1.652744] systemd[1]: Reached target Path Units.1868client # [ 1.653635] systemd[1]: Reached target Slice Units.1869client # [ 1.654510] systemd[1]: Reached target Swaps.1870client # [ 1.655335] systemd[1]: Reached target Timer Units.1871client # [ 1.656319] systemd[1]: Listening on D-Bus System Message Bus Socket.1872client # [ 1.657555] systemd[1]: Listening on Journal Socket (/dev/log).1873worker1 # [ 1.635973] systemd[1]: Queued start job for default target Initrd Default Target.1874client # [ 1.658715] systemd[1]: Listening on Journal Sockets.1875client # [ 1.659667] systemd[1]: Listening on udev Control Socket.1876client # [ 1.660641] systemd[1]: Listening on udev Kernel Socket.1877worker1 # [ 1.639857] systemd[1]: Created slice Slice /system/modprobe.1878client # [ 1.661529] systemd[1]: Reached target Socket Units.1879worker1 # [ 1.641072] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1880client # [ 1.663334] systemd[1]: Starting Create List of Static Device Nodes...1881worker1 # [ 1.642559] systemd[1]: Expecting device /dev/disk/by-label/nixos...1882worker1 # [ 1.643688] systemd[1]: Reached target Path Units.1883worker1 # [ 1.644604] systemd[1]: Reached target Slice Units.1884worker1 # [ 1.645489] systemd[1]: Reached target Swaps.1885worker1 # [ 1.646285] systemd[1]: Reached target Timer Units.1886client # [ 1.667674] systemd[1]: Starting Load Kernel Module configfs...1887worker1 # [ 1.647275] systemd[1]: Listening on D-Bus System Message Bus Socket.1888worker1 # [ 1.648555] systemd[1]: Listening on Journal Socket (/dev/log).1889worker1 # [ 1.649729] systemd[1]: Listening on Journal Sockets.1890worker1 # [ 1.650732] systemd[1]: Listening on udev Control Socket.1891worker1 # [ 1.651807] systemd[1]: Listening on udev Kernel Socket.1892worker1 # [ 1.652809] systemd[1]: Reached target Socket Units.1893worker1 # [ 1.654802] systemd[1]: Starting Create List of Static Device Nodes...1894worker1 # [ 1.659526] systemd[1]: Starting Load Kernel Module configfs...1895lb # [ 1.653408] systemd[1]: Queued start job for default target Initrd Default Target.1896lb # [ 1.657399] systemd[1]: Created slice Slice /system/modprobe.1897lb # [ 1.658569] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1898client # [ 1.676268] systemd[1]: Starting Journal Service...1899lb # [ 1.659986] systemd[1]: Expecting device /dev/disk/by-label/nixos...1900lb # [ 1.661057] systemd[1]: Reached target Path Units.1901lb # [ 1.661938] systemd[1]: Reached target Slice Units.1902lb # [ 1.662832] systemd[1]: Reached target Swaps.1903lb # [ 1.663650] systemd[1]: Reached target Timer Units.1904worker2 # [ 1.688762] systemd[1]: Queued start job for default target Initrd Default Target.1905lb # [ 1.664625] systemd[1]: Listening on D-Bus System Message Bus Socket.1906lb # [ 1.665861] systemd[1]: Listening on Journal Socket (/dev/log).1907lb # [ 1.667055] systemd[1]: Listening on Journal Sockets.1908worker2 # [ 1.691962] systemd[1]: Created slice Slice /system/modprobe.1909lb # [ 1.668054] systemd[1]: Listening on udev Control Socket.1910worker2 # [ 1.693107] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1911lb # [ 1.669067] systemd[1]: Listening on udev Kernel Socket.1912lb # [ 1.670037] systemd[1]: Reached target Socket Units.1913worker2 # [ 1.694494] systemd[1]: Expecting device /dev/disk/by-label/nixos...1914worker2 # [ 1.695570] systemd[1]: Reached target Path Units.1915worker2 # [ 1.696440] systemd[1]: Reached target Slice Units.1916lb # [ 1.671768] systemd[1]: Starting Create List of Static Device Nodes...1917client # [ 1.700226] systemd[1]: Starting Load Kernel Modules...1918worker2 # [ 1.697304] systemd[1]: Reached target Swaps.1919worker2 # [ 1.698094] systemd[1]: Reached target Timer Units.1920worker1 # [ 1.669004] systemd[1]: Starting Journal Service...1921worker2 # [ 1.699051] systemd[1]: Listening on D-Bus System Message Bus Socket.1922worker2 # [ 1.700243] systemd[1]: Listening on Journal Socket (/dev/log).1923lb # [ 1.676275] systemd[1]: Starting Load Kernel Module configfs...1924worker2 # [ 1.701356] systemd[1]: Listening on Journal Sockets.1925client # [ 1.704687] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1926worker2 # [ 1.702309] systemd[1]: Listening on udev Control Socket.1927worker2 # [ 1.703296] systemd[1]: Listening on udev Kernel Socket.1928worker2 # [ 1.704237] systemd[1]: Reached target Socket Units.1929worker2 # [ 1.706019] systemd[1]: Starting Create List of Static Device Nodes...1930client # [ 1.711678] systemd[1]: Starting Coldplug All udev Devices...1931worker2 # [ 1.709744] systemd[1]: Starting Load Kernel Module configfs...1932worker1 # [ 1.695170] systemd[1]: Starting Load Kernel Modules...1933worker1 # [ 1.696285] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1934lb # [ 1.684222] systemd[1]: Starting Journal Service...1935client # [ 1.724380] systemd-journald[66]: Collecting audit messages is disabled.1936client # [ 1.726105] systemd[1]: Finished Create List of Static Device Nodes.1937client # [ 1.732124] systemd[1]: modprobe@configfs.service: Deactivated successfully.1938worker2 # [ 1.718876] systemd[1]: Starting Journal Service...1939worker1 # [ 1.712546] systemd[1]: Starting Coldplug All udev Devices...1940lb # [ 1.709359] systemd[1]: Starting Load Kernel Modules...1941client # [ 1.740011] systemd[1]: Finished Load Kernel Module configfs.1942lb # [ 1.713249] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1943worker1 # [ 1.720888] systemd-journald[66]: Collecting audit messages is disabled.1944lb # [ 1.719911] systemd[1]: Starting Coldplug All udev Devices...1945client # [ 1.747038] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1946worker2 # [ 1.744862] systemd[1]: Starting Load Kernel Modules...1947worker1 # [ 1.728828] systemd[1]: Finished Create List of Static Device Nodes.1948worker2 # [ 1.748775] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1949worker1 # [ 1.731974] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950client # [ 1.757668] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1951worker2 # [ 1.755854] systemd[1]: Starting Coldplug All udev Devices...1952worker1 # [ 1.739891] systemd[1]: Finished Load Kernel Module configfs.1953lb # [ 1.733620] systemd-journald[66]: Collecting audit messages is disabled.1954lb # [ 1.734929] systemd[1]: Finished Create List of Static Device Nodes.1955worker1 # [ 1.744905] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1956lb # [ 1.739733] systemd[1]: modprobe@configfs.service: Deactivated successfully.1957client # [ 1.767380] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1958worker2 # [ 1.770061] systemd-journald[66]: Collecting audit messages is disabled.1959lb # [ 1.746557] systemd[1]: Finished Load Kernel Module configfs.1960worker2 # [ 1.772729] systemd[1]: Finished Create List of Static Device Nodes.1961client # [ 1.775801] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1962worker1 # [ 1.757684] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1963lb # [ 1.752648] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1964worker2 # [ 1.778235] systemd[1]: modprobe@configfs.service: Deactivated successfully.1965worker1 # [ 1.761915] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1966worker2 # [ 1.786053] systemd[1]: Finished Load Kernel Module configfs.1967lb # [ 1.763221] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1968worker2 # [ 1.790132] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1969worker1 # [ 1.774667] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1970client # [ 1.799175] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1971lb # [ 1.772984] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1972worker2 # [ 1.802746] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1973client # [ 1.807824] systemd[1]: Starting Create Static Device Nodes in /dev...1974lb # [ 1.782298] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1975worker2 # [ 1.808502] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1976client # [ 1.819700] systemd[1]: Finished Load Kernel Modules.1977worker2 # [ 1.819916] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1978client # [ 1.827882] systemd[1]: Starting Apply Kernel Variables...1979worker1 # [ 1.810105] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1980lb # [ 1.804716] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1981worker1 # [ 1.818692] systemd[1]: Starting Create Static Device Nodes in /dev...1982lb # [ 1.814348] systemd[1]: Starting Create Static Device Nodes in /dev...1983worker2 # [ 1.843281] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1984client # [ 1.712422] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.849206] systemd[1]: Started Journal Service.1985worker1 # [ 1.829035] systemd[1]: Finished Load Kernel Modules.1986client # 1987lb # [ 1.825675] systemd[1]: Finished Load Kernel Modules.1988client # [ 1.716316] systemd-modules-load[67]: Inserted module 'virtio_balloon'1989client # [ 1.719364] systemd-modules-load[67]: Inserted module 'virtio_gpu'1990worker2 # [ 1.854040] systemd[1]: Starting Create Static Device Nodes in /dev...1991client # [ 1.724088] systemd[1]: Finished Create Static Device Nodes in /dev.1992worker1 # [ 1.701706] systemd-modules-load[67]: Inserted module 'dm_mod'1993lb # [ 1.834371] systemd[1]: Starting Apply Kernel Variables...1994worker1 # [ 1.704962] systemd-modules-load[67]: Inserted module 'virtio_balloon'1995worker1 # [ 1.706141] systemd-modules-load[67]: Inserted module 'virtio_gpu'1996client # [ 1.728862] systemd[1]: Reached target Preparation for Local File Systems.1997worker1 # [ 1.843710] systemd[1]: Starting Apply Kernel Variables...1998client # [ 1.731099] systemd[1]: Reached target Local File Systems.1999worker2 # [ 1.864225] systemd[1]: Finished Load Kernel Modules.2000client # [ 1.733317] systemd[1]: Starting Create System Files and Directories...
loading 2,498 hidden lines…
4499client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4500client # copying 3 paths...4501client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4502client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4503client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4504client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4505(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 4.50 seconds)4506subtest: repeat is answered Cached by the scheduler without building4507worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4508worker1 # finding garbage collector roots...4509worker1 # deleting specified paths...4510worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4511worker1 # deleting unused links...4512worker1 # note: hard linking is currently saving 0.0 KiB4513worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4514worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4515worker2 # finding garbage collector roots...4516worker2 # deleting specified paths...4517worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4518worker2 # deleting unused links...4519worker2 # note: hard linking is currently saving 0.0 KiB4520worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4521client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24522worker2 # [ 50.155238] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427564523client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4524client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4525worker2 # [ 50.171093] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427564526worker2 # [ 50.180413] nix-grpc-daemon[872]: time=2026-09-23T16:12:59.354Z level=INFO msg="All 2 paths already cached"4527worker1 # [ 50.177334] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14528client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4529client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4530client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4531lb: must succeed: curl -sf localhost:9901/clusters4532lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4533lb: must succeed: curl -sf localhost:9901/clusters4534lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4535worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4536worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4537(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.80 seconds)4538subtest: inputs already in the cache are not sent to the scheduler4539lb: must succeed: curl -sf localhost:9901/clusters4540lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4541lb: must succeed: curl -sf localhost:9901/clusters4542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4543worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4544worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4545client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24546worker1 # [ 50.684969] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418704547client # don't know how to build these paths:4548client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4549worker1 # [ 50.693593] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704550worker1 # [ 50.714202] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04551worker2 # [ 50.736792] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564552worker1 # [ 50.718469] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14553worker2 # [ 50.746979] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427564554worker2 # [ 50.760580] nix-grpc-daemon[872]: time=2026-09-23T16:12:59.933Z level=INFO msg="All 2 paths already cached"4555client # copying 1 paths to 'lb:50051'...4556client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4557worker2 # [ 50.769295] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427564558worker2 # [ 50.781309] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4559worker2 # [ 50.882970] nix-daemon[860]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4560worker2 # [ 50.897564] nix-daemon[860]: reaped child process 991, status = succeeded4561lb # [ 50.880885] niks3-server[952]: 2026/09/23 16:12:59 INFO Received push request method=POST path=/api/pushes4562worker2 # [ 50.914857] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.088Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4563worker2 # [ 50.916299] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.088Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4564lb # [ 50.912202] niks3-server[952]: 2026/09/23 16:12:59 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4565lb # [ 50.915620] niks3-server[952]: 2026/09/23 16:12:59 INFO Signed narinfos id=7 count=14566worker2 # [ 50.942424] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.116Z level=INFO msg="Uploading 1 narinfos"4567lb # [ 50.924136] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4568lb # [ 50.928694] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4569lb # [ 50.936987] niks3-server[952]: 2026/09/23 16:12:59 INFO Received complete push request method=POST path=/api/pushes/7/complete4570lb # [ 50.941973] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4571worker2 # [ 50.969236] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.142Z level=INFO msg="Upload complete. (175ms)"4572worker2 # [ 50.973453] nix-grpc-daemon[802]: ts=2026-09-23T16:13:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=7524573worker2 # [ 50.977970] nix-grpc-daemon[802]: ts=2026-09-23T16:13:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564574worker2 # [ 50.988585] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4575worker2 # [ 50.992744] nix-daemon[860]: reaped child process 986, status = succeeded4576worker2 # [ 50.996483] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4577client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4578client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4579client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4580client # farm-top> LOG-farm-top-t1b4581worker2 # [ 52.236210] nix-daemon[860]: reaped child process 995, status = succeeded4582worker2 # [ 52.317588] nix-daemon[860]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4583worker2 # [ 52.329496] nix-daemon[860]: reaped child process 1014, status = succeeded4584lb # [ 52.313127] niks3-server[952]: 2026/09/23 16:13:00 INFO Received push request method=POST path=/api/pushes4585worker2 # [ 52.344508] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4586worker2 # [ 52.346331] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.519Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4587lb # [ 52.356557] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4588lb # [ 52.361376] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4589lb # [ 52.367183] niks3-server[952]: 2026/09/23 16:13:00 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4590worker2 # [ 52.396822] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.570Z level=INFO msg="Uploading 1 narinfos"4591lb # [ 52.373995] niks3-server[952]: 2026/09/23 16:13:00 INFO Signed narinfos id=8 count=14592lb # [ 52.379564] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4593lb # [ 52.390185] niks3-server[952]: 2026/09/23 16:13:00 INFO Received complete push request method=POST path=/api/pushes/8/complete4594worker2 # [ 52.421158] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.594Z level=INFO msg="Upload complete. (216ms)"4595lb # [ 52.399147] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4596worker2 # [ 52.426701] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14597worker2 # [ 52.440948] nix-daemon[860]: reaped child process 993, status = succeeded4598client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.90 seconds)4599client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4600client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4601lb: must succeed: curl -sf localhost:9901/clusters4602lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4603lb: must succeed: curl -sf localhost:9901/clusters4604lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4605worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4606worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4607(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.20 seconds)4608subtest: outputs read back through any worker carry the cache signature4609client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24610worker2 # [ 52.814645] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427564611client # don't know how to build these paths:4612client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4613worker2 # [ 52.823223] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427564614worker1 # [ 52.828216] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04615worker1 # [ 52.834492] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14616worker1 # [ 52.839231] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704617worker1 # [ 52.849599] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704618client # copying 1 paths to 'lb:50051'...4619client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4620worker1 # [ 52.861974] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418704621worker1 # [ 52.870813] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4622worker1 # [ 52.893535] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=info event=niks3_push_started pid=9184623worker1 # [ 53.018171] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.264Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4624worker1 # [ 53.107521] nix-daemon[868]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4625worker1 # [ 53.122273] nix-daemon[868]: reaped child process 934, status = succeeded4626lb # [ 53.123467] niks3-server[952]: 2026/09/23 16:13:01 INFO Received push request method=POST path=/api/pushes4627worker1 # [ 53.134918] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.382Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4628worker1 # [ 53.136474] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.383Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4629lb # [ 53.159128] niks3-server[952]: 2026/09/23 16:13:01 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4630lb # [ 53.165128] niks3-server[952]: 2026/09/23 16:13:01 INFO Signed narinfos id=9 count=14631worker1 # [ 53.172180] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.419Z level=INFO msg="Uploading 1 narinfos"4632lb # [ 53.172501] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4633lb # [ 53.189119] niks3-server[952]: 2026/09/23 16:13:01 INFO Received complete push request method=POST path=/api/pushes/9/complete4634lb # [ 53.191818] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4635worker1 # [ 53.201825] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.449Z level=INFO msg="Upload complete. (179ms)"4636lb # [ 53.199943] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4637worker1 # [ 53.204654] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=5044638worker1 # [ 53.210762] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704639worker1 # [ 53.220559] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4640worker1 # [ 53.223364] nix-daemon[868]: reaped child process 916, status = succeeded4641worker1 # [ 53.227218] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4642client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4643client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4644client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4645client # farm-a> LOG-farm-a-sig4646worker1 # [ 54.242360] nix-daemon[868]: reaped child process 938, status = succeeded4647worker1 # [ 54.316507] nix-daemon[868]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4648worker1 # [ 54.328456] nix-daemon[868]: reaped child process 957, status = succeeded4649lb # [ 54.330941] niks3-server[952]: 2026/09/23 16:13:02 INFO Received push request method=POST path=/api/pushes4650worker1 # [ 54.343002] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.590Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4651worker1 # [ 54.344967] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.591Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4652lb # [ 54.377296] niks3-server[952]: 2026/09/23 16:13:02 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4653lb # [ 54.382764] niks3-server[952]: 2026/09/23 16:13:02 INFO Signed narinfos id=10 count=14654worker1 # [ 54.389482] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.636Z level=INFO msg="Uploading 1 narinfos"4655lb # [ 54.396478] niks3-server[952]: 2026/09/23 16:13:02 INFO Received complete push request method=POST path=/api/pushes/10/complete4656lb # [ 54.409576] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4657lb # [ 54.415692] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4658lb # [ 54.422801] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4659lb # [ 54.428080] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4660worker1 # [ 54.435487] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.682Z level=INFO msg="Upload complete. (201ms)"4661worker1 # [ 54.441966] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14662worker1 # [ 54.453650] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04663worker1 # [ 54.458623] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14664worker1 # [ 54.462199] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704665worker1 # [ 54.472296] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704666worker1 # [ 54.474672] nix-daemon[868]: reaped child process 936, status = succeeded4667worker1 # [ 54.479274] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.726Z level=INFO msg="All 1 paths already cached"4668client # copying 1 paths to 'lb:50051'...4669worker1 # [ 54.484584] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418704670client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4671worker1 # [ 54.496148] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4672worker1 # [ 54.579611] nix-daemon[868]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4673worker1 # [ 54.592835] nix-daemon[868]: reaped child process 966, status = succeeded4674lb # [ 54.596865] niks3-server[952]: 2026/09/23 16:13:02 INFO Received push request method=POST path=/api/pushes4675worker1 # [ 54.608480] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4676worker1 # [ 54.610471] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.857Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4677lb # [ 54.624742] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4678lb # [ 54.630348] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=11 count=14679worker1 # [ 54.637736] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.884Z level=INFO msg="Uploading 1 narinfos"4680lb # [ 54.639835] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4681lb # [ 54.649255] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4682lb # [ 54.657935] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/11/complete4683lb # [ 54.665973] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4684worker1 # [ 54.673147] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.920Z level=INFO msg="Upload complete. (172ms)"4685worker1 # [ 54.677922] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=6244686worker1 # [ 54.684276] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704687worker1 # [ 54.693290] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4688worker1 # [ 54.700793] nix-daemon[868]: reaped child process 961, status = succeeded4689worker1 # [ 54.705454] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4690client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4691client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4692client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4693client # farm-b> LOG-farm-b-sig4694worker1 # [ 55.138316] nix-daemon[868]: reaped child process 970, status = succeeded4695worker1 # [ 55.212794] nix-daemon[868]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4696worker1 # [ 55.224869] nix-daemon[868]: reaped child process 989, status = succeeded4697lb # [ 55.228447] niks3-server[952]: 2026/09/23 16:13:03 INFO Received push request method=POST path=/api/pushes4698worker1 # [ 55.240766] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4699worker1 # [ 55.242127] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.487Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4700lb # [ 55.267063] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4701lb # [ 55.270761] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4702lb # [ 55.279171] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=12 count=14703worker1 # [ 55.287157] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.534Z level=INFO msg="Uploading 1 narinfos"4704lb # [ 55.284599] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4705lb # [ 55.291887] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4706lb # [ 55.301325] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/12/complete4707lb # [ 55.309789] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4708worker1 # [ 55.312662] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.560Z level=INFO msg="Upload complete. (201ms)"4709worker1 # [ 55.318692] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14710worker1 # [ 55.330127] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04711worker1 # [ 55.335366] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14712worker1 # [ 55.339516] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704713worker1 # [ 55.349152] nix-daemon[868]: reaped child process 968, status = succeeded4714worker1 # [ 55.350780] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704715worker1 # [ 55.359417] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.606Z level=INFO msg="All 2 paths already cached"4716client # copying 1 paths to 'lb:50051'...4717worker1 # [ 55.365322] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418704718client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4719worker1 # [ 55.371344] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4720worker1 # [ 55.456926] nix-daemon[868]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4721worker1 # [ 55.470404] nix-daemon[868]: reaped child process 998, status = succeeded4722lb # [ 55.473274] niks3-server[952]: 2026/09/23 16:13:03 INFO Received push request method=POST path=/api/pushes4723worker1 # [ 55.491787] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.739Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4724worker1 # [ 55.493443] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.740Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4725lb # [ 55.510160] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4726lb # [ 55.520201] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=13 count=14727worker1 # [ 55.528664] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.775Z level=INFO msg="Uploading 1 narinfos"4728lb # [ 55.527120] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4729lb # [ 55.531395] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4730lb # [ 55.543311] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/13/complete4731lb # [ 55.550776] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4732worker1 # [ 55.562676] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.810Z level=INFO msg="Upload complete. (186ms)"4733worker1 # [ 55.568444] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=7524734worker1 # [ 55.575331] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704735worker1 # [ 55.584654] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4736worker1 # [ 55.596386] nix-daemon[868]: reaped child process 993, status = succeeded4737client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4738worker1 # [ 55.603394] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4739client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4740client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4741client # farm-top> LOG-farm-top-sig4742worker1 # [ 56.842670] nix-daemon[868]: reaped child process 1002, status = succeeded4743lb # [ 56.868779] envoy[921]: conn peer=[2001:db8:1::1]:42406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9434 tx=4339 ms=40154744lb # [ 56.872593] envoy[921]: conn peer=[2001:db8:1::1]:42408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=554745lb # [ 56.876005] envoy[921]: conn peer=[2001:db8:1::1]:42414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1899 ms=17344746worker1 # [ 56.919491] nix-daemon[868]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)4747worker1 # [ 56.930577] nix-daemon[868]: reaped child process 1021, status = succeeded4748lb # [ 56.935361] niks3-server[952]: 2026/09/23 16:13:05 INFO Received push request method=POST path=/api/pushes4749worker1 # [ 56.947760] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.194Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4750worker1 # [ 56.949257] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.196Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4751lb # [ 56.977517] niks3-server[952]: 2026/09/23 16:13:05 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4752lb # [ 56.981889] niks3-server[952]: 2026/09/23 16:13:05 INFO Signed narinfos id=14 count=14753worker1 # [ 56.988350] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.235Z level=INFO msg="Uploading 1 narinfos"4754lb # [ 56.991640] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4755lb # [ 56.996711] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4756lb # [ 57.000888] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4757lb # [ 57.007827] niks3-server[952]: 2026/09/23 16:13:05 INFO Received complete push request method=POST path=/api/pushes/14/complete4758lb # [ 57.014064] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4759worker1 # [ 57.017537] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.264Z level=INFO msg="Upload complete. (207ms)"4760worker1 # [ 57.025628] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14761worker1 # [ 57.038741] nix-daemon[868]: reaped child process 1000, status = succeeded4762client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.40 seconds)4763client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4764client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4765client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4766worker1 # [ 57.283192] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:609304767worker1 # [ 57.288779] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:609304768client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.15 seconds)4769client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4770worker2 # [ 57.444305] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:383284771client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4772client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4773client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4774client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4775worker2 # [ 57.469264] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:383284776worker2 # [ 57.475695] nix-grpc-daemon[802]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4777worker2 # [ 57.489354] nix-grpc-daemon[802]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4778worker2 # [ 57.501357] nix-grpc-daemon[802]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4779client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)4780client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4781client # finding garbage collector roots...4782client # removing stale temporary roots file "/nix/var/nix/temproots/903"4783client # removing stale temporary roots file "/nix/var/nix/temproots/847"4784client # removing stale temporary roots file "/nix/var/nix/temproots/880"4785client # removing stale temporary roots file "/nix/var/nix/temproots/867"4786client # removing stale temporary roots file "/nix/var/nix/temproots/859"4787client # removing stale temporary roots file "/nix/var/nix/temproots/826"4788client # removing stale temporary roots file "/nix/var/nix/temproots/811"4789client # removing stale temporary roots file "/nix/var/nix/temproots/888"4790client # deleting specified paths...4791client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4792client # deleting unused links...4793client # note: hard linking is currently saving 0.0 KiB4794worker1 # [ 57.766392] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418704795worker1 # [ 57.772115] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704796worker1 # [ 57.776456] nix-grpc-daemon[918]: time=2026-09-23T16:13:06.023Z level=INFO msg="All 1 paths already cached"4797worker2 # [ 57.803341] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427564798worker1 # [ 57.791409] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418704799client # copying 3 paths...4800worker2 # [ 57.820322] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427564801client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4802worker1 # [ 57.817311] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418704803worker1 # [ 57.822689] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=2324804worker2 # [ 57.843561] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:427564805worker1 # [ 57.830624] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418704806worker2 # [ 57.849062] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=1764807worker1 # [ 57.835570] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=1284808client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4809client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4810client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.35 seconds)4811(finished: subtest: outputs read back through any worker carry the cache signature, in 5.21 seconds)4812subtest: two clients wanting the same drv build it once4813client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4814client # [ 57.913648] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4815client # Running as unit: dedup1.service; invocation ID: 8c933fa1c6354e2cb434908e2905059e4816client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4817client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4818client # [ 58.043305] nix[956]: don't know how to build these paths:4819client # [ 58.044663] nix[956]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4820worker2 # [ 58.038944] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427564821worker1 # [ 58.031366] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04822worker2 # [ 58.051692] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564823worker1 # [ 58.035517] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14824worker2 # [ 58.063323] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427564825client # [ 58.076472] nix[956]: copying 1 paths to 'lb:50051'...4826client # [ 58.077354] nix[956]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4827worker2 # [ 58.074485] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427564828worker2 # [ 58.084987] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4829worker2 # [ 58.187950] nix-daemon[860]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4830worker2 # [ 58.201772] nix-daemon[860]: reaped child process 1024, status = succeeded4831lb # [ 58.185316] niks3-server[952]: 2026/09/23 16:13:06 INFO Received push request method=POST path=/api/pushes4832worker2 # [ 58.216735] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.389Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4833worker2 # [ 58.218138] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.389Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4834lb # [ 58.213958] niks3-server[952]: 2026/09/23 16:13:06 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4835worker2 # [ 58.242560] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.416Z level=INFO msg="Uploading 1 narinfos"4836lb # [ 58.220459] niks3-server[952]: 2026/09/23 16:13:06 INFO Signed narinfos id=15 count=14837lb # [ 58.227611] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4838lb # [ 58.234794] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4839lb # [ 58.246685] niks3-server[952]: 2026/09/23 16:13:06 INFO Received complete push request method=POST path=/api/pushes/15/complete4840lb # [ 58.252929] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4841worker2 # [ 58.281441] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.455Z level=INFO msg="Upload complete. (185ms)"4842worker2 # [ 58.285703] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=4324843worker2 # [ 58.291268] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564844worker2 # [ 58.300250] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4845worker2 # [ 58.309130] nix-daemon[860]: reaped child process 1019, status = succeeded4846worker2 # [ 58.312325] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4847client # [ 58.315590] nix[956]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4848client # [ 58.967623] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4849client # Running as unit: dedup2.service; invocation ID: b57ae4626980468aa04207299271cf4a4850client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4851client: must succeed: sleep 24852worker1 # [ 59.116776] nix-grpc-daemon[512]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418704853client # [ 59.144791] nix[972]: don't know how to build these paths:4854client # [ 59.145784] nix[972]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4855worker1 # [ 59.129427] nix-grpc-daemon[512]: ts=2026-09-23T16:13:07Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04856worker2 # [ 59.153377] nix-grpc-daemon[802]: ts=2026-09-23T16:13:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564857client # [ 59.159406] nix[972]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4858client: (finished: must succeed: sleep 2, in 2.05 seconds)4859client: waiting for success: ! systemctl is-active dedup1 dedup24860worker2 # [ 61.343129] nix-daemon[860]: reaped child process 1028, status = succeeded4861worker2 # [ 61.421415] nix-daemon[860]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4862worker2 # [ 61.432846] nix-daemon[860]: reaped child process 1057, status = succeeded4863lb # [ 61.417343] niks3-server[952]: 2026/09/23 16:13:09 INFO Received push request method=POST path=/api/pushes4864worker2 # [ 61.449130] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4865worker2 # [ 61.450695] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.624Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4866lb # [ 61.464797] niks3-server[952]: 2026/09/23 16:13:09 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4867lb # [ 61.470108] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4868lb # [ 61.475535] niks3-server[952]: 2026/09/23 16:13:09 INFO Signed narinfos id=16 count=14869worker2 # [ 61.498738] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.672Z level=INFO msg="Uploading 1 narinfos"4870lb # [ 61.479364] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4871lb # [ 61.486217] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4872lb # [ 61.490449] niks3-server[952]: 2026/09/23 16:13:09 INFO Received complete push request method=POST path=/api/pushes/16/complete4873worker2 # [ 61.519863] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.693Z level=INFO msg="Upload complete. (198ms)"4874lb # [ 61.499968] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4875worker2 # [ 61.526452] nix-grpc-daemon[802]: ts=2026-09-23T16:13:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14876worker2 # [ 61.533496] nix-grpc-daemon[802]: ts=2026-09-23T16:13:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14877worker2 # [ 61.545821] nix-daemon[860]: reaped child process 1026, status = succeeded4878client # [ 61.573536] systemd[1]: dedup2.service: Deactivated successfully.4879client # [ 61.576748] systemd[1]: dedup2.service: Consumed 135ms CPU time over 2.608s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4880client # [ 61.581640] systemd[1]: dedup1.service: Deactivated successfully.4881client # [ 61.584595] systemd[1]: dedup1.service: Consumed 114ms CPU time over 3.670s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.4882client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4883client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4884client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4885worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4886worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4887worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4888worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4889worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4890worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4891worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4892worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4893(finished: subtest: two clients wanting the same drv build it once, in 4.64 seconds)4894subtest: --timeout and --max-silent-time reach the builder4895worker1 # [ 62.650846] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418704896worker1 # [ 62.663856] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04897worker1 # [ 62.670576] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14898worker1 # [ 62.675352] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704899worker1 # [ 62.682801] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704900worker1 # [ 62.693877] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418704901worker1 # [ 62.704275] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4902worker1 # [ 62.798182] nix-daemon[868]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4903worker1 # [ 62.812260] nix-daemon[868]: reaped child process 1051, status = succeeded4904lb # [ 62.816481] niks3-server[952]: 2026/09/23 16:13:11 INFO Received push request method=POST path=/api/pushes4905worker1 # [ 62.827336] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.074Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4906worker1 # [ 62.829237] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.076Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4907lb # [ 62.847588] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4908lb # [ 62.852609] niks3-server[952]: 2026/09/23 16:13:11 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4909lb # [ 62.861307] niks3-server[952]: 2026/09/23 16:13:11 INFO Signed narinfos id=17 count=14910worker1 # [ 62.868608] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.115Z level=INFO msg="Uploading 1 narinfos"4911lb # [ 62.869808] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4912lb # [ 62.881259] niks3-server[952]: 2026/09/23 16:13:11 INFO Received complete push request method=POST path=/api/pushes/17/complete4913worker1 # [ 62.892868] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.140Z level=INFO msg="Upload complete. (177ms)"4914lb # [ 62.889882] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4915worker1 # [ 62.896691] nix-grpc-daemon[512]: ts=2026-09-23T16:13:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=4244916worker1 # [ 62.901501] nix-grpc-daemon[512]: ts=2026-09-23T16:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704917worker1 # [ 62.913105] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4918worker1 # [ 62.916840] nix-daemon[868]: reaped child process 1046, status = succeeded4919worker1 # [ 62.921762] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4920lb # [ 66.869966] envoy[921]: conn peer=[2001:db8:1::1]:36184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=4336 ms=42424921lb # [ 66.873821] envoy[921]: conn peer=[2001:db8:1::1]:36200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2872 ms=944922lb # [ 66.877378] envoy[921]: conn peer=[2001:db8:1::1]:36208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=424923lb # [ 66.880913] envoy[921]: conn peer=[2001:db8:1::1]:36222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=294924lb # [ 66.884478] envoy[921]: conn peer=[2001:db8:1::1]:36220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=354925lb # [ 66.887899] envoy[921]: conn peer=[2001:db8:1::1]:36232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1521 ms=35004926lb # [ 66.891471] envoy[921]: conn peer=[2001:db8:1::1]:36234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=24114927worker1 # [ 69.134211] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04928worker1 # [ 69.146137] nix-daemon[868]: reaped child process 1053, status = succeeded4929worker1 # [ 69.147307] nix-daemon[868]: reaped child process 1055, status = succeeded4930worker1 # [ 69.299618] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418704931worker1 # [ 69.309957] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04932worker2 # [ 69.333388] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564933worker1 # [ 69.314525] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14934worker2 # [ 69.345695] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427564935worker2 # [ 69.356649] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427564936worker2 # [ 69.367621] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4937worker2 # [ 69.473368] nix-daemon[860]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4938worker2 # [ 69.487744] nix-daemon[860]: reaped child process 1073, status = succeeded4939lb # [ 69.472948] niks3-server[952]: 2026/09/23 16:13:17 INFO Received push request method=POST path=/api/pushes4940worker2 # [ 69.502232] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.675Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4941worker2 # [ 69.503648] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.675Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4942lb # [ 69.506730] niks3-server[952]: 2026/09/23 16:13:17 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4943lb # [ 69.513289] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4944worker2 # [ 69.540855] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.714Z level=INFO msg="Uploading 1 narinfos"4945lb # [ 69.517630] niks3-server[952]: 2026/09/23 16:13:17 INFO Signed narinfos id=18 count=14946lb # [ 69.531155] niks3-server[952]: 2026/09/23 16:13:17 INFO Received complete push request method=POST path=/api/pushes/18/complete4947lb # [ 69.534755] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4948worker2 # [ 69.562141] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.735Z level=INFO msg="Upload complete. (182ms)"4949lb # [ 69.540966] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4950worker2 # [ 69.566407] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=4244951worker2 # [ 69.570993] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427564952worker2 # [ 69.580686] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4953worker2 # [ 69.588145] nix-daemon[860]: reaped child process 1068, status = succeeded4954worker2 # [ 69.591153] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)4955worker2 # [ 75.804484] nix-grpc-daemon[802]: ts=2026-09-23T16:13:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04956worker2 # [ 75.815771] nix-daemon[860]: reaped child process 1075, status = succeeded4957worker2 # [ 75.816829] nix-daemon[860]: reaped child process 1077, status = succeeded4958(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.38 seconds)4959subtest: the first client leaving does not fail the second4960client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4961client # [ 75.953553] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4962client # Running as unit: share1.service; invocation ID: c53babbde545413b890b57e1eda7b9174963client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)4964client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4965worker2 # [ 76.088277] nix-grpc-daemon[802]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427564966client # [ 76.094687] nix[1045]: don't know how to build these paths:4967client # [ 76.096091] nix[1045]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4968worker1 # [ 76.085298] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04969worker1 # [ 76.089364] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=14970worker1 # [ 76.094504] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704971worker1 # [ 76.105661] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418704972client # [ 76.138089] nix[1045]: copying 1 paths to 'lb:50051'...4973client # [ 76.138880] nix[1045]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...4974worker1 # [ 76.116747] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418704975worker1 # [ 76.127337] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4976worker1 # [ 76.225979] nix-daemon[868]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4977worker1 # [ 76.238400] nix-daemon[868]: reaped child process 1085, status = succeeded4978lb # [ 76.241456] niks3-server[952]: 2026/09/23 16:13:24 INFO Received push request method=POST path=/api/pushes4979worker1 # [ 76.254160] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4980worker1 # [ 76.256123] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.503Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"4981lb # [ 76.284775] niks3-server[952]: 2026/09/23 16:13:24 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4982worker1 # [ 76.294212] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.541Z level=INFO msg="Uploading 1 narinfos"4983lb # [ 76.290803] niks3-server[952]: 2026/09/23 16:13:24 INFO Signed narinfos id=19 count=14984lb # [ 76.293555] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst4985lb # [ 76.303718] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls4986lb # [ 76.318438] niks3-server[952]: 2026/09/23 16:13:24 INFO Received complete push request method=POST path=/api/pushes/19/complete4987lb # [ 76.329337] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo4988worker1 # [ 76.333197] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.580Z level=INFO msg="Upload complete. (194ms)"4989worker1 # [ 76.338910] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=4324990worker1 # [ 76.344254] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418704991worker1 # [ 76.357180] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4992worker1 # [ 76.364097] nix-daemon[868]: reaped child process 1080, status = succeeded4993client # [ 76.393706] nix[1045]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...4994worker1 # [ 76.372697] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)4995lb # [ 76.870661] envoy[921]: conn peer=[2001:db8:1::1]:36248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=64984996lb # [ 76.874450] envoy[921]: conn peer=[2001:db8:1::1]:60702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=64984997client # [ 77.019849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4998client # Running as unit: share2.service; invocation ID: dbfa7d3c04eb441a9c9556aa805707a84999client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5000worker1 # [ 77.148077] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418705001client # [ 77.179287] nix[1061]: this derivation will be built:5002worker2 # [ 77.178496] nix-grpc-daemon[802]: ts=2026-09-23T16:13:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565003worker2 # [ 77.190749] nix-grpc-daemon[802]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'...5004client # [ 77.210089] nix[1061]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5005worker1 # [ 77.193198] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05006worker1 # [ 77.196242] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418705007client # [ 77.219324] nix[1061]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5008client: must succeed: sleep 2; systemctl kill -s INT share15009client # [ 80.236655] systemd[1]: share1.service: Sent signal SIGINT to main process 1045 (nix) on client request.5010client # [ 80.244933] nix[1045]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5011client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5012client: waiting for success: ! systemctl is-active share15013client # [ 80.251899] nix[1045]: error: interrupted by the user5014client # [ 80.294329] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5015client # [ 80.295873] systemd[1]: share1.service: Failed with result 'exit-code'.5016client # [ 80.297896] systemd[1]: share1.service: Consumed 117ms CPU time over 4.344s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5017client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5018client: waiting for success: ! systemctl is-active share25019worker1 # [ 81.595109] nix-daemon[868]: reaped child process 1089, status = succeeded5020worker1 # [ 81.669578] nix-daemon[868]: accepted connection from pid 1125, user nix-grpc-daemon (trusted)5021worker1 # [ 81.682082] nix-daemon[868]: reaped child process 1128, status = succeeded5022lb # [ 81.682897] niks3-server[952]: 2026/09/23 16:13:30 INFO Received push request method=POST path=/api/pushes5023worker1 # [ 81.697354] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5024worker1 # [ 81.699130] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.946Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5025lb # [ 81.732298] niks3-server[952]: 2026/09/23 16:13:30 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5026worker1 # [ 81.740592] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.987Z level=INFO msg="Uploading 1 narinfos"5027lb # [ 81.737226] niks3-server[952]: 2026/09/23 16:13:30 INFO Signed narinfos id=20 count=15028lb # [ 81.744955] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5029lb # [ 81.749885] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5030lb # [ 81.754707] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5031lb # [ 81.765340] niks3-server[952]: 2026/09/23 16:13:30 INFO Received complete push request method=POST path=/api/pushes/20/complete5032worker1 # [ 81.774862] nix-grpc-daemon[918]: time=2026-09-23T16:13:30.022Z level=INFO msg="Upload complete. (189ms)"5033lb # [ 81.773194] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5034worker1 # [ 81.784168] nix-grpc-daemon[512]: ts=2026-09-23T16:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15035worker1 # [ 81.789860] nix-grpc-daemon[512]: ts=2026-09-23T16:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15036worker1 # [ 81.802659] nix-daemon[868]: reaped child process 1087, status = succeeded5037client # [ 81.839607] systemd[1]: share2.service: Deactivated successfully.5038client # [ 81.844245] systemd[1]: share2.service: Consumed 130ms CPU time over 4.823s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic.5039client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5040client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5041client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5042(finished: subtest: the first client leaving does not fail the second, in 6.71 seconds)5043subtest: build hook: nix-daemon with builders = grpc://lb5044client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5045client # [ 82.716871] systemd[1]: Started Nix Daemon.5046client # [ 82.809198] nix-daemon[1112]: accepted connection from pid 1109, user root (trusted)5047worker1 # [ 82.910166] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418705048worker1 # [ 82.915237] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705049worker1 # [ 82.924902] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05050worker2 # [ 82.948343] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565051worker1 # [ 82.929185] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15052worker2 # [ 82.959698] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565053worker2 # [ 82.970557] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427565054worker2 # [ 82.981460] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5055worker2 # [ 83.080865] nix-daemon[860]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)5056worker2 # [ 83.093802] nix-daemon[860]: reaped child process 1120, status = succeeded5057lb # [ 83.077378] niks3-server[952]: 2026/09/23 16:13:31 INFO Received push request method=POST path=/api/pushes5058worker2 # [ 83.109148] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5059worker2 # [ 83.111093] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.284Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5060lb # [ 83.112112] niks3-server[952]: 2026/09/23 16:13:31 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5061lb # [ 83.116382] niks3-server[952]: 2026/09/23 16:13:31 INFO Signed narinfos id=21 count=15062worker2 # [ 83.142520] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.316Z level=INFO msg="Uploading 1 narinfos"5063lb # [ 83.124564] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5064lb # [ 83.134867] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5065lb # [ 83.142711] niks3-server[952]: 2026/09/23 16:13:31 INFO Received complete push request method=POST path=/api/pushes/21/complete5066worker2 # [ 83.176112] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.349Z level=INFO msg="Upload complete. (183ms)"5067lb # [ 83.155982] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5068worker2 # [ 83.181179] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=5125069worker2 # [ 83.186979] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565070worker2 # [ 83.198204] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5071worker2 # [ 83.211218] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5072worker2 # [ 83.213406] nix-daemon[860]: reaped child process 1115, status = succeeded5073worker2 # [ 84.251549] nix-daemon[860]: reaped child process 1124, status = succeeded5074worker2 # [ 84.326373] nix-daemon[860]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)5075worker2 # [ 84.337817] nix-daemon[860]: reaped child process 1143, status = succeeded5076lb # [ 84.321688] niks3-server[952]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes5077worker2 # [ 84.354144] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5078worker2 # [ 84.355906] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.529Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5079lb # [ 84.364673] niks3-server[952]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5080worker2 # [ 84.395736] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.569Z level=INFO msg="Uploading 1 narinfos"5081lb # [ 84.372935] niks3-server[952]: 2026/09/23 16:13:32 INFO Signed narinfos id=22 count=15082lb # [ 84.380130] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5083lb # [ 84.384733] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5084lb # [ 84.389468] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5085lb # [ 84.398657] niks3-server[952]: 2026/09/23 16:13:32 INFO Received complete push request method=POST path=/api/pushes/22/complete5086worker2 # [ 84.433236] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.606Z level=INFO msg="Upload complete. (213ms)"5087lb # [ 84.412112] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5088worker2 # [ 84.439711] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15089worker2 # [ 84.447462] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565090worker2 # [ 84.459378] nix-daemon[860]: reaped child process 1122, status = succeeded5091worker1 # [ 84.441896] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418705092worker1 # [ 84.453270] nix-grpc-daemon[512]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5093worker1 # [ 84.467274] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=1285094worker2 # [ 84.652275] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:427565095worker1 # [ 84.640726] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705096worker1 # [ 84.652124] nix-grpc-daemon[918]: time=2026-09-23T16:13:32.898Z level=INFO msg="All 1 paths already cached"5097worker1 # [ 84.666872] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05098worker2 # [ 84.688600] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565099worker1 # [ 84.671449] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15100worker2 # [ 84.700509] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565101worker2 # [ 84.713092] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.886Z level=INFO msg="All 1 paths already cached"5102worker2 # [ 84.720703] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427565103worker2 # [ 84.731164] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5104worker2 # [ 84.829553] nix-daemon[860]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5105worker2 # [ 84.842690] nix-daemon[860]: reaped child process 1150, status = succeeded5106lb # [ 84.825409] niks3-server[952]: 2026/09/23 16:13:33 INFO Received push request method=POST path=/api/pushes5107worker2 # [ 84.856285] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.030Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5108worker2 # [ 84.857820] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.031Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5109lb # [ 84.858917] niks3-server[952]: 2026/09/23 16:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5110lb # [ 84.863149] niks3-server[952]: 2026/09/23 16:13:33 INFO Signed narinfos id=23 count=15111worker2 # [ 84.890313] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.064Z level=INFO msg="Uploading 1 narinfos"5112lb # [ 84.871552] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5113lb # [ 84.879669] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5114lb # [ 84.890118] niks3-server[952]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/23/complete5115lb # [ 84.897946] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5116worker2 # [ 84.926175] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.099Z level=INFO msg="Upload complete. (182ms)"5117worker2 # [ 84.930465] nix-grpc-daemon[802]: ts=2026-09-23T16:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=6245118worker2 # [ 84.936338] nix-grpc-daemon[802]: ts=2026-09-23T16:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565119worker2 # [ 84.949113] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5120worker2 # [ 84.959100] nix-daemon[860]: reaped child process 1145, status = succeeded5121worker2 # [ 84.962034] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5122worker2 # [ 85.776610] nix-daemon[860]: reaped child process 1154, status = succeeded5123worker2 # [ 85.850035] nix-daemon[860]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)5124worker2 # [ 85.861318] nix-daemon[860]: reaped child process 1173, status = succeeded5125lb # [ 85.846365] niks3-server[952]: 2026/09/23 16:13:34 INFO Received push request method=POST path=/api/pushes5126worker2 # [ 85.879732] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.053Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5127worker2 # [ 85.881255] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.054Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5128lb # [ 85.890365] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5129lb # [ 85.900233] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5130lb # [ 85.904390] niks3-server[952]: 2026/09/23 16:13:34 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5131lb # [ 85.909391] niks3-server[952]: 2026/09/23 16:13:34 INFO Signed narinfos id=24 count=15132worker2 # [ 85.935841] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.109Z level=INFO msg="Uploading 1 narinfos"5133lb # [ 85.917232] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5134lb # [ 85.926502] niks3-server[952]: 2026/09/23 16:13:34 INFO Received complete push request method=POST path=/api/pushes/24/complete5135worker2 # [ 85.957309] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.130Z level=INFO msg="Upload complete. (189ms)"5136lb # [ 85.935850] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5137worker2 # [ 85.963704] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15138worker2 # [ 85.971344] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565139worker2 # [ 85.982434] nix-daemon[860]: reaped child process 1152, status = succeeded5140worker1 # [ 85.970583] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418705141worker1 # [ 85.980318] nix-grpc-daemon[512]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5142worker1 # [ 85.993332] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=1765143worker2 # [ 86.184185] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:427565144worker2 # [ 86.190101] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565145worker2 # [ 86.195821] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.367Z level=INFO msg="All 2 paths already cached"5146worker1 # [ 86.186642] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05147worker1 # [ 86.191554] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15148worker2 # [ 86.213568] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565149worker2 # [ 86.225290] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565150worker2 # [ 86.237874] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.411Z level=INFO msg="All 2 paths already cached"5151worker2 # [ 86.245842] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427565152worker2 # [ 86.256139] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5153worker2 # [ 86.355877] nix-daemon[860]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5154worker2 # [ 86.370549] nix-daemon[860]: reaped child process 1180, status = succeeded5155lb # [ 86.352673] niks3-server[952]: 2026/09/23 16:13:34 INFO Received push request method=POST path=/api/pushes5156worker2 # [ 86.384196] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.557Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5157worker2 # [ 86.385847] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.559Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5158lb # [ 86.384499] niks3-server[952]: 2026/09/23 16:13:34 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5159worker2 # [ 86.415448] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.589Z level=INFO msg="Uploading 1 narinfos"5160lb # [ 86.392933] niks3-server[952]: 2026/09/23 16:13:34 INFO Signed narinfos id=25 count=15161lb # [ 86.399094] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5162lb # [ 86.404388] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5163lb # [ 86.415859] niks3-server[952]: 2026/09/23 16:13:34 INFO Received complete push request method=POST path=/api/pushes/25/complete5164lb # [ 86.426426] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5165worker2 # [ 86.453140] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.626Z level=INFO msg="Upload complete. (184ms)"5166worker2 # [ 86.458125] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=7525167worker2 # [ 86.463702] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565168worker2 # [ 86.476108] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5169worker2 # [ 86.484275] nix-daemon[860]: reaped child process 1175, status = succeeded5170worker2 # [ 86.489511] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5171lb # [ 86.870552] envoy[921]: conn peer=[2001:db8:1::1]:44796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=770 ms=41715172lb # [ 86.874273] envoy[921]: conn peer=[2001:db8:1::1]:44800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1598 ms=46445173lb # [ 86.877742] envoy[921]: conn peer=[2001:db8:1::1]:50520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3412 tx=1945 ms=15705174lb # [ 86.881363] envoy[921]: conn peer=[2001:db8:1::1]:50524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=334 ms=385175lb # [ 86.884722] envoy[921]: conn peer=[2001:db8:1::1]:50526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3833 tx=2124 ms=13725176lb # [ 86.888373] envoy[921]: conn peer=[2001:db8:1::1]:50538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=379 ms=355177worker2 # [ 87.525553] nix-daemon[860]: reaped child process 1184, status = succeeded5178worker2 # [ 87.607764] nix-daemon[860]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5179worker2 # [ 87.619219] nix-daemon[860]: reaped child process 1203, status = succeeded5180lb # [ 87.603918] niks3-server[952]: 2026/09/23 16:13:36 INFO Received push request method=POST path=/api/pushes5181worker2 # [ 87.635528] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.809Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5182worker2 # [ 87.637254] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.810Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5183lb # [ 87.650960] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5184lb # [ 87.658423] niks3-server[952]: 2026/09/23 16:13:36 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5185lb # [ 87.661805] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5186worker2 # [ 87.691072] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.864Z level=INFO msg="Uploading 1 narinfos"5187lb # [ 87.668891] niks3-server[952]: 2026/09/23 16:13:36 INFO Signed narinfos id=26 count=15188lb # [ 87.671719] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5189lb # [ 87.682888] niks3-server[952]: 2026/09/23 16:13:36 INFO Received complete push request method=POST path=/api/pushes/26/complete5190worker2 # [ 87.715196] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.888Z level=INFO msg="Upload complete. (217ms)"5191lb # [ 87.693761] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5192worker2 # [ 87.721657] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15193worker2 # [ 87.728952] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565194worker2 # [ 87.741397] nix-daemon[860]: reaped child process 1182, status = succeeded5195worker2 # [ 87.746639] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:427565196worker2 # [ 87.750544] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=2325197client # [ 87.824782] nix-daemon[1112]: reaped child process 1114, status = succeeded5198client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.21 seconds)5199client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5200client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5201client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5202client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5203client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5204client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5205(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.27 seconds)5206subtest: an input only one worker has still reaches the builder5207client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5208client # this derivation will be built:5209client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5210client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5211client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds)5212client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5213client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5214worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5215worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5216worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5217worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5218client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25219client # [ 88.515346] nix-daemon[1112]: accepted connection from pid 1195, user root (trusted)5220client # this derivation will be built:5221client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5222worker2 # [ 88.636142] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:427565223client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5224worker2 # [ 88.641443] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565225client # copying 1 paths...5226client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5227worker2 # [ 88.652856] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427565228worker2 # [ 88.664201] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5229worker2 # [ 88.765954] nix-daemon[860]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5230worker2 # [ 88.778394] nix-daemon[860]: reaped child process 1213, status = succeeded5231lb # [ 88.762652] niks3-server[952]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes5232worker2 # [ 88.794857] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5233worker2 # [ 88.796757] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.970Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5234lb # [ 88.792420] niks3-server[952]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5235lb # [ 88.797967] niks3-server[952]: 2026/09/23 16:13:37 INFO Signed narinfos id=27 count=15236worker2 # [ 88.824206] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.997Z level=INFO msg="Uploading 1 narinfos"5237lb # [ 88.809176] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5238lb # [ 88.815703] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5239lb # [ 88.821440] niks3-server[952]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/27/complete5240lb # [ 88.832275] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5241worker2 # [ 88.854414] nix-grpc-daemon[872]: time=2026-09-23T16:13:38.028Z level=INFO msg="Upload complete. (178ms)"5242worker2 # [ 88.861283] nix-grpc-daemon[802]: ts=2026-09-23T16:13:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=1205243worker2 # [ 88.872152] nix-daemon[860]: reaped child process 1208, status = succeeded5244worker1 # [ 88.855945] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05245worker1 # [ 88.860880] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15246worker1 # [ 88.867973] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418705247worker1 # [ 88.877124] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705248client # copying 2 paths to 'lb:50051'...5249client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5250worker1 # [ 88.889740] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418705251client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5252worker1 # [ 88.901395] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5253worker1 # [ 88.999459] nix-daemon[868]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5254worker1 # [ 89.014267] nix-daemon[868]: reaped child process 1150, status = succeeded5255lb # [ 89.014284] niks3-server[952]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes5256worker1 # [ 89.027542] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.274Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5257worker1 # [ 89.030216] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.276Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5258worker1 # [ 89.033470] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.276Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5259lb # [ 89.056650] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5260lb # [ 89.068466] niks3-server[952]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5261lb # [ 89.071775] niks3-server[952]: 2026/09/23 16:13:37 INFO Signed narinfos id=28 count=25262worker1 # [ 89.076373] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.323Z level=INFO msg="Uploading 2 narinfos"5263lb # [ 89.074492] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5264lb # [ 89.077058] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5265lb # [ 89.079801] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5266lb # [ 89.087440] niks3-server[952]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/28/complete5267worker1 # [ 89.097270] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.344Z level=INFO msg="Upload complete. (183ms)"5268lb # [ 89.096714] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5269lb # [ 89.098527] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5270worker1 # [ 89.101637] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=2 nar_bytes_in=9765271worker1 # [ 89.107803] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418705272worker1 # [ 89.117498] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5273worker1 # [ 89.121195] nix-daemon[868]: reaped child process 1145, status = succeeded5274worker1 # [ 89.125846] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5275client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5276client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5277client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5278worker1 # [ 90.542884] nix-daemon[868]: reaped child process 1154, status = succeeded5279worker1 # [ 90.615956] nix-daemon[868]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)5280worker1 # [ 90.629334] nix-daemon[868]: reaped child process 1173, status = succeeded5281lb # [ 90.631706] niks3-server[952]: 2026/09/23 16:13:39 INFO Received push request method=POST path=/api/pushes5282worker1 # [ 90.644104] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.891Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5283lb # [ 90.665554] niks3-server[952]: 2026/09/23 16:13:39 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5284worker1 # [ 90.675900] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.923Z level=INFO msg="Uploading 1 narinfos"5285lb # [ 90.671634] niks3-server[952]: 2026/09/23 16:13:39 INFO Signed narinfos id=29 count=15286lb # [ 90.679421] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5287lb # [ 90.684758] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5288lb # [ 90.695352] niks3-server[952]: 2026/09/23 16:13:39 INFO Received complete push request method=POST path=/api/pushes/29/complete5289lb # [ 90.705939] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5290worker1 # [ 90.714927] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.962Z level=INFO msg="Upload complete. (181ms)"5291worker1 # [ 90.721962] nix-grpc-daemon[512]: ts=2026-09-23T16:13:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15292client # copying 1 paths...5293worker1 # [ 90.737827] nix-daemon[868]: reaped child process 1152, status = succeeded5294worker2 # [ 90.756115] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565295worker2 # [ 90.765422] nix-grpc-daemon[802]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5296client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5297worker2 # [ 90.795480] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:427565298worker2 # [ 90.800677] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=1205299client # [ 90.875347] nix-daemon[1112]: reaped child process 1198, status = succeeded5300client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.48 seconds)5301client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25302client # [ 90.975109] nix-daemon[1112]: accepted connection from pid 1217, user root (trusted)5303client # this derivation will be built:5304client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5305worker2 # [ 91.099273] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:427565306worker1 # [ 91.087447] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705307client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5308worker1 # [ 91.094117] nix-grpc-daemon[918]: time=2026-09-23T16:13:39.341Z level=INFO msg="All 1 paths already cached"5309worker1 # [ 91.104187] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05310worker1 # [ 91.106524] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15311worker2 # [ 91.127122] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565312worker2 # [ 91.138625] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427565313client # copying 1 paths to 'lb:50051'...5314client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5315worker2 # [ 91.154842] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427565316worker2 # [ 91.166180] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5317worker2 # [ 91.181993] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)5318worker2 # [ 91.267713] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 2/5)5319worker2 # [ 91.328806] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 250 ms (attempt 3/5)5320worker2 # [ 91.586769] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 224 ms (attempt 4/5)5321worker2 # [ 91.813880] nix-grpc-daemon[802]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5322worker2 # [ 91.817861] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5323worker2 # [ 91.822970] nix-grpc-daemon[802]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5324worker2 # [ 91.938516] nix-daemon[860]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5325worker2 # [ 91.955088] nix-daemon[860]: reaped child process 1226, status = succeeded5326lb # [ 91.938740] niks3-server[952]: 2026/09/23 16:13:40 INFO Received push request method=POST path=/api/pushes5327worker2 # [ 91.971965] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5328worker2 # [ 91.973485] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.147Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5329lb # [ 91.973611] niks3-server[952]: 2026/09/23 16:13:40 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5330lb # [ 91.981655] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5331lb # [ 91.989229] niks3-server[952]: 2026/09/23 16:13:40 INFO Signed narinfos id=30 count=15332worker2 # [ 92.016110] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.189Z level=INFO msg="Uploading 1 narinfos"5333lb # [ 91.994138] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5334lb # [ 92.008642] niks3-server[952]: 2026/09/23 16:13:40 INFO Received complete push request method=POST path=/api/pushes/30/complete5335lb # [ 92.017276] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5336worker2 # [ 92.045338] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.218Z level=INFO msg="Upload complete. (204ms)"5337worker2 # [ 92.050142] nix-grpc-daemon[802]: ts=2026-09-23T16:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=5605338worker2 # [ 92.055876] nix-grpc-daemon[802]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427565339worker2 # [ 92.067202] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5340worker2 # [ 92.079264] nix-daemon[860]: reaped child process 1216, status = succeeded5341worker2 # [ 92.084196] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted)5342client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5343client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5344client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5345worker2 # [ 92.921678] nix-daemon[860]: reaped child process 1230, status = succeeded5346worker2 # [ 93.002168] nix-daemon[860]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5347worker2 # [ 93.014803] nix-daemon[860]: reaped child process 1249, status = succeeded5348lb # [ 92.998041] niks3-server[952]: 2026/09/23 16:13:41 INFO Received push request method=POST path=/api/pushes5349worker2 # [ 93.026187] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.199Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5350lb # [ 93.014446] niks3-server[952]: 2026/09/23 16:13:41 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5351worker2 # [ 93.039297] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.213Z level=INFO msg="Uploading 1 narinfos"5352lb # [ 93.016337] niks3-server[952]: 2026/09/23 16:13:41 INFO Signed narinfos id=31 count=15353lb # [ 93.018930] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5354lb # [ 93.025593] niks3-server[952]: 2026/09/23 16:13:41 INFO Received complete push request method=POST path=/api/pushes/31/complete5355lb # [ 93.029195] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5356worker2 # [ 93.058060] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.231Z level=INFO msg="Upload complete. (165ms)"5357lb # [ 93.036666] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5358worker2 # [ 93.060532] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15359client # copying 1 paths...5360worker2 # [ 93.067241] nix-daemon[860]: reaped child process 1228, status = succeeded5361worker1 # [ 93.049658] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418705362worker1 # [ 93.058058] nix-grpc-daemon[512]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5363client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5364worker2 # [ 93.108749] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:427565365worker2 # [ 93.113552] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=1205366client # [ 93.188989] nix-daemon[1112]: reaped child process 1220, status = succeeded5367client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.32 seconds)5368client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25369client # [ 93.294121] nix-daemon[1112]: accepted connection from pid 1239, user root (trusted)5370client # this derivation will be built:5371client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5372worker1 # [ 93.396229] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418705373client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5374worker1 # [ 93.401999] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705375worker1 # [ 93.407465] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.653Z level=INFO msg="All 1 paths already cached"5376worker1 # [ 93.415742] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05377worker1 # [ 93.421977] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15378worker1 # [ 93.426980] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418705379worker1 # [ 93.434524] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418705380worker1 # [ 93.444620] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.691Z level=INFO msg="All 1 paths already cached"5381client # copying 1 paths to 'lb:50051'...5382client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5383worker1 # [ 93.451531] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418705384worker1 # [ 93.459905] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5385worker1 # [ 93.553356] nix-daemon[868]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5386worker1 # [ 93.565506] nix-daemon[868]: reaped child process 1185, status = succeeded5387lb # [ 93.568455] niks3-server[952]: 2026/09/23 16:13:41 INFO Received push request method=POST path=/api/pushes5388worker1 # [ 93.577811] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.825Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5389worker1 # [ 93.579476] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.826Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5390lb # [ 93.587326] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5391lb # [ 93.590970] niks3-server[952]: 2026/09/23 16:13:41 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5392worker1 # [ 93.597547] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.844Z level=INFO msg="Uploading 1 narinfos"5393lb # [ 93.593580] niks3-server[952]: 2026/09/23 16:13:41 INFO Signed narinfos id=32 count=15394lb # [ 93.599917] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5395lb # [ 93.604216] niks3-server[952]: 2026/09/23 16:13:42 INFO Received complete push request method=POST path=/api/pushes/32/complete5396worker1 # [ 93.617031] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.864Z level=INFO msg="Upload complete. (145ms)"5397lb # [ 93.613821] niks3-server[952]: 2026/09/23 16:13:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5398worker1 # [ 93.619221] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=5605399worker1 # [ 93.623189] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418705400worker1 # [ 93.628939] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5401worker1 # [ 93.631087] nix-daemon[868]: reaped child process 1180, status = succeeded5402worker1 # [ 93.636214] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted)5403client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5404client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5405client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5406worker1 # [ 94.476428] nix-daemon[868]: reaped child process 1189, status = succeeded5407worker1 # [ 94.563469] nix-daemon[868]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5408worker1 # [ 94.574434] nix-daemon[868]: reaped child process 1208, status = succeeded5409lb # [ 94.578465] niks3-server[952]: 2026/09/23 16:13:42 INFO Received push request method=POST path=/api/pushes5410worker1 # [ 94.591309] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.838Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5411lb # [ 94.611142] niks3-server[952]: 2026/09/23 16:13:43 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5412worker1 # [ 94.620479] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.867Z level=INFO msg="Uploading 1 narinfos"5413lb # [ 94.617079] niks3-server[952]: 2026/09/23 16:13:43 INFO Signed narinfos id=33 count=15414lb # [ 94.619619] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5415lb # [ 94.630673] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5416lb # [ 94.641238] niks3-server[952]: 2026/09/23 16:13:43 INFO Received complete push request method=POST path=/api/pushes/33/complete5417worker1 # [ 94.651967] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.899Z level=INFO msg="Upload complete. (209ms)"5418lb # [ 94.653347] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5419worker1 # [ 94.657470] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15420client # copying 1 paths...5421worker1 # [ 94.670197] nix-daemon[868]: reaped child process 1187, status = succeeded5422worker2 # [ 94.687307] nix-grpc-daemon[802]: ts=2026-09-23T16:13:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:427565423worker2 # [ 94.697803] nix-grpc-daemon[802]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5424client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5425worker1 # [ 94.708905] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418705426worker1 # [ 94.714144] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=1205427client # [ 94.807955] nix-daemon[1112]: reaped child process 1242, status = succeeded5428client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.62 seconds)5429(finished: subtest: an input only one worker has still reaches the builder, in 6.92 seconds)5430subtest: upload whose reference the worker lacks is completed from the cache5431client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5432client # this derivation will be built:5433client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5434client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5435client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5436client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5437client # this derivation will be built:5438client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5439client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5440client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.26 seconds)5441client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5442client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5443worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5444worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5445worker2 # [ 95.589812] systemd[1]: Stopping Nix worker-protocol over gRPC...5446worker2 # [ 95.874753] nix-grpc-daemon[802]: ts=2026-09-23T16:13:45Z level=info event=signal_exit5447lb # [ 96.871957] envoy[921]: conn peer=[2001:db8:1::1]:50552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4325 tx=2371 ms=15775448lb # [ 96.876632] envoy[921]: conn peer=[2001:db8:1::1]:50562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=155449lb # [ 96.880679] envoy[921]: conn peer=[2001:db8:1::1]:50574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4225 tx=2051 ms=21765450lb # [ 96.884830] envoy[921]: conn peer=[2001:db8:1::1]:50576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=175451lb # [ 96.888862] envoy[921]: conn peer=[2001:db8:1::1]:50582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2126 ms=20255452lb # [ 96.893246] envoy[921]: conn peer=[2001:db8:1::1]:60056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=175453lb # [ 96.897773] envoy[921]: conn peer=[2001:db8:1::1]:60068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3717 tx=2083 ms=13295454lb # [ 96.901928] envoy[921]: conn peer=[2001:db8:1::1]:60070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=185455worker1 # [ 97.856435] nix-grpc-daemon[512]: ts=2026-09-23T16:13:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:418385456worker1 # [ 97.860554] nix-grpc-daemon[512]: ts=2026-09-23T16:13:46Z level=info event=worker_gone addr=192.168.1.4:500515457worker2 # [ 97.871866] nix-grpc-daemon[802]: ts=2026-09-23T16:13:47Z level=info event=scheduler_restarting5458worker2 # [ 97.891402] nix-grpc-daemon[802]: ts=2026-09-23T16:13:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5459worker2 # [ 97.911934] nix-daemon[860]: reaped child process 862, status = succeeded5460worker2 # [ 98.580308] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5461worker2 # [ 98.584321] systemd[1]: Stopped Nix worker-protocol over gRPC.5462worker2 # [ 98.589292] systemd[1]: nix-grpc-daemon.service: Consumed 2.331s CPU time over 1min 5.867s wall clock time, 65.8M memory peak, 180K incoming IP traffic, 243.9K outgoing IP traffic.5463lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5464lb: must succeed: curl -sf localhost:9901/clusters5465lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5466lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5467client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25468worker1 # [ 98.781398] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418705469worker1 # [ 98.783867] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418705470client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5471client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5472worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5473worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5474worker2 # [ 98.882347] systemd[1]: Starting scheduler-token.service...5475worker2 # [ 98.918154] systemd[1]: scheduler-token.service: Deactivated successfully.5476worker2 # [ 98.919809] systemd[1]: Finished scheduler-token.service.5477worker2 # [ 98.922164] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 37ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5478worker2 # [ 98.925605] systemd[1]: Starting Nix worker-protocol over gRPC...5479worker2 # [ 99.000925] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5480worker2 # [ 99.002955] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5481worker2 # [ 99.006132] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15482worker2 # [ 99.010533] nix-grpc-daemon[1274]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5483worker2 # [ 99.011971] nix-grpc-daemon[1274]: E0000 00:00:1790180028.184210 1274 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5484worker2 # [ 99.016046] nix-grpc-daemon[1274]: E0000 00:00:1790180028.189674 1274 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5485worker2 # [ 99.020081] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5486worker2 # [ 99.023228] systemd[1]: Started Nix worker-protocol over gRPC.5487worker2 # [ 99.030048] nix-grpc-daemon[1274]: E0000 00:00:1790180028.201208 1316 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5488worker2 # [ 99.041064] nix-grpc-daemon[1274]: E0000 00:00:1790180028.214738 1316 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5489worker2 # [ 99.045145] nix-grpc-daemon[1274]: E0000 00:00:1790180028.217164 1316 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5490worker2 # [ 99.055735] nix-grpc-daemon[1274]: E0000 00:00:1790180028.229437 1316 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5491worker2 # [ 99.067183] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=scheduler_connected addr=lb:500515492worker1 # [ 99.051833] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:418385493worker1 # [ 99.060915] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05494worker1 # [ 99.083630] systemd[1]: Stopping Nix worker-protocol over gRPC...5495worker1 # [ 99.877639] nix-grpc-daemon[512]: ts=2026-09-23T16:13:48Z level=info event=signal_exit5496lb # [ 99.921900] niks3-server[952]: 2026/09/23 16:13:48 INFO lead: released remote=192.168.1.3:413865497worker1 # [ 100.925912] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=scheduler_restarting5498lb # [ 100.920644] envoy[921]: [2026-09-23 16:13:49.319][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5499worker2 # [ 100.953295] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_restarting addr=lb:500515500worker1 # [ 100.933756] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=scheduler_restarting addr=lb:500515501worker1 # [ 100.941364] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:418385502worker1 # [ 100.945466] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_gone addr=192.168.1.4:500515503worker1 # [ 100.948807] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:418385504worker1 # [ 100.952187] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_gone addr=192.168.1.3:500515505worker1 # [ 100.978602] nix-daemon[868]: reaped child process 870, status = succeeded5506worker2 # [ 101.153934] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:500515507worker2 # [ 101.158707] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5508worker1 # [ 101.150361] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5509worker2 # [ 101.359800] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:500515510worker2 # [ 101.363835] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5511worker2 # [ 101.564952] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:500515512worker2 # [ 101.568915] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5513worker2 # [ 101.769951] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:500515514worker2 # [ 101.774146] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker2 # [ 101.975645] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:500515516worker2 # [ 101.979507] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5517worker2 # [ 102.180725] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:500515518worker2 # [ 102.184679] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5519worker2 # [ 102.385844] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:500515520worker2 # [ 102.389867] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5521worker1 # [ 102.526452] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5522worker1 # [ 102.531878] systemd[1]: Stopped Nix worker-protocol over gRPC.5523worker1 # [ 102.534683] systemd[1]: nix-grpc-daemon.service: Consumed 1.972s CPU time over 1min 34.957s wall clock time, 65.1M memory peak, 417.9K incoming IP traffic, 264.5K outgoing IP traffic.5524lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5525lb: must succeed: curl -sf localhost:9901/clusters5526worker2 # [ 102.591088] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:500515527worker2 # [ 102.594989] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5529lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5530client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25531worker2 # [ 102.772069] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362325532worker2 # [ 102.782951] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5533worker2 # [ 102.795487] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:500515534worker2 # [ 102.799279] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker2 # [ 102.807816] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)5536worker2 # [ 102.847284] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5)5537worker2 # [ 103.001489] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515538worker2 # [ 103.005334] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5539worker2 # [ 103.049327] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 366 ms (attempt 3/5)5540worker2 # [ 103.206399] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515541worker2 # [ 103.210410] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5542worker2 # [ 103.411725] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515543worker2 # [ 103.416437] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5544worker2 # [ 103.421474] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 515 ms (attempt 4/5)5545worker2 # [ 103.617158] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515546worker2 # [ 103.621180] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5547worker2 # [ 103.822270] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515548worker2 # [ 103.826237] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5549worker2 # [ 103.945348] nix-grpc-daemon[1274]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5550worker2 # [ 103.948423] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5551client # copying 2 paths...5552client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5553worker2 # [ 103.971756] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362325554client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5555worker2 # [ 103.984301] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5556worker2 # [ 104.007071] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=niks3_push_started pid=13345557worker2 # [ 104.027423] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_connected addr=lb:500515558lb # [ 104.006983] niks3-server[952]: 2026/09/23 16:13:52 INFO lead: acquired remote=192.168.1.4:405985559worker2 # [ 104.032361] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker2 # [ 104.036268] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_take_over5561worker2 # [ 104.119736] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.293Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5562worker2 # [ 104.205141] nix-daemon[860]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5563worker2 # [ 104.216727] nix-daemon[860]: reaped child process 1349, status = succeeded5564lb # [ 104.200885] niks3-server[952]: 2026/09/23 16:13:52 INFO Received push request method=POST path=/api/pushes5565worker2 # [ 104.231578] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_connected addr=lb:500515566worker2 # [ 104.233355] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.407Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5567worker2 # [ 104.235822] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.408Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5568worker2 # [ 104.239103] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.411Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5569worker2 # [ 104.246867] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=oidc_jwks provider=mock keys=15570worker2 # [ 104.249394] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:511205571worker2 # [ 104.251684] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05572lb # [ 104.267224] niks3-server[952]: 2026/09/23 16:13:52 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5573worker2 # [ 104.296123] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.469Z level=INFO msg="Uploading 2 narinfos"5574lb # [ 104.274041] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5575lb # [ 104.275479] niks3-server[952]: 2026/09/23 16:13:52 INFO Signed narinfos id=34 count=25576lb # [ 104.278331] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5577lb # [ 104.287595] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5578lb # [ 104.289890] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5579lb # [ 104.297693] niks3-server[952]: 2026/09/23 16:13:52 INFO Received complete push request method=POST path=/api/pushes/34/complete5580lb # [ 104.302076] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5581lb # [ 104.307888] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5582worker2 # [ 104.334601] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.508Z level=INFO msg="Upload complete. (211ms)"5583worker2 # [ 104.339290] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=2 nar_bytes_in=3125584worker2 # [ 104.350858] nix-daemon[860]: reaped child process 1332, status = succeeded5585client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.75 seconds)5586worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5587worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5588worker1 # [ 104.421954] systemd[1]: Starting Nix worker-protocol over gRPC...5589worker1 # [ 104.511342] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5590worker1 # [ 104.513493] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5591worker1 # [ 104.516801] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15592worker1 # [ 104.521061] nix-grpc-daemon[1231]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5593worker1 # [ 104.524097] nix-grpc-daemon[1231]: E0000 00:00:1790180032.768312 1231 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5594worker1 # [ 104.526557] nix-grpc-daemon[1231]: E0000 00:00:1790180032.773933 1231 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5595worker1 # [ 104.530279] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5596worker1 # [ 104.534123] systemd[1]: Started Nix worker-protocol over gRPC.5597worker1 # [ 104.540087] nix-grpc-daemon[1231]: E0000 00:00:1790180032.785389 1274 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5598lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5599lb: must succeed: curl -sf localhost:9901/clusters5600worker1 # [ 104.552075] nix-grpc-daemon[1231]: E0000 00:00:1790180032.799350 1274 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5601worker1 # [ 104.555169] nix-grpc-daemon[1231]: E0000 00:00:1790180032.801726 1274 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5602worker1 # [ 104.566388] nix-grpc-daemon[1231]: E0000 00:00:1790180032.813840 1274 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5603worker1 # [ 104.579109] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:500515604worker2 # [ 104.600158] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:511205605worker2 # [ 104.604656] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05606lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5607lb: must succeed: curl -sf localhost:9901/clusters5608lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5609lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds)5610(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.86 seconds)5611subtest: a niks3 restart does not cost the leader its role5612lb: must succeed: curl -sf localhost:9901/clusters5613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5616lb # [ 105.716936] systemd[1]: Stopping niks3 server...5617lb # [ 105.718648] niks3-server[952]: 2026/09/23 16:13:54 INFO Shutdown signal received, draining in-flight requests timeout=10s5618lb # [ 105.722063] niks3-server[952]: 2026/09/23 16:13:54 INFO lead: released remote=192.168.1.4:405985619worker1 # [ 105.728766] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:53Z level=info event=scheduler_lock_lost status=2005620worker2 # [ 105.748688] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:54Z level=info event=scheduler_lock_lost status=2005621lb # [ 105.737632] systemd[1]: niks3.service: Deactivated successfully.5622lb # [ 105.740083] systemd[1]: Stopped niks3 server.5623lb # [ 105.740753] systemd[1]: niks3.service: Consumed 736ms CPU time over 1min 28.042s wall clock time, 28.6M memory peak, 97.5K incoming IP traffic, 139.7K outgoing IP traffic.5624lb # [ 105.747291] systemd[1]: Starting niks3 server...5625lb # [ 105.796145] niks3-server[1142]: 2026/09/23 16:13:54 goose: no migrations to run. current version: 202609231200005626lb # [ 105.801282] niks3-server[1142]: 2026/09/23 16:13:54 OK 1_commit_pending_closure.sql (5.89ms)5627lb # [ 105.804966] niks3-server[1142]: 2026/09/23 16:13:54 OK 2_object_stats_trigger.sql (3.64ms)5628lb # [ 105.808541] niks3-server[1142]: 2026/09/23 16:13:54 OK 3_commit_push.sql (3.22ms)5629lb # [ 105.809555] niks3-server[1142]: 2026/09/23 16:13:54 goose: up to current file version: 35630lb # [ 105.810843] niks3-server[1142]: 2026/09/23 16:13:54 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5631lb # [ 105.821519] niks3-server[1142]: 2026/09/23 16:13:54 INFO Read proxy enabled — serving cache objects from S35632lb # [ 105.823915] niks3-server[1142]: 2026/09/23 16:13:54 INFO Using socket-activated listener address=0.0.0.0:57515633lb # [ 105.825457] niks3-server[1142]: 2026/09/23 16:13:54 INFO systemd watchdog enabled interval=15s5634lb # [ 105.827236] systemd[1]: Started niks3 server.5635lb # [ 105.828627] niks3-server[1142]: 2026/09/23 16:13:54 INFO Starting HTTP server address=0.0.0.0:57515636lb: waiting for unit niks3.service5637lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5638lb: must succeed: curl -sf localhost:9901/clusters5639lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5640worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5641worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5642lb: must succeed: curl -sf localhost:9901/clusters5643lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5644lb: must succeed: curl -sf localhost:9901/clusters5645lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5646worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5647worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5648client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25649worker2 # [ 106.179270] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362325650client # don't know how to build these paths:5651client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5652worker2 # [ 106.188877] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362325653worker2 # [ 106.213723] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05654worker2 # [ 106.220246] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15655worker2 # [ 106.224741] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325656worker2 # [ 106.235990] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362325657client # copying 1 paths to 'lb:50051'...5658client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5659worker2 # [ 106.248280] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362325660worker2 # [ 106.259752] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5661worker2 # [ 106.354400] nix-daemon[860]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5662worker2 # [ 106.367725] nix-daemon[860]: reaped child process 1369, status = succeeded5663lb # [ 106.350317] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received push request method=POST path=/api/pushes5664worker2 # [ 106.386094] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.559Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5665worker2 # [ 106.388246] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.561Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5666lb # [ 106.392689] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5667lb # [ 106.404417] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5668lb # [ 106.419084] niks3-server[1142]: 2026/09/23 16:13:54 INFO Signed narinfos id=35 count=15669worker2 # [ 106.444283] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.617Z level=INFO msg="Uploading 1 narinfos"5670lb # [ 106.424640] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5671lb # [ 106.429916] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received complete push request method=POST path=/api/pushes/35/complete5672worker2 # [ 106.460698] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.634Z level=INFO msg="Upload complete. (191ms)"5673lb # [ 106.441122] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5674worker2 # [ 106.465888] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=5365675worker2 # [ 106.471494] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325676worker2 # [ 106.482486] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5677worker2 # [ 106.491933] nix-daemon[860]: reaped child process 1364, status = succeeded5678worker2 # [ 106.496990] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5679client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5680client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5681lb # [ 106.730985] niks3-server[1142]: 2026/09/23 16:13:55 INFO lead: acquired remote=192.168.1.4:567385682lb # [ 106.873174] envoy[921]: conn peer=[2001:db8:1::4]:45820 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3164 tx=1089 ms=649855683lb # [ 106.876919] envoy[921]: conn peer=[2001:db8:1::1]:60082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=135684lb # [ 106.880365] envoy[921]: conn peer=192.168.1.3:39430 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2024 tx=886 ms=673605685lb # [ 106.883898] envoy[921]: conn peer=[2001:db8:1::1]:60090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=305 ms=15825686client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5687client # farm-a> LOG-farm-a-after-niks3-restart5688worker2 # [ 107.933945] nix-daemon[860]: reaped child process 1373, status = succeeded5689worker2 # [ 108.019086] nix-daemon[860]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5690worker2 # [ 108.031571] nix-daemon[860]: reaped child process 1394, status = succeeded5691lb # [ 108.014527] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received push request method=POST path=/api/pushes5692worker2 # [ 108.050244] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5693worker2 # [ 108.053252] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.225Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5694lb # [ 108.057209] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5695lb # [ 108.066168] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5696lb # [ 108.075367] niks3-server[1142]: 2026/09/23 16:13:56 INFO Signed narinfos id=36 count=15697worker2 # [ 108.104856] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.278Z level=INFO msg="Uploading 1 narinfos"5698lb # [ 108.087432] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5699lb # [ 108.094782] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received complete push request method=POST path=/api/pushes/36/complete5700lb # [ 108.102142] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5701lb # [ 108.105505] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5702worker2 # [ 108.132204] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.305Z level=INFO msg="Upload complete. (227ms)"5703worker2 # [ 108.148153] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15704worker2 # [ 108.155853] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05705worker2 # [ 108.159766] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15706worker2 # [ 108.164244] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325707worker2 # [ 108.173517] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362325708worker2 # [ 108.177351] nix-daemon[860]: reaped child process 1371, status = succeeded5709worker2 # [ 108.182450] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.356Z level=INFO msg="All 1 paths already cached"5710client # copying 1 paths to 'lb:50051'...5711worker2 # [ 108.188128] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362325712client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5713worker2 # [ 108.193100] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5714worker2 # [ 108.276142] nix-daemon[860]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5715worker2 # [ 108.288263] nix-daemon[860]: reaped child process 1403, status = succeeded5716lb # [ 108.272868] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received push request method=POST path=/api/pushes5717worker2 # [ 108.303861] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5718worker2 # [ 108.305854] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.479Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5719lb # [ 108.302527] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5720worker2 # [ 108.331327] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.504Z level=INFO msg="Uploading 1 narinfos"5721lb # [ 108.308077] niks3-server[1142]: 2026/09/23 16:13:56 INFO Signed narinfos id=37 count=15722lb # [ 108.317255] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5723lb # [ 108.329122] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5724lb # [ 108.333121] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received complete push request method=POST path=/api/pushes/37/complete5725worker2 # [ 108.365193] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.538Z level=INFO msg="Upload complete. (166ms)"5726lb # [ 108.346658] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5727worker2 # [ 108.369342] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=6565728worker2 # [ 108.374311] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325729worker2 # [ 108.385682] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5730worker2 # [ 108.388278] nix-daemon[860]: reaped child process 1398, status = succeeded5731worker2 # [ 108.393665] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5732client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5733client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5734client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5735worker2 # [ 109.411263] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:511245736client # farm-b> LOG-farm-b-after-niks3-restart5737worker2 # [ 109.608854] nix-daemon[860]: reaped child process 1407, status = succeeded5738worker2 # [ 109.682451] nix-daemon[860]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5739worker2 # [ 109.694832] nix-daemon[860]: reaped child process 1426, status = succeeded5740lb # [ 109.679341] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes5741worker2 # [ 109.711076] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5742worker2 # [ 109.712470] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.886Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5743lb # [ 109.714612] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5744lb # [ 109.729269] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5745lb # [ 109.736051] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5746lb # [ 109.739194] niks3-server[1142]: 2026/09/23 16:13:58 INFO Signed narinfos id=38 count=15747worker2 # [ 109.765367] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.938Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 109.749885] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5749lb # [ 109.753700] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/38/complete5750lb # [ 109.761914] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5751worker2 # [ 109.783771] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.957Z level=INFO msg="Upload complete. (181ms)"5752worker2 # [ 109.790146] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15753worker2 # [ 109.800717] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05754worker2 # [ 109.805581] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15755worker2 # [ 109.810505] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325756worker2 # [ 109.820282] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362325757worker2 # [ 109.823233] nix-daemon[860]: reaped child process 1405, status = succeeded5758worker2 # [ 109.829062] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.002Z level=INFO msg="All 2 paths already cached"5759client # copying 1 paths to 'lb:50051'...5760worker2 # [ 109.834642] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362325761client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5762worker2 # [ 109.839717] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5763worker2 # [ 109.922242] nix-daemon[860]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5764worker2 # [ 109.936477] nix-daemon[860]: reaped child process 1435, status = succeeded5765lb # [ 109.917531] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes5766worker2 # [ 109.949947] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.123Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5767worker2 # [ 109.951887] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.125Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5768lb # [ 109.946890] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5769lb # [ 109.954614] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5770lb # [ 109.959984] niks3-server[1142]: 2026/09/23 16:13:58 INFO Signed narinfos id=39 count=15771worker2 # [ 109.985716] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.159Z level=INFO msg="Uploading 1 narinfos"5772lb # [ 109.966212] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5773lb # [ 109.975913] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/39/complete5774lb # [ 109.981584] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5775worker2 # [ 110.008427] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.182Z level=INFO msg="Upload complete. (163ms)"5776worker2 # [ 110.012591] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=7845777worker2 # [ 110.017407] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362325778worker2 # [ 110.028067] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5779worker2 # [ 110.032459] nix-daemon[860]: reaped child process 1430, status = succeeded5780worker2 # [ 110.036657] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted)5781client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5782client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5783client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5784client # farm-top> LOG-farm-top-after-niks3-restart5785worker2 # [ 110.850656] nix-daemon[860]: reaped child process 1439, status = succeeded5786worker2 # [ 110.926619] nix-daemon[860]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5787worker2 # [ 110.938629] nix-daemon[860]: reaped child process 1458, status = succeeded5788lb # [ 110.922527] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received push request method=POST path=/api/pushes5789worker2 # [ 110.953987] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.127Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5790worker2 # [ 110.956359] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.129Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5791lb # [ 110.966346] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5792lb # [ 110.970272] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5793worker2 # [ 111.000397] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.173Z level=INFO msg="Uploading 1 narinfos"5794lb # [ 110.977043] niks3-server[1142]: 2026/09/23 16:13:59 INFO Signed narinfos id=40 count=15795lb # [ 110.983996] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5796lb # [ 110.987990] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5797lb # [ 110.997568] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received complete push request method=POST path=/api/pushes/40/complete5798lb # [ 111.002115] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5799worker2 # [ 111.024376] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.197Z level=INFO msg="Upload complete. (181ms)"5800worker2 # [ 111.031258] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15801worker2 # [ 111.043871] nix-daemon[860]: reaped child process 1437, status = succeeded5802client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.04 seconds)5803client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5804client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)5805(finished: subtest: a niks3 restart does not cost the leader its role, in 5.52 seconds)5806subtest: scheduler down: the other node takes the lock and keeps it5807lb: must succeed: curl -sf localhost:9901/clusters5808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5809lb: must succeed: curl -sf localhost:9901/clusters5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5811lb: must succeed: curl -sf localhost:9901/clusters5812lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5813lb: must succeed: curl -sf localhost:9901/clusters5814lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5815worker2 # [ 111.348113] systemd[1]: Stopping Nix worker-protocol over gRPC...5816worker2 # [ 112.024623] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:01Z level=info event=signal_exit5817lb # [ 112.735776] niks3-server[1142]: 2026/09/23 16:14:01 INFO lead: released remote=192.168.1.4:567385818worker2 # [ 113.758827] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=scheduler_restarting5819lb # [ 113.735687] envoy[921]: [2026-09-23 16:14:02.134][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5820worker1 # [ 113.747792] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:01Z level=info event=scheduler_restarting addr=lb:500515821worker2 # [ 113.766963] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:511205822worker2 # [ 113.772264] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_gone addr=192.168.1.3:500515823worker2 # [ 113.775447] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=scheduler_restarting addr=lb:500515824worker2 # [ 113.780408] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:511205825worker2 # [ 113.784253] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_gone addr=192.168.1.4:500515826worker2 # [ 113.804397] nix-daemon[860]: reaped child process 1326, status = succeeded5827worker1 # [ 113.948088] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:500515828worker1 # [ 113.952353] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5829worker2 # [ 113.981335] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5830worker1 # [ 114.153160] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:500515831worker1 # [ 114.157197] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5832worker1 # [ 114.358139] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:500515833worker1 # [ 114.362161] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5834worker1 # [ 114.562952] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:500515835worker1 # [ 114.566959] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker1 # [ 114.768493] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:500515837worker1 # [ 114.772273] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker1 # [ 114.973123] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:500515839worker1 # [ 114.977129] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5840worker1 # [ 115.177944] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:500515841worker1 # [ 115.181985] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker1 # [ 115.382871] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:500515843worker1 # [ 115.386846] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5844worker1 # [ 115.587660] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:500515845worker1 # [ 115.591601] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846worker1 # [ 115.793590] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:500515847worker1 # [ 115.797186] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5848worker2 # [ 115.971356] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5849worker2 # [ 115.975364] systemd[1]: Stopped Nix worker-protocol over gRPC.5850worker2 # [ 115.979454] systemd[1]: nix-grpc-daemon.service: Consumed 897ms CPU time over 17.048s wall clock time, 38.5M memory peak, 201.3K incoming IP traffic, 128.2K outgoing IP traffic.5851worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5852worker1 # [ 115.998436] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:500515853worker1 # [ 116.002507] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5854worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5855worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5856worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5857worker1 # [ 116.203233] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:500515858worker1 # [ 116.207080] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5859worker1 # [ 116.408068] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:500515860worker1 # [ 116.412086] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5861worker1 # [ 116.612885] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:500515862worker1 # [ 116.616856] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5863worker1 # [ 116.734829] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_take_over5864lb # [ 116.731382] niks3-server[1142]: 2026/09/23 16:14:05 INFO lead: acquired remote=192.168.1.3:422865865worker1 # [ 116.818447] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:500515866worker1 # [ 116.822308] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5867lb # [ 116.873732] envoy[921]: conn peer=[2001:db8:1::1]:53200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9662 tx=4392 ms=48715868lb # [ 116.877753] envoy[921]: conn peer=[2001:db8:1::4]:59242 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=15028 tx=1063 ms=149095869worker1 # [ 117.023263] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:500515870worker1 # [ 117.027190] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5871worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5872worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5873worker1 # [ 117.227847] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:500515874worker1 # [ 117.239602] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:459625875worker1 # [ 117.243263] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05876worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5877worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5878client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25879worker1 # [ 118.296596] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:459725880worker1 # [ 118.307783] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5881client # don't know how to build these paths:5882client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5883worker1 # [ 118.322117] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459725884worker1 # [ 118.332264] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)5885worker1 # [ 118.412403] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 2/5)5886worker1 # [ 118.609299] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 281 ms (attempt 3/5)5887worker1 # [ 118.683569] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:459865888worker1 # [ 118.892302] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 4/5)5889worker1 # [ 118.956090] nix-grpc-daemon[1231]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5890worker1 # [ 118.959324] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5891worker1 # [ 118.986839] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05892worker1 # [ 118.992272] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15893worker1 # [ 118.998375] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459725894worker1 # [ 119.010103] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459725895client # copying 1 paths to 'lb:50051'...5896client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5897worker1 # [ 119.022427] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459725898worker1 # [ 119.030559] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5899worker1 # [ 119.050855] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=info event=niks3_push_started pid=13105900worker1 # [ 119.148259] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.395Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5901worker1 # [ 119.233624] nix-daemon[868]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5902worker1 # [ 119.245386] nix-daemon[868]: reaped child process 1326, status = succeeded5903lb # [ 119.249180] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes5904worker1 # [ 119.260992] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5905worker1 # [ 119.262490] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.509Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5906lb # [ 119.284158] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5907lb # [ 119.290543] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5908worker1 # [ 119.300431] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.547Z level=INFO msg="Uploading 1 narinfos"5909lb # [ 119.295698] niks3-server[1142]: 2026/09/23 16:14:07 INFO Signed narinfos id=41 count=15910lb # [ 119.309576] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5911lb # [ 119.317221] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/41/complete5912worker1 # [ 119.330384] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.577Z level=INFO msg="Upload complete. (178ms)"5913lb # [ 119.327871] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5914worker1 # [ 119.332928] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=5205915worker1 # [ 119.336520] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459725916worker1 # [ 119.343236] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5917worker1 # [ 119.345539] nix-daemon[868]: reaped child process 1308, status = succeeded5918worker1 # [ 119.350429] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5919client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5920client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5921client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5922client # farm-a> LOG-farm-a-on-standby5923worker1 # [ 120.987529] nix-daemon[868]: reaped child process 1330, status = succeeded5924worker1 # [ 121.068964] nix-daemon[868]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5925worker1 # [ 121.081343] nix-daemon[868]: reaped child process 1349, status = succeeded5926lb # [ 121.084528] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes5927worker1 # [ 121.101927] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5928worker1 # [ 121.103509] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.350Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5929lb # [ 121.128396] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5930lb # [ 121.134086] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5931lb # [ 121.140099] niks3-server[1142]: 2026/09/23 16:14:09 INFO Signed narinfos id=42 count=15932worker1 # [ 121.146712] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.393Z level=INFO msg="Uploading 1 narinfos"5933lb # [ 121.147549] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5934lb # [ 121.150392] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5935lb # [ 121.154278] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/42/complete5936lb # [ 121.161788] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5937worker1 # [ 121.170824] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.418Z level=INFO msg="Upload complete. (212ms)"5938worker1 # [ 121.176992] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15939worker1 # [ 121.189505] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05940worker1 # [ 121.192988] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15941worker1 # [ 121.197202] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459725942worker1 # [ 121.205291] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459725943worker1 # [ 121.208098] nix-daemon[868]: reaped child process 1328, status = succeeded5944worker1 # [ 121.212286] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.459Z level=INFO msg="All 1 paths already cached"5945client # copying 1 paths to 'lb:50051'...5946worker1 # [ 121.217344] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459725947client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5948worker1 # [ 121.222142] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5949worker1 # [ 121.304803] nix-daemon[868]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5950worker1 # [ 121.317607] nix-daemon[868]: reaped child process 1358, status = succeeded5951lb # [ 121.320059] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes5952worker1 # [ 121.333388] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5953worker1 # [ 121.335429] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.582Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5954lb # [ 121.352456] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5955lb # [ 121.357237] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5956lb # [ 121.361418] niks3-server[1142]: 2026/09/23 16:14:09 INFO Signed narinfos id=43 count=15957worker1 # [ 121.369824] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.617Z level=INFO msg="Uploading 1 narinfos"5958lb # [ 121.370286] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5959lb # [ 121.377335] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/43/complete5960worker1 # [ 121.386610] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.634Z level=INFO msg="Upload complete. (159ms)"5961lb # [ 121.383532] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5962worker1 # [ 121.388572] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=6405963worker1 # [ 121.391834] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459725964worker1 # [ 121.400074] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5965worker1 # [ 121.401197] nix-daemon[868]: reaped child process 1353, status = succeeded5966worker1 # [ 121.406168] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5967client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5968client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5969client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5970client # farm-b> LOG-farm-b-on-standby5971worker1 # [ 122.234187] nix-daemon[868]: reaped child process 1362, status = succeeded5972worker1 # [ 122.308945] nix-daemon[868]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5973worker1 # [ 122.321278] nix-daemon[868]: reaped child process 1381, status = succeeded5974lb # [ 122.324986] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received push request method=POST path=/api/pushes5975worker1 # [ 122.337188] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.584Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5976worker1 # [ 122.338678] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.586Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5977lb # [ 122.362738] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5978worker1 # [ 122.373448] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.620Z level=INFO msg="Uploading 1 narinfos"5979lb # [ 122.369299] niks3-server[1142]: 2026/09/23 16:14:10 INFO Signed narinfos id=44 count=15980lb # [ 122.381942] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5981lb # [ 122.387891] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5982lb # [ 122.393995] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5983lb # [ 122.402265] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received complete push request method=POST path=/api/pushes/44/complete5984lb # [ 122.409941] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5985worker1 # [ 122.412760] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.660Z level=INFO msg="Upload complete. (200ms)"5986worker1 # [ 122.423266] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15987worker1 # [ 122.431488] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05988worker1 # [ 122.436323] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15989worker1 # [ 122.439828] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459725990worker1 # [ 122.449554] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459725991worker1 # [ 122.452767] nix-daemon[868]: reaped child process 1360, status = succeeded5992worker1 # [ 122.457356] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.704Z level=INFO msg="All 2 paths already cached"5993client # copying 1 paths to 'lb:50051'...5994worker1 # [ 122.462881] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459725995client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5996worker1 # [ 122.467860] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5997worker1 # [ 122.551852] nix-daemon[868]: accepted connection from pid 1387, user nix-grpc-daemon (trusted)5998worker1 # [ 122.564204] nix-daemon[868]: reaped child process 1390, status = succeeded5999lb # [ 122.567437] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received push request method=POST path=/api/pushes6000worker1 # [ 122.580239] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.827Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6001worker1 # [ 122.582218] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.829Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6002lb # [ 122.602226] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6003lb # [ 122.606497] niks3-server[1142]: 2026/09/23 16:14:11 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign6004lb # [ 122.610436] niks3-server[1142]: 2026/09/23 16:14:11 INFO Signed narinfos id=45 count=16005worker1 # [ 122.619640] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.866Z level=INFO msg="Uploading 1 narinfos"6006lb # [ 122.619874] niks3-server[1142]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6007lb # [ 122.631398] niks3-server[1142]: 2026/09/23 16:14:11 INFO Received complete push request method=POST path=/api/pushes/45/complete6008lb # [ 122.637574] niks3-server[1142]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6009worker1 # [ 122.644943] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.892Z level=INFO msg="Upload complete. (171ms)"6010worker1 # [ 122.648143] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=7686011worker1 # [ 122.652938] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459726012worker1 # [ 122.664107] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)6013worker1 # [ 122.667764] nix-daemon[868]: reaped child process 1385, status = succeeded6014worker1 # [ 122.672958] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)6015client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6016client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6017client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6018client # farm-top> LOG-farm-top-on-standby6019worker1 # [ 124.113740] nix-daemon[868]: reaped child process 1394, status = succeeded6020worker1 # [ 124.194869] nix-daemon[868]: accepted connection from pid 1410, user nix-grpc-daemon (trusted)6021worker1 # [ 124.206145] nix-daemon[868]: reaped child process 1413, status = succeeded6022lb # [ 124.209752] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received push request method=POST path=/api/pushes6023worker1 # [ 124.222444] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6024worker1 # [ 124.223900] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.471Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6025lb # [ 124.250517] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6026lb # [ 124.256507] niks3-server[1142]: 2026/09/23 16:14:12 INFO Signed narinfos id=46 count=16027worker1 # [ 124.263782] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.510Z level=INFO msg="Uploading 1 narinfos"6028lb # [ 124.264793] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6029lb # [ 124.271677] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6030lb # [ 124.277126] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6031lb # [ 124.286284] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received complete push request method=POST path=/api/pushes/46/complete6032lb # [ 124.295288] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6033worker1 # [ 124.303077] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.550Z level=INFO msg="Upload complete. (221ms)"6034worker1 # [ 124.310106] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16035worker1 # [ 124.325720] nix-daemon[868]: reaped child process 1392, status = succeeded6036client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 6.20 seconds)6037client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6038client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6039worker2 # [ 124.539429] systemd[1]: Starting scheduler-token.service...6040worker2 # [ 124.581755] systemd[1]: scheduler-token.service: Deactivated successfully.6041worker2 # [ 124.583359] systemd[1]: Finished scheduler-token.service.6042worker2 # [ 124.584632] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 46ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6043worker2 # [ 124.589296] systemd[1]: Starting Nix worker-protocol over gRPC...6044worker2 # [ 124.664241] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6045worker2 # [ 124.669063] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6046worker2 # [ 124.672197] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16047worker2 # [ 124.674221] nix-grpc-daemon[1480]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6048worker2 # [ 124.675506] nix-grpc-daemon[1480]: E0000 00:00:1790180053.845039 1480 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6049worker2 # [ 124.678761] nix-grpc-daemon[1480]: E0000 00:00:1790180053.852426 1480 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6050worker2 # [ 124.686679] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6051worker2 # [ 124.688574] systemd[1]: Started Nix worker-protocol over gRPC.6052worker2 # [ 124.693085] nix-grpc-daemon[1480]: E0000 00:00:1790180053.864570 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6053lb: must succeed: curl -sf localhost:9901/clusters6054worker2 # [ 124.703909] nix-grpc-daemon[1480]: E0000 00:00:1790180053.877645 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6055worker2 # [ 124.707141] nix-grpc-daemon[1480]: E0000 00:00:1790180053.879888 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6056worker2 # [ 124.718050] nix-grpc-daemon[1480]: E0000 00:00:1790180053.891740 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6057worker2 # [ 124.728484] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=scheduler_connected addr=lb:500516058worker1 # [ 124.717169] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=oidc_jwks provider=mock keys=16059worker1 # [ 124.721292] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:459626060worker1 # [ 124.728421] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06061lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6062worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6063worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6064lb: must succeed: curl -sf localhost:9901/clusters6065lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6066lb: must succeed: curl -sf localhost:9901/clusters6067lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6068worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6069worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6070(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.71 seconds)6071subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6072client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6073client # [ 124.948942] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6074client # Running as unit: rs.service; invocation ID: 3d430631a43040c9923b371fb8fb26256075client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6076worker2 # [ 125.090943] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:421306077worker2 # [ 125.102531] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)6078client # [ 125.122833] nix[1363]: don't know how to build these paths:6079client # [ 125.124279] nix[1363]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6080worker1 # [ 125.109637] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06081worker1 # [ 125.115229] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16082worker1 # [ 125.121508] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459726083worker1 # [ 125.132641] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:459726084client # [ 125.165157] nix[1363]: copying 1 paths to 'lb:50051'...6085client # [ 125.168180] nix[1363]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6086worker1 # [ 125.146422] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:459726087worker1 # [ 125.157624] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)6088worker1 # [ 125.253826] nix-daemon[868]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6089worker1 # [ 125.266236] nix-daemon[868]: reaped child process 1429, status = succeeded6090lb # [ 125.269595] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes6091worker1 # [ 125.282184] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6092worker1 # [ 125.283552] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.529Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6093lb # [ 125.299229] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6094lb # [ 125.308258] niks3-server[1142]: 2026/09/23 16:14:13 INFO Signed narinfos id=47 count=16095worker1 # [ 125.315626] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.562Z level=INFO msg="Uploading 1 narinfos"6096lb # [ 125.317676] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6097lb # [ 125.329694] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/47/complete6098worker1 # [ 125.344351] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.591Z level=INFO msg="Upload complete. (175ms)"6099lb # [ 125.344916] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6100lb # [ 125.347582] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6101worker1 # [ 125.349786] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=4166102worker1 # [ 125.357312] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:459726103worker1 # [ 125.366142] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)6104worker1 # [ 125.374830] nix-daemon[868]: reaped child process 1424, status = succeeded6105worker1 # [ 125.379431] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)6106client # [ 125.401662] nix[1363]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6107lb # [ 126.874146] envoy[921]: conn peer=[2001:db8:1::1]:49598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4373 ms=60316108lb: must succeed: curl -sf localhost:9901/clusters6109lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6110lb: must succeed: curl -sf localhost:9901/clusters6111lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6112lb: must succeed: curl -sf localhost:9901/clusters6113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6114lb: must succeed: curl -sf localhost:9901/clusters6115lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6116worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6117worker2 # [ 127.611139] systemd-journald[282]: Received client request to rotate journal, rotating.6118worker2 # [ 127.632097] systemd-journald[282]: Vacuuming done, freed 0B of archived journals from /var/log/journal/bd1869688d4b467bbe7e20bf3feec49a.6119worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6120worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6121worker1 # [ 127.518248] systemd[1]: Stopping Nix worker-protocol over gRPC...6122worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6123client: must succeed: sleep 5.06124worker1 # [ 127.538163] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:15Z level=info event=draining6125client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6126worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6127worker1 # [ 132.820313] nix-daemon[868]: reaped child process 1433, status = succeeded6128worker1 # [ 132.897905] nix-daemon[868]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)6129worker1 # [ 132.910453] nix-daemon[868]: reaped child process 1477, status = succeeded6130lb # [ 132.913591] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received push request method=POST path=/api/pushes6131worker1 # [ 132.929776] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6132worker1 # [ 132.931335] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.178Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6133lb # [ 132.960346] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6134lb # [ 132.966718] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6135worker1 # [ 132.979880] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.227Z level=INFO msg="Uploading 1 narinfos"6136lb # [ 132.976114] niks3-server[1142]: 2026/09/23 16:14:21 INFO Signed narinfos id=48 count=16137lb # [ 132.980955] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6138lb # [ 132.986726] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6139lb # [ 132.996774] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received complete push request method=POST path=/api/pushes/48/complete6140worker1 # [ 133.006058] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.253Z level=INFO msg="Upload complete. (212ms)"6141lb # [ 133.003931] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6142worker1 # [ 133.013386] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16143worker1 # [ 133.028597] nix-daemon[868]: reaped child process 1431, status = succeeded6144client # [ 133.074558] systemd[1]: rs.service: Deactivated successfully.6145client # [ 133.079287] systemd[1]: rs.service: Consumed 140ms CPU time over 8.128s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.6146worker1 # [ 133.540199] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:21Z level=info event=signal_exit6147lb # [ 133.734164] niks3-server[1142]: 2026/09/23 16:14:22 INFO lead: released remote=192.168.1.3:422866148lb # [ 134.665775] niks3-server[1142]: 2026/09/23 16:14:23 INFO lead: acquired remote=192.168.1.4:571366149worker2 # [ 134.689179] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:23Z level=info event=scheduler_take_over6150worker1 # [ 134.740542] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=scheduler_restarting6151lb # [ 134.735480] envoy[921]: [2026-09-23 16:14:23.133][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6152worker1 # [ 134.746693] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=scheduler_restarting addr=lb:500516153worker2 # [ 134.766171] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:23Z level=info event=scheduler_restarting addr=lb:500516154worker1 # [ 134.753881] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:459626155worker1 # [ 134.757873] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_gone addr=192.168.1.3:500516156worker1 # [ 134.761289] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:459626157worker1 # [ 134.764987] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_gone addr=192.168.1.4:500516158worker1 # [ 134.784694] nix-daemon[868]: reaped child process 1300, status = succeeded6159worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6160client: waiting for success: ! systemctl is-active rs6161client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6162client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6163client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6164client: must fail: journalctl -u rs | grep -q 'reconnecting'6165client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6166lb: must succeed: curl -sf localhost:9901/clusters6167worker2 # [ 134.972308] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_connected addr=lb:500516168worker2 # [ 134.976319] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6169worker1 # [ 134.959841] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6171worker2 # [ 135.177168] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_connected addr=lb:500516172worker2 # [ 135.190155] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=oidc_jwks provider=mock keys=16173worker2 # [ 135.193897] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:528186174worker2 # [ 135.197713] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06175worker2 # [ 135.560269] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:528326176lb: must succeed: curl -sf localhost:9901/clusters6177lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6178worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6179worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6180worker1 # [ 136.812828] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6181worker1 # [ 136.816466] systemd[1]: Stopped Nix worker-protocol over gRPC.6182worker1 # [ 136.818554] systemd[1]: nix-grpc-daemon.service: Consumed 1.104s CPU time over 32.391s wall clock time, 37.7M memory peak, 332.8K incoming IP traffic, 146.9K outgoing IP traffic.6183worker1 # [ 136.834562] systemd[1]: Starting Nix worker-protocol over gRPC...6184lb # [ 136.874387] envoy[921]: conn peer=[2001:db8:1::1]:37894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1498 ms=79606185lb # [ 136.878220] envoy[921]: conn peer=[2001:db8:1::3]:57574 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4814 tx=1191 ms=303786186worker1 # [ 136.922321] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6187worker1 # [ 136.926573] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6188worker1 # [ 136.929866] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16189worker1 # [ 136.933578] nix-grpc-daemon[1485]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6190worker1 # [ 136.934906] nix-grpc-daemon[1485]: E0000 00:00:1790180065.179468 1485 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6191worker1 # [ 136.937089] nix-grpc-daemon[1485]: E0000 00:00:1790180065.180860 1485 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6192worker1 # [ 136.941540] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6193worker1 # [ 136.943533] systemd[1]: Started Nix worker-protocol over gRPC.6194worker1 # [ 136.947057] nix-grpc-daemon[1485]: E0000 00:00:1790180065.194450 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6195worker1 # [ 136.959138] nix-grpc-daemon[1485]: E0000 00:00:1790180065.206538 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6196worker1 # [ 136.963130] nix-grpc-daemon[1485]: E0000 00:00:1790180065.208695 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6197worker1 # [ 136.973051] nix-grpc-daemon[1485]: E0000 00:00:1790180065.220466 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6198worker1 # [ 136.982314] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=scheduler_connected addr=lb:500516199worker2 # [ 137.003845] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:528186200worker2 # [ 137.008442] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06201lb: must succeed: curl -sf localhost:9901/clusters6202lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6203worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6204worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6205(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.27 seconds)6206subtest: scheduler restart mid-build: build finishes, no double build6207client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6208client # [ 137.222986] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6209client # Running as unit: mid.service; invocation ID: 4a3ee47dfb484f2da4e28102b09d18706210client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.07 seconds)6211worker1 # [ 137.349846] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588186212worker1 # [ 137.360893] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6213client # [ 137.395446] nix[1399]: don't know how to build these paths:6214client # [ 137.396445] nix[1399]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6215worker2 # [ 137.400322] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06216worker2 # [ 137.406992] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16217worker2 # [ 137.412505] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421306218worker2 # [ 137.422236] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6219worker2 # [ 137.444684] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)6220worker2 # [ 137.461315] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 207 ms (attempt 3/5)6221worker2 # [ 137.670614] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 649 ms (attempt 4/5)6222worker2 # [ 138.322924] nix-grpc-daemon[1480]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6223worker2 # [ 138.326713] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6224worker2 # [ 138.344219] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421306225client # [ 138.358114] nix[1399]: copying 1 paths to 'lb:50051'...6226client # [ 138.362156] nix[1399]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6227worker2 # [ 138.359302] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421306228worker2 # [ 138.370297] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)6229worker2 # [ 138.393392] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=info event=niks3_push_started pid=16046230worker2 # [ 138.507099] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.679Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6231worker2 # [ 138.591648] nix-daemon[860]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6232worker2 # [ 138.603414] nix-daemon[860]: reaped child process 1621, status = succeeded6233lb # [ 138.587848] niks3-server[1142]: 2026/09/23 16:14:26 INFO Received push request method=POST path=/api/pushes6234worker2 # [ 138.622215] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6235worker2 # [ 138.623748] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.797Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6236lb # [ 138.625234] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6237lb # [ 138.631858] niks3-server[1142]: 2026/09/23 16:14:27 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6238lb # [ 138.636732] niks3-server[1142]: 2026/09/23 16:14:27 INFO Signed narinfos id=49 count=16239worker2 # [ 138.662727] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.836Z level=INFO msg="Uploading 1 narinfos"6240lb # [ 138.655121] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6241lb # [ 138.662119] niks3-server[1142]: 2026/09/23 16:14:27 INFO Received complete push request method=POST path=/api/pushes/49/complete6242lb # [ 138.669406] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6243worker2 # [ 138.697350] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.870Z level=INFO msg="Upload complete. (190ms)"6244worker2 # [ 138.702252] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42130 duration_s=0 paths=1 nar_bytes_in=4246245worker2 # [ 138.707274] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421306246worker2 # [ 138.716706] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)6247worker2 # [ 138.722559] nix-daemon[860]: reaped child process 1600, status = succeeded6248worker2 # [ 138.727428] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)6249client # [ 138.731393] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6250lb: must succeed: curl -sf localhost:9901/clusters6251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6252lb: must succeed: curl -sf localhost:9901/clusters6253lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6254worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6255worker2 # [ 139.768285] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1480 (nix-grpc-daemon) on client request.6256worker2 # [ 139.770332] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6257lb # [ 139.752858] niks3-server[1142]: 2026/09/23 16:14:28 INFO lead: released remote=192.168.1.4:571366258worker2 # [ 139.782756] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6259client # [ 139.785437] nix[1399]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6260worker2 # [ 139.786060] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6261worker1 # [ 139.762565] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6262worker2 # [ 139.787269] systemd[1]: nix-grpc-daemon.service: Consumed 382ms CPU time over 15.195s wall clock time, 36M memory peak, 214.2K incoming IP traffic, 53.2K outgoing IP traffic.6263lb # [ 139.760401] envoy[921]: [2026-09-23 16:14:28.158][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6264client # [ 139.790669] nix[1399]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6265worker2 # [ 139.797718] systemd[1]: Starting scheduler-token.service...6266worker2 # [ 139.813312] nix-daemon[860]: reaped child process 1623, status = succeeded6267worker2 # [ 139.815163] nix-daemon[860]: reaped child process 1539, status = succeeded6268worker2 # [ 139.824408] nix-daemon[1625]: unexpected Nix daemon error: error: interrupted by the user6269worker2 # [ 139.829922] nix-daemon[860]: reaped child process 1625, status = failed with exit code 16270worker2 # [ 139.848480] systemd[1]: scheduler-token.service: Deactivated successfully.6271worker2 # [ 139.849715] systemd[1]: Finished scheduler-token.service.6272worker2 # [ 139.851520] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 51ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6273worker2 # [ 139.853695] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6274worker2 # [ 139.856877] systemd[1]: Starting scheduler-token.service...6275worker2 # [ 139.889420] systemd[1]: scheduler-token.service: Deactivated successfully.6276worker2 # [ 139.890985] systemd[1]: Finished scheduler-token.service.6277worker2 # [ 139.893164] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 33ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6278worker2 # [ 139.896600] systemd[1]: Starting Nix worker-protocol over gRPC...6279worker2 # [ 139.971894] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6280worker2 # [ 139.973824] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6281worker2 # [ 139.976960] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16282worker2 # [ 139.981326] nix-grpc-daemon[1664]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6283worker2 # [ 139.982951] nix-grpc-daemon[1664]: E0000 00:00:1790180069.154921 1664 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6284worker1 # [ 139.963285] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 139.987088] nix-grpc-daemon[1664]: E0000 00:00:1790180069.160714 1664 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6286worker2 # [ 139.990897] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6287worker2 # [ 139.994382] systemd[1]: Started Nix worker-protocol over gRPC.6288lb # [ 139.976379] niks3-server[1142]: 2026/09/23 16:14:28 INFO lead: acquired remote=192.168.1.4:393786289worker2 # [ 140.000079] nix-grpc-daemon[1664]: E0000 00:00:1790180069.172296 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6290worker2 # [ 140.004242] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=scheduler_take_over6291worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.27 seconds)6292client: must succeed: sleep 5.06293worker2 # [ 140.011674] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:528726294worker2 # [ 140.014216] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06295worker2 # [ 140.021048] nix-grpc-daemon[1664]: E0000 00:00:1790180069.193709 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6296worker2 # [ 140.025167] nix-grpc-daemon[1664]: E0000 00:00:1790180069.193914 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6297worker2 # [ 140.034397] nix-grpc-daemon[1664]: E0000 00:00:1790180069.208093 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6298worker2 # [ 140.043768] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:500516299worker2 # [ 140.049247] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=oidc_jwks provider=mock keys=16300worker2 # [ 140.050614] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:528726301worker2 # [ 140.052247] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06302worker2 # [ 140.082037] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:528886303worker2 # [ 140.292184] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06304worker1 # [ 140.277081] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588186305worker2 # [ 140.297263] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16306worker1 # [ 140.289682] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)6307worker1 # [ 140.300652] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 2/5)6308worker1 # [ 140.372590] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 299 ms (attempt 3/5)6309worker1 # [ 140.673867] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 440 ms (attempt 4/5)6310worker1 # [ 141.116883] nix-grpc-daemon[1485]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6311worker1 # [ 141.121129] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6312worker1 # [ 141.138640] nix-grpc-daemon[1485]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6313worker1 # [ 141.163360] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6314worker1 # [ 141.177577] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6315client # [ 141.199391] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6316client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6317worker1 # [ 145.196512] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:33Z level=info event=niks3_push_started pid=15876318client: waiting for success: ! systemctl is-active mid6319worker1 # [ 145.216853] nix-daemon[868]: reaped child process 1562, status = succeeded6320worker1 # [ 145.296793] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.543Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6321worker1 # [ 145.381208] nix-daemon[868]: accepted connection from pid 1600, user nix-grpc-daemon (trusted)6322worker1 # [ 145.394395] nix-daemon[868]: reaped child process 1603, status = succeeded6323lb # [ 145.397509] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received push request method=POST path=/api/pushes6324worker1 # [ 145.409657] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6325worker1 # [ 145.411142] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.658Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6326lb # [ 145.439325] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6327lb # [ 145.452230] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6328worker1 # [ 145.465635] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.713Z level=INFO msg="Uploading 1 narinfos"6329lb # [ 145.461418] niks3-server[1142]: 2026/09/23 16:14:33 INFO Signed narinfos id=50 count=16330lb # [ 145.472160] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6331lb # [ 145.487178] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received complete push request method=POST path=/api/pushes/50/complete6332lb # [ 145.490351] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6333worker1 # [ 145.496908] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.744Z level=INFO msg="Upload complete. (196ms)"6334lb # [ 145.494803] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6335worker1 # [ 145.502682] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16336worker1 # [ 145.519895] nix-daemon[868]: reaped child process 1560, status = succeeded6337client # [ 145.558828] systemd[1]: mid.service: Deactivated successfully.6338client # [ 145.562114] systemd[1]: mid.service: Consumed 153ms CPU time over 8.335s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic.6339client: (finished: waiting for success: ! systemctl is-active mid, in 1.07 seconds)6340client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6341client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6342(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.15 seconds)6343subtest: interrupting the client stops the build on the worker6344client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6345client # [ 146.345346] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6346client # Running as unit: intr.service; invocation ID: eeb9ba1beaef4c0696d5a04d15e5ed056347client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6348worker2 # [ 146.480257] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367346349worker2 # [ 146.491368] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6350client # [ 146.507249] nix[1435]: don't know how to build these paths:6351client # [ 146.508294] nix[1435]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6352worker2 # [ 146.513745] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06353worker2 # [ 146.518650] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16354worker2 # [ 146.525469] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346355worker2 # [ 146.534932] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)6356worker2 # [ 146.579341] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 2/5)6357worker2 # [ 146.716503] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5)6358worker2 # [ 146.815927] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 762 ms (attempt 4/5)6359lb # [ 146.874851] envoy[921]: conn peer=[2001:db8:1::4]:43236 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2914 tx=393 ms=150576360lb # [ 146.878763] envoy[921]: conn peer=[2001:db8:1::1]:58866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3878 tx=1949 ms=81696361worker2 # [ 147.594841] nix-grpc-daemon[1664]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6362worker2 # [ 147.598579] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6363worker2 # [ 147.616088] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346364client # [ 147.629800] nix[1435]: copying 1 paths to 'lb:50051'...6365client # [ 147.633694] nix[1435]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6366worker2 # [ 147.630668] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346367worker2 # [ 147.642669] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6368worker2 # [ 147.662694] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=info event=niks3_push_started pid=17446369worker2 # [ 147.764461] nix-grpc-daemon[1744]: time=2026-09-23T16:14:36.937Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6370worker2 # [ 147.850124] nix-daemon[860]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6371worker2 # [ 147.862303] nix-daemon[860]: reaped child process 1760, status = succeeded6372lb # [ 147.847749] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received push request method=POST path=/api/pushes6373worker2 # [ 147.878941] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.052Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6374worker2 # [ 147.880430] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.054Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6375lb # [ 147.883927] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6376lb # [ 147.891205] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6377lb # [ 147.898599] niks3-server[1142]: 2026/09/23 16:14:36 INFO Signed narinfos id=51 count=16378worker2 # [ 147.924733] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.098Z level=INFO msg="Uploading 1 narinfos"6379lb # [ 147.908928] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6380lb # [ 147.917903] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received complete push request method=POST path=/api/pushes/51/complete6381lb # [ 147.927557] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6382worker2 # [ 147.954073] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.127Z level=INFO msg="Upload complete. (186ms)"6383worker2 # [ 147.958511] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=4246384worker2 # [ 147.963856] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346385worker2 # [ 147.972353] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6386worker2 # [ 147.982286] nix-daemon[860]: reaped child process 1742, status = succeeded6387worker2 # [ 147.985621] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6388client # [ 147.988631] nix[1435]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6389client: must succeed: systemctl kill -s INT intr6390client # [ 149.742865] systemd[1]: intr.service: Sent signal SIGINT to main process 1435 (nix) on client request.6391worker2 # [ 149.745969] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:38Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6392client # [ 149.751130] nix[1435]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6393client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6394client # [ 149.759609] nix[1435]: error: interrupted by the user6395client # [ 149.788234] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6396client # [ 149.789958] systemd[1]: intr.service: Failed with result 'exit-code'.6397client # [ 149.792090] systemd[1]: intr.service: Consumed 135ms CPU time over 3.442s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6398worker2 # [ 149.810733] nix-daemon[860]: reaped child process 1762, status = succeeded6399worker2 # [ 149.812455] nix-daemon[1764]: unexpected Nix daemon error: error: interrupted by the user6400worker2 # [ 149.820792] nix-daemon[860]: reaped child process 1764, status = failed with exit code 16401(finished: subtest: interrupting the client stops the build on the worker, in 3.51 seconds)6402subtest: a deploy mid-build drains: the build finishes, the new generation takes over6403client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6404client # [ 149.854254] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6405client # Running as unit: sw.service; invocation ID: 89227e95768146f79350c4c9c148a5f56406client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.03 seconds)6407worker1 # [ 149.962513] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588186408client # [ 149.986347] nix[1455]: don't know how to build these paths:6409client # [ 149.987333] nix[1455]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6410worker2 # [ 149.989870] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:39Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06411worker1 # [ 149.977378] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588186412worker2 # [ 149.994717] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:39Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16413worker1 # [ 149.985507] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588186414client # [ 150.014229] nix[1455]: copying 1 paths to 'lb:50051'...6415client # [ 150.015149] nix[1455]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6416worker1 # [ 149.992658] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588186417worker1 # [ 149.998858] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6418worker1 # [ 150.084070] nix-daemon[868]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6419worker1 # [ 150.097106] nix-daemon[868]: reaped child process 1634, status = succeeded6420lb # [ 150.099760] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received push request method=POST path=/api/pushes6421worker1 # [ 150.114182] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6422worker1 # [ 150.116474] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.363Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6423lb # [ 150.129176] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6424lb # [ 150.146111] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6425lb # [ 150.155407] niks3-server[1142]: 2026/09/23 16:14:38 INFO Signed narinfos id=52 count=16426worker1 # [ 150.162238] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.409Z level=INFO msg="Uploading 1 narinfos"6427lb # [ 150.158196] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6428lb # [ 150.170157] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received complete push request method=POST path=/api/pushes/52/complete6429worker1 # [ 150.178303] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.425Z level=INFO msg="Upload complete. (174ms)"6430lb # [ 150.175582] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6431worker1 # [ 150.182213] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=0 paths=1 nar_bytes_in=4166432worker1 # [ 150.187315] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588186433worker1 # [ 150.198138] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6434worker1 # [ 150.204144] nix-daemon[868]: reaped child process 1629, status = succeeded6435client # [ 150.229567] nix[1455]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6436worker1 # [ 150.207924] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)6437worker1: must succeed: systemctl reload nix-grpc-daemon.service6438worker1 # [ 152.167145] systemd[1]: Reloading Nix worker-protocol over gRPC...6439worker1 # [ 152.194703] systemd[1]: Reloaded Nix worker-protocol over gRPC.6440worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6441worker1: must succeed: pgrep -f 'read -t [9]0 x'6442worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6443client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26444client # don't know how to build these paths:6445client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6446worker2 # [ 152.383208] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367346447worker2 # [ 152.389085] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346448worker2 # [ 152.411629] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06449worker2 # [ 152.416995] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16450worker2 # [ 152.422391] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346451worker2 # [ 152.434150] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346452client # copying 1 paths to 'lb:50051'...6453client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6454worker2 # [ 152.445842] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346455worker2 # [ 152.458181] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6456worker2 # [ 152.551386] nix-daemon[860]: accepted connection from pid 1815, user nix-grpc-daemon (trusted)6457worker2 # [ 152.563932] nix-daemon[860]: reaped child process 1818, status = succeeded6458lb # [ 152.549990] niks3-server[1142]: 2026/09/23 16:14:40 INFO Received push request method=POST path=/api/pushes6459worker2 # [ 152.582445] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.755Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6460worker2 # [ 152.584028] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.757Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6461lb # [ 152.584906] niks3-server[1142]: 2026/09/23 16:14:40 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6462worker2 # [ 152.612064] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.785Z level=INFO msg="Uploading 1 narinfos"6463lb # [ 152.588841] niks3-server[1142]: 2026/09/23 16:14:40 INFO Signed narinfos id=53 count=16464lb # [ 152.597748] niks3-server[1142]: 2026/09/23 16:14:40 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6465lb # [ 152.605928] niks3-server[1142]: 2026/09/23 16:14:41 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6466lb # [ 152.609532] niks3-server[1142]: 2026/09/23 16:14:41 INFO Received complete push request method=POST path=/api/pushes/53/complete6467worker2 # [ 152.638483] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.812Z level=INFO msg="Upload complete. (169ms)"6468lb # [ 152.618247] niks3-server[1142]: 2026/09/23 16:14:41 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6469worker2 # [ 152.642444] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=5286470worker2 # [ 152.648122] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346471worker2 # [ 152.656555] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6472worker2 # [ 152.661874] nix-daemon[860]: reaped child process 1813, status = succeeded6473worker2 # [ 152.665495] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6474client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6475client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6476worker1 # [ 152.945632] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:41Z level=info event=draining6477client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6478client # farm-a> LOG-farm-a-during-drain6479worker2 # [ 153.899562] nix-daemon[860]: reaped child process 1822, status = succeeded6480worker2 # [ 153.986820] nix-daemon[860]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6481worker2 # [ 153.999578] nix-daemon[860]: reaped child process 1841, status = succeeded6482lb # [ 153.981684] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received push request method=POST path=/api/pushes6483worker2 # [ 154.015700] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6484worker2 # [ 154.017172] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.190Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6485lb # [ 154.023456] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6486lb # [ 154.031063] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6487lb # [ 154.037087] niks3-server[1142]: 2026/09/23 16:14:42 INFO Signed narinfos id=54 count=16488worker2 # [ 154.063064] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.236Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 154.044155] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6490lb # [ 154.048080] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6491lb # [ 154.057731] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received complete push request method=POST path=/api/pushes/54/complete6492lb # [ 154.063580] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6493worker2 # [ 154.089501] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.263Z level=INFO msg="Upload complete. (216ms)"6494worker2 # [ 154.096367] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16495worker2 # [ 154.107435] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06496worker2 # [ 154.112656] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16497worker2 # [ 154.116185] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346498worker2 # [ 154.126300] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346499worker2 # [ 154.129635] nix-daemon[860]: reaped child process 1820, status = succeeded6500worker2 # [ 154.134840] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.308Z level=INFO msg="All 1 paths already cached"6501client # copying 1 paths to 'lb:50051'...6502worker2 # [ 154.141463] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346503client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6504worker2 # [ 154.160845] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6505worker2 # [ 154.246048] nix-daemon[860]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6506worker2 # [ 154.258911] nix-daemon[860]: reaped child process 1850, status = succeeded6507lb # [ 154.241865] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received push request method=POST path=/api/pushes6508worker2 # [ 154.273736] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.447Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6509worker2 # [ 154.275200] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.448Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6510lb # [ 154.276276] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6511worker2 # [ 154.304191] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.477Z level=INFO msg="Uploading 1 narinfos"6512lb # [ 154.281548] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6513lb # [ 154.285169] niks3-server[1142]: 2026/09/23 16:14:42 INFO Signed narinfos id=55 count=16514lb # [ 154.299209] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received complete push request method=POST path=/api/pushes/55/complete6515worker2 # [ 154.332336] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.505Z level=INFO msg="Upload complete. (163ms)"6516lb # [ 154.310783] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6517worker2 # [ 154.335026] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=6406518lb # [ 154.315374] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6519worker2 # [ 154.338768] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346520worker2 # [ 154.345357] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6521worker2 # [ 154.347646] nix-daemon[860]: reaped child process 1845, status = succeeded6522worker2 # [ 154.352722] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6523client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6524client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6525client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6526client # farm-b> LOG-farm-b-during-drain6527worker2 # [ 155.977100] nix-daemon[860]: reaped child process 1854, status = succeeded6528worker2 # [ 156.047212] nix-daemon[860]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6529worker2 # [ 156.060570] nix-daemon[860]: reaped child process 1873, status = succeeded6530lb # [ 156.044657] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received push request method=POST path=/api/pushes6531worker2 # [ 156.076539] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.250Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6532worker2 # [ 156.078082] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.251Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6533lb # [ 156.083308] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6534worker2 # [ 156.112131] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.285Z level=INFO msg="Uploading 1 narinfos"6535lb # [ 156.090138] niks3-server[1142]: 2026/09/23 16:14:44 INFO Signed narinfos id=56 count=16536lb # [ 156.101520] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6537lb # [ 156.106401] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6538lb # [ 156.112136] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6539lb # [ 156.121258] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received complete push request method=POST path=/api/pushes/56/complete6540lb # [ 156.127672] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6541worker2 # [ 156.154608] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.328Z level=INFO msg="Upload complete. (193ms)"6542worker2 # [ 156.165238] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16543worker2 # [ 156.173923] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06544worker2 # [ 156.180218] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16545worker2 # [ 156.183827] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346546worker2 # [ 156.193281] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346547worker2 # [ 156.196569] nix-daemon[860]: reaped child process 1852, status = succeeded6548worker2 # [ 156.201932] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.375Z level=INFO msg="All 2 paths already cached"6549client # copying 1 paths to 'lb:50051'...6550worker2 # [ 156.207633] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346551client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6552worker2 # [ 156.223144] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6553worker2 # [ 156.307306] nix-daemon[860]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6554worker2 # [ 156.320139] nix-daemon[860]: reaped child process 1882, status = succeeded6555lb # [ 156.303963] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received push request method=POST path=/api/pushes6556worker2 # [ 156.335645] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.509Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6557worker2 # [ 156.337592] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.510Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6558lb # [ 156.333695] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6559worker2 # [ 156.365730] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.539Z level=INFO msg="Uploading 1 narinfos"6560lb # [ 156.342531] niks3-server[1142]: 2026/09/23 16:14:44 INFO Signed narinfos id=57 count=16561lb # [ 156.346557] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6562lb # [ 156.354422] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6563lb # [ 156.365128] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received complete push request method=POST path=/api/pushes/57/complete6564worker2 # [ 156.395992] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.569Z level=INFO msg="Upload complete. (167ms)"6565lb # [ 156.373879] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6566worker2 # [ 156.399422] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=7686567worker2 # [ 156.404245] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346568worker2 # [ 156.413673] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6569worker2 # [ 156.419255] nix-daemon[860]: reaped child process 1877, status = succeeded6570worker2 # [ 156.424156] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6571client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6572client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6573lb # [ 156.876335] envoy[921]: conn peer=[2001:db8:1::1]:54152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=32916574client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6575client # farm-top> LOG-farm-top-during-drain6576worker2 # [ 157.867293] nix-daemon[860]: reaped child process 1886, status = succeeded6577worker2 # [ 157.943479] nix-daemon[860]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6578worker2 # [ 157.956037] nix-daemon[860]: reaped child process 1905, status = succeeded6579lb # [ 157.939420] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received push request method=POST path=/api/pushes6580worker2 # [ 157.971838] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.145Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6581worker2 # [ 157.973287] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.147Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6582lb # [ 157.981192] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6583lb # [ 157.988972] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6584lb # [ 157.995358] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6585lb # [ 158.002478] niks3-server[1142]: 2026/09/23 16:14:46 INFO Signed narinfos id=58 count=16586worker2 # [ 158.028851] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.202Z level=INFO msg="Uploading 1 narinfos"6587lb # [ 158.012425] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6588lb # [ 158.019175] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received complete push request method=POST path=/api/pushes/58/complete6589lb # [ 158.028550] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6590worker2 # [ 158.050570] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.224Z level=INFO msg="Upload complete. (217ms)"6591worker2 # [ 158.059212] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16592worker2 # [ 158.072699] nix-daemon[860]: reaped child process 1884, status = succeeded6593client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.84 seconds)6594client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6595client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6596worker1: must succeed: pkill -f 'read -t [9]0 x'6597worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6598client: waiting for success: ! systemctl is-active sw6599worker1 # [ 158.429124] nix-daemon[868]: reaped child process 1638, status = succeeded6600worker1 # [ 158.503696] nix-daemon[868]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6601worker1 # [ 158.516887] nix-daemon[868]: reaped child process 1682, status = succeeded6602lb # [ 158.518679] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received push request method=POST path=/api/pushes6603worker1 # [ 158.531345] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6604worker1 # [ 158.532814] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.780Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6605lb # [ 158.558599] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6606lb # [ 158.563790] niks3-server[1142]: 2026/09/23 16:14:46 INFO Signed narinfos id=59 count=16607worker1 # [ 158.570991] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.818Z level=INFO msg="Uploading 1 narinfos"6608lb # [ 158.575506] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6609lb # [ 158.581112] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6610lb # [ 158.588956] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6611lb # [ 158.597206] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received complete push request method=POST path=/api/pushes/59/complete6612worker1 # [ 158.605774] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.853Z level=INFO msg="Upload complete. (183ms)"6613lb # [ 158.603221] niks3-server[1142]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6614worker1 # [ 158.608454] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16615worker1 # [ 158.615984] nix-daemon[868]: reaped child process 1636, status = succeeded6616client # [ 158.664227] systemd[1]: sw.service: Deactivated successfully.6617client # [ 158.667824] systemd[1]: sw.service: Consumed 129ms CPU time over 8.812s wall clock time, 41.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.6618worker1 # [ 158.949287] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:47Z level=info event=signal_exit6619client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6620client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6621client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6622worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6623worker1 # [ 159.951848] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:48Z level=info event=scheduler_restarting6624worker1 # [ 159.960671] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6625worker2 # [ 159.975846] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:528726626worker2 # [ 159.984070] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:49Z level=info event=worker_gone addr=192.168.1.3:500516627worker1 # [ 159.971442] nix-daemon[868]: reaped child process 1541, status = succeeded6628worker1 # [ 160.963988] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6629worker1 # [ 160.965726] systemd[1]: nix-grpc-daemon.service: Consumed 529ms CPU time over 24.130s wall clock time, 34.7M memory peak, 47.7K incoming IP traffic, 54.4K outgoing IP traffic.6630worker1 # [ 160.970720] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6631worker1 # [ 160.974239] systemd[1]: Starting Nix worker-protocol over gRPC...6632worker1 # [ 161.050625] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6633worker1 # [ 161.054900] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6634worker1 # [ 161.058121] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16635worker1 # [ 161.061592] nix-grpc-daemon[1697]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6636worker1 # [ 161.062943] nix-grpc-daemon[1697]: E0000 00:00:1790180089.305055 1697 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6637worker1 # [ 161.065052] nix-grpc-daemon[1697]: E0000 00:00:1790180089.308779 1697 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6638worker1 # [ 161.072647] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6639worker1 # [ 161.076081] systemd[1]: Started Nix worker-protocol over gRPC.6640worker1 # [ 161.076922] nix-grpc-daemon[1697]: E0000 00:00:1790180089.322895 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6641worker1 # [ 161.092066] nix-grpc-daemon[1697]: E0000 00:00:1790180089.336971 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6642worker1 # [ 161.094218] nix-grpc-daemon[1697]: E0000 00:00:1790180089.337078 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6643worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.78 seconds)6644lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6645lb: must succeed: curl -sf localhost:9901/clusters6646worker1 # [ 161.104650] nix-grpc-daemon[1697]: E0000 00:00:1790180089.352059 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6647worker1 # [ 161.116867] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=scheduler_connected addr=lb:500516648worker2 # [ 161.140749] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:528726649worker2 # [ 161.147594] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06650lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6651lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.05 seconds)6652lb: must succeed: curl -sf localhost:9901/clusters6653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6654worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6655worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6656(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.41 seconds)6657subtest: queue gauges show a burst while every build blocks6658lb: must succeed: curl -sf localhost:9901/clusters6659lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6660lb: must succeed: curl -sf localhost:9901/clusters6661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6662worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6663worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6664client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6665client # [ 161.372791] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6666client # Running as unit: burst.service; invocation ID: a0ccec8a3d2e45c2a42a044c1cdea70e6667client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)6668lb: must succeed: curl -sf localhost:9901/clusters6669lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6670lb: must succeed: curl -sf localhost:9901/clusters6671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6672worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6673worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6674worker1 # [ 161.492434] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:370046675worker1 # [ 161.497672] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6676client # [ 161.526386] nix[1511]: don't know how to build these paths:6677client # [ 161.527359] nix[1511]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6678client # [ 161.528286] nix[1511]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6679client # [ 161.529454] nix[1511]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6680client # [ 161.530683] nix[1511]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6681worker1 # [ 161.513249] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046682worker2 # [ 161.529964] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06683worker1 # [ 161.518078] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)6684worker2 # [ 161.536790] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16685worker2 # [ 161.541957] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06686worker2 # [ 161.546312] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16687worker2 # [ 161.551703] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06688worker2 # [ 161.555175] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06689worker2 # [ 161.558336] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346690worker1 # [ 161.544223] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)6691worker2 # [ 161.564939] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346692client # [ 161.576451] nix[1511]: copying 1 paths to 'lb:50051'...6693client # [ 161.578726] nix[1511]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6694worker2 # [ 161.576535] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346695worker1 # [ 161.564100] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)6696worker2 # [ 161.585884] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6697worker2 # [ 161.670950] nix-daemon[860]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6698worker2 # [ 161.683800] nix-daemon[860]: reaped child process 1923, status = succeeded6699lb # [ 161.667616] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received push request method=POST path=/api/pushes6700worker2 # [ 161.699473] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.872Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6701worker2 # [ 161.701550] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.874Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6702worker1 # [ 161.682313] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 409 ms (attempt 4/5)6703lb # [ 161.698220] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6704lb # [ 161.707339] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6705lb # [ 161.713863] niks3-server[1142]: 2026/09/23 16:14:50 INFO Signed narinfos id=60 count=16706worker2 # [ 161.740088] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.913Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 161.720873] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6708lb # [ 161.732655] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received complete push request method=POST path=/api/pushes/60/complete6709worker2 # [ 161.762149] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.935Z level=INFO msg="Upload complete. (168ms)"6710lb # [ 161.741438] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6711worker2 # [ 161.766434] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=4166712worker2 # [ 161.771515] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346713worker2 # [ 161.781763] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6714worker2 # [ 161.787124] nix-daemon[860]: reaped child process 1918, status = succeeded6715worker2 # [ 161.791861] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6716client # [ 161.794835] nix[1511]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6717worker1 # [ 162.093212] nix-grpc-daemon[1697]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6718worker1 # [ 162.094697] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6719worker1 # [ 162.105871] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046720client # [ 162.135293] nix[1511]: copying 1 paths to 'lb:50051'...6721worker1 # [ 162.113821] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370046722client # [ 162.137905] nix[1511]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6723worker1 # [ 162.121676] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6724worker1 # [ 162.134982] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=info event=niks3_push_started pid=17576725worker1 # [ 162.225570] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.472Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6726worker1 # [ 162.310752] nix-daemon[868]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6727worker1 # [ 162.321609] nix-daemon[868]: reaped child process 1773, status = succeeded6728lb # [ 162.326365] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received push request method=POST path=/api/pushes6729worker1 # [ 162.338733] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.586Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6730worker1 # [ 162.340197] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.587Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6731lb # [ 162.361149] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6732lb # [ 162.368496] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6733worker1 # [ 162.376955] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.624Z level=INFO msg="Uploading 1 narinfos"6734lb # [ 162.373192] niks3-server[1142]: 2026/09/23 16:14:50 INFO Signed narinfos id=61 count=16735lb # [ 162.385993] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6736lb # [ 162.388503] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received complete push request method=POST path=/api/pushes/61/complete6737worker1 # [ 162.396943] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.644Z level=INFO msg="Upload complete. (167ms)"6738lb # [ 162.394133] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6739worker1 # [ 162.399251] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=4166740worker1 # [ 162.402344] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046741worker1 # [ 162.407833] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6742worker1 # [ 162.412038] nix-daemon[868]: reaped child process 1755, status = succeeded6743worker1 # [ 162.416541] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6744client # [ 162.438498] nix[1511]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6745lb: must succeed: curl -sf localhost:9901/clusters6746lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6747lb: must succeed: curl -sf localhost:9901/clusters6748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6749worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6750worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6751worker2 # [ 162.820758] nix-daemon[860]: reaped child process 1927, status = succeeded6752worker2 # [ 162.899089] nix-daemon[860]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6753worker2 # [ 162.910665] nix-daemon[860]: reaped child process 1954, status = succeeded6754lb # [ 162.894708] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes6755worker2 # [ 162.927083] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6756worker2 # [ 162.929148] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.102Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6757lb # [ 162.937280] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6758lb # [ 162.941323] niks3-server[1142]: 2026/09/23 16:14:51 INFO Signed narinfos id=62 count=16759worker2 # [ 162.966936] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.140Z level=INFO msg="Uploading 1 narinfos"6760lb # [ 162.948213] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6761lb # [ 162.952974] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6762lb # [ 162.961650] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/62/complete6763lb # [ 162.964936] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6764lb # [ 162.969357] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6765worker2 # [ 162.991696] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.165Z level=INFO msg="Upload complete. (193ms)"6766worker2 # [ 162.998156] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=16767worker2 # [ 163.005373] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16768worker2 # [ 163.010248] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346769worker2 # [ 163.018292] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367346770worker2 # [ 163.022395] nix-daemon[860]: reaped child process 1925, status = succeeded6771worker2 # [ 163.024335] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367346772client # [ 163.027868] nix[1511]: copying 1 paths to 'lb:50051'...6773client # [ 163.030317] nix[1511]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6774worker2 # [ 163.032697] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6775worker2 # [ 163.116154] nix-daemon[860]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6776worker2 # [ 163.131297] nix-daemon[860]: reaped child process 1961, status = succeeded6777lb # [ 163.113221] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes6778worker2 # [ 163.145094] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.318Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6779worker2 # [ 163.146847] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.320Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6780lb # [ 163.148248] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6781lb # [ 163.152599] niks3-server[1142]: 2026/09/23 16:14:51 INFO Signed narinfos id=63 count=16782worker2 # [ 163.178151] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.351Z level=INFO msg="Uploading 1 narinfos"6783lb # [ 163.160140] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6784lb # [ 163.163632] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6785lb # [ 163.172654] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/63/complete6786worker2 # [ 163.199597] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.373Z level=INFO msg="Upload complete. (161ms)"6787lb # [ 163.178476] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6788worker2 # [ 163.202888] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=4166789worker2 # [ 163.208235] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367346790worker2 # [ 163.215170] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6791worker2 # [ 163.222457] nix-daemon[860]: reaped child process 1956, status = succeeded6792worker2 # [ 163.225961] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6793client # [ 163.228891] nix[1511]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6794worker1 # [ 163.832405] nix-daemon[868]: reaped child process 1777, status = succeeded6795worker1 # [ 163.905363] nix-daemon[868]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6796worker1 # [ 163.918366] nix-daemon[868]: reaped child process 1805, status = succeeded6797lb # [ 163.922806] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes6798worker1 # [ 163.932866] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6799worker1 # [ 163.934581] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.181Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6800lb # [ 163.959206] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6801lb # [ 163.973365] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6802worker1 # [ 163.984071] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.231Z level=INFO msg="Uploading 1 narinfos"6803lb # [ 163.980794] niks3-server[1142]: 2026/09/23 16:14:52 INFO Signed narinfos id=64 count=16804lb # [ 163.985069] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6805lb # [ 163.990543] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6806lb # [ 164.001044] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/64/complete6807worker1 # [ 164.007784] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.255Z level=INFO msg="Upload complete. (184ms)"6808lb # [ 164.004923] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6809worker1 # [ 164.010540] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=16810worker2 # [ 164.030991] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:53Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16811worker1 # [ 164.014552] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046812worker1 # [ 164.021559] nix-daemon[868]: reaped child process 1775, status = succeeded6813worker1 # [ 164.023282] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046814worker1 # [ 164.028678] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370046815client # [ 164.051840] nix[1511]: copying 1 paths to 'lb:50051'...6816client # [ 164.053740] nix[1511]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6817worker1 # [ 164.037235] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6818worker1 # [ 164.120934] nix-daemon[868]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6819worker1 # [ 164.133612] nix-daemon[868]: reaped child process 1812, status = succeeded6820lb # [ 164.136749] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes6821worker1 # [ 164.147720] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6822worker1 # [ 164.149647] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.396Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6823lb # [ 164.163720] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6824worker1 # [ 164.173931] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.421Z level=INFO msg="Uploading 1 narinfos"6825lb # [ 164.170107] niks3-server[1142]: 2026/09/23 16:14:52 INFO Signed narinfos id=65 count=16826lb # [ 164.179976] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6827lb # [ 164.187399] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6828lb # [ 164.195223] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/65/complete6829worker1 # [ 164.207919] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.455Z level=INFO msg="Upload complete. (165ms)"6830lb # [ 164.205024] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6831worker1 # [ 164.210185] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=4166832worker1 # [ 164.213141] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046833worker1 # [ 164.218436] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6834worker1 # [ 164.223788] nix-daemon[868]: reaped child process 1807, status = succeeded6835worker1 # [ 164.227804] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6836client # [ 164.249746] nix[1511]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6837worker2 # [ 165.062417] nix-daemon[860]: reaped child process 1965, status = succeeded6838worker2 # [ 165.143415] nix-daemon[860]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6839worker2 # [ 165.156241] nix-daemon[860]: reaped child process 1993, status = succeeded6840lb # [ 165.140553] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received push request method=POST path=/api/pushes6841worker2 # [ 165.172125] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.345Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6842worker2 # [ 165.173711] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.347Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6843lb # [ 165.181273] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6844worker2 # [ 165.208839] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.382Z level=INFO msg="Uploading 1 narinfos"6845lb # [ 165.186237] niks3-server[1142]: 2026/09/23 16:14:53 INFO Signed narinfos id=66 count=16846lb # [ 165.195639] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6847lb # [ 165.206130] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6848lb # [ 165.212121] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6849lb # [ 165.216107] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received complete push request method=POST path=/api/pushes/66/complete6850worker2 # [ 165.244191] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.417Z level=INFO msg="Upload complete. (208ms)"6851lb # [ 165.222898] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6852worker2 # [ 165.251153] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=16853worker2 # [ 165.264538] nix-daemon[860]: reaped child process 1963, status = succeeded6854worker1 # [ 166.043204] nix-daemon[868]: reaped child process 1816, status = succeeded6855worker1 # [ 166.116936] nix-daemon[868]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)6856worker1 # [ 166.128965] nix-daemon[868]: reaped child process 1844, status = succeeded6857lb # [ 166.132547] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received push request method=POST path=/api/pushes6858worker1 # [ 166.145084] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6859worker1 # [ 166.146610] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.394Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6860lb # [ 166.165383] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6861lb # [ 166.183621] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6862lb # [ 166.186646] niks3-server[1142]: 2026/09/23 16:14:54 INFO Signed narinfos id=67 count=16863worker1 # [ 166.193630] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.440Z level=INFO msg="Uploading 1 narinfos"6864lb # [ 166.192757] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6865lb # [ 166.196713] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6866lb # [ 166.208575] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received complete push request method=POST path=/api/pushes/67/complete6867worker1 # [ 166.215734] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.463Z level=INFO msg="Upload complete. (181ms)"6868lb # [ 166.213221] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6869worker1 # [ 166.218217] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=16870worker1 # [ 166.225441] nix-daemon[868]: reaped child process 1814, status = succeeded6871client # [ 166.274689] systemd[1]: burst.service: Deactivated successfully.6872client # [ 166.278946] systemd[1]: burst.service: Consumed 155ms CPU time over 4.904s wall clock time, 51.9M memory peak, 13.9K incoming IP traffic, 18.7K outgoing IP traffic.6873lb # [ 166.877501] envoy[921]: conn peer=[2001:db8:1::1]:54156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9594 tx=4361 ms=56876874lb # [ 166.881286] envoy[921]: conn peer=[2001:db8:1::1]:54154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1522 ms=86586875lb # [ 166.884724] envoy[921]: conn peer=[2001:db8:1::3]:39038 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=482 ms=229786876lb # [ 166.888325] envoy[921]: conn peer=[2001:db8:1::1]:38872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=5171 ms=47426877client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6878client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6879lb: must succeed: curl -sf localhost:9901/clusters6880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6881lb: must succeed: curl -sf localhost:9901/clusters6882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6883worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6884worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6885(finished: subtest: queue gauges show a burst while every build blocks, in 6.10 seconds)6886subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6887client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16888worker2 # [ 167.485803] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367346889worker1 # [ 167.475236] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046890worker2 # [ 167.533202] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06891worker2 # [ 167.537418] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16892worker2 # [ 167.541670] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06893worker2 # [ 167.545907] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06894worker1 # [ 167.529355] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046895worker1 # [ 167.536955] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046896worker1 # [ 167.544290] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370046897worker1 # [ 167.549561] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6898worker1 # [ 167.632323] nix-daemon[868]: accepted connection from pid 1852, user nix-grpc-daemon (trusted)6899worker1 # [ 167.644420] nix-daemon[868]: reaped child process 1855, status = succeeded6900lb # [ 167.648534] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received push request method=POST path=/api/pushes6901worker1 # [ 167.660666] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6902worker1 # [ 167.662637] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.909Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6903lb # [ 167.680453] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6904lb # [ 167.685610] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6905worker1 # [ 167.696034] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.943Z level=INFO msg="Uploading 1 narinfos"6906lb # [ 167.691804] niks3-server[1142]: 2026/09/23 16:14:56 INFO Signed narinfos id=68 count=16907lb # [ 167.698351] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6908lb # [ 167.712518] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received complete push request method=POST path=/api/pushes/68/complete6909worker1 # [ 167.725488] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.972Z level=INFO msg="Upload complete. (171ms)"6910lb # [ 167.723595] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6911worker1 # [ 167.727744] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=5046912worker1 # [ 167.732030] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046913worker1 # [ 167.737886] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6914worker1 # [ 167.740229] nix-daemon[868]: reaped child process 1850, status = succeeded6915worker1 # [ 167.745216] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6916worker1 # [ 168.959789] nix-daemon[868]: reaped child process 1859, status = succeeded6917worker1 # [ 169.034796] nix-daemon[868]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6918worker1 # [ 169.045564] nix-daemon[868]: reaped child process 1878, status = succeeded6919lb # [ 169.049941] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received push request method=POST path=/api/pushes6920worker1 # [ 169.062217] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6921worker1 # [ 169.063730] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.311Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6922lb # [ 169.087319] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6923lb # [ 169.097979] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6924worker1 # [ 169.110332] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.357Z level=INFO msg="Uploading 1 narinfos"6925lb # [ 169.105981] niks3-server[1142]: 2026/09/23 16:14:57 INFO Signed narinfos id=69 count=16926lb # [ 169.113630] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6927lb # [ 169.118787] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6928lb # [ 169.127690] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received complete push request method=POST path=/api/pushes/69/complete6929worker1 # [ 169.137901] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.385Z level=INFO msg="Upload complete. (186ms)"6930lb # [ 169.135551] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6931worker1 # [ 169.140554] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=16932worker1 # [ 169.148399] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046933worker2 # [ 169.162972] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:58Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16934worker1 # [ 169.151608] nix-daemon[868]: reaped child process 1857, status = succeeded6935worker1 # [ 169.155805] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046936worker2 # [ 169.173409] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:58Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06937worker1 # [ 169.161332] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370046938worker1 # [ 169.166138] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6939worker1 # [ 169.248848] nix-daemon[868]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6940worker1 # [ 169.261965] nix-daemon[868]: reaped child process 1885, status = succeeded6941lb # [ 169.264132] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received push request method=POST path=/api/pushes6942worker1 # [ 169.275698] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.523Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6943worker1 # [ 169.277666] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.524Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6944lb # [ 169.293379] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6945lb # [ 169.299708] niks3-server[1142]: 2026/09/23 16:14:57 INFO Signed narinfos id=70 count=16946worker1 # [ 169.307202] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.554Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 169.307947] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6948lb # [ 169.317169] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6949lb # [ 169.325757] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received complete push request method=POST path=/api/pushes/70/complete6950worker1 # [ 169.338343] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.585Z level=INFO msg="Upload complete. (167ms)"6951lb # [ 169.336048] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6952worker1 # [ 169.340403] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=5046953worker1 # [ 169.344572] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046954worker1 # [ 169.350688] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6955worker1 # [ 169.353065] nix-daemon[868]: reaped child process 1880, status = succeeded6956worker1 # [ 169.357928] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6957worker1 # [ 170.373907] nix-daemon[868]: reaped child process 1889, status = succeeded6958worker1 # [ 170.446764] nix-daemon[868]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6959worker1 # [ 170.459088] nix-daemon[868]: reaped child process 1908, status = succeeded6960lb # [ 170.464804] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received push request method=POST path=/api/pushes6961worker1 # [ 170.477326] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6962worker1 # [ 170.478907] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.726Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6963lb # [ 170.508461] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6964lb # [ 170.513445] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6965lb # [ 170.522431] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6966lb # [ 170.527948] niks3-server[1142]: 2026/09/23 16:14:58 INFO Signed narinfos id=71 count=16967worker1 # [ 170.534762] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.782Z level=INFO msg="Uploading 1 narinfos"6968lb # [ 170.534884] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6969lb # [ 170.544860] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received complete push request method=POST path=/api/pushes/71/complete6970worker1 # [ 170.552556] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.799Z level=INFO msg="Upload complete. (187ms)"6971lb # [ 170.550281] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6972worker1 # [ 170.555347] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=16973worker1 # [ 170.559558] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046974worker2 # [ 170.579076] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:59Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16975worker1 # [ 170.566328] nix-daemon[868]: reaped child process 1887, status = succeeded6976worker1 # [ 170.567887] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370046977worker2 # [ 170.586737] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:59Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06978worker1 # [ 170.573322] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370046979worker1 # [ 170.577932] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6980worker1 # [ 170.659597] nix-daemon[868]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)6981worker1 # [ 170.673152] nix-daemon[868]: reaped child process 1915, status = succeeded6982lb # [ 170.674259] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received push request method=POST path=/api/pushes6983worker1 # [ 170.686264] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6984worker1 # [ 170.688321] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.935Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6985lb # [ 170.701630] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6986lb # [ 170.712074] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6987lb # [ 170.718427] niks3-server[1142]: 2026/09/23 16:14:59 INFO Signed narinfos id=72 count=16988worker1 # [ 170.725558] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.972Z level=INFO msg="Uploading 1 narinfos"6989lb # [ 170.726771] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6990lb # [ 170.737106] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received complete push request method=POST path=/api/pushes/72/complete6991worker1 # [ 170.748495] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.995Z level=INFO msg="Upload complete. (166ms)"6992lb # [ 170.745418] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6993worker1 # [ 170.750507] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=5046994worker1 # [ 170.754522] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370046995worker1 # [ 170.760873] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6996worker1 # [ 170.763212] nix-daemon[868]: reaped child process 1910, status = succeeded6997worker1 # [ 170.768181] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6998worker1 # [ 172.183260] nix-daemon[868]: reaped child process 1919, status = succeeded6999worker1 # [ 172.257630] nix-daemon[868]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)7000worker1 # [ 172.270755] nix-daemon[868]: reaped child process 1938, status = succeeded7001lb # [ 172.273223] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received push request method=POST path=/api/pushes7002worker1 # [ 172.285468] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.532Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7003worker1 # [ 172.286932] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.534Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7004lb # [ 172.316173] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign7005worker1 # [ 172.325418] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.572Z level=INFO msg="Uploading 1 narinfos"7006lb # [ 172.321724] niks3-server[1142]: 2026/09/23 16:15:00 INFO Signed narinfos id=73 count=17007lb # [ 172.332526] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7008lb # [ 172.337667] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7009lb # [ 172.342674] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7010lb # [ 172.351855] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received complete push request method=POST path=/api/pushes/73/complete7011worker1 # [ 172.360268] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.607Z level=INFO msg="Upload complete. (185ms)"7012lb # [ 172.357104] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7013worker1 # [ 172.363092] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=17014worker1 # [ 172.366994] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047015worker2 # [ 172.386547] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:01Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17016worker1 # [ 172.374265] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047017worker1 # [ 172.376291] nix-daemon[868]: reaped child process 1917, status = succeeded7018worker2 # [ 172.394507] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:01Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07019worker1 # [ 172.380891] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.628Z level=INFO msg="All 1 paths already cached"7020worker1 # [ 172.384992] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047021worker1 # [ 172.389697] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7022worker1 # [ 172.471254] nix-daemon[868]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)7023worker1 # [ 172.483337] nix-daemon[868]: reaped child process 1945, status = succeeded7024lb # [ 172.487119] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received push request method=POST path=/api/pushes7025worker1 # [ 172.498662] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.746Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7026worker1 # [ 172.500612] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.747Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7027lb # [ 172.515848] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7028worker1 # [ 172.526265] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.773Z level=INFO msg="Uploading 1 narinfos"7029lb # [ 172.522169] niks3-server[1142]: 2026/09/23 16:15:00 INFO Signed narinfos id=74 count=17030lb # [ 172.532285] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7031lb # [ 172.539994] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7032lb # [ 172.548362] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received complete push request method=POST path=/api/pushes/74/complete7033worker1 # [ 172.559218] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.806Z level=INFO msg="Upload complete. (165ms)"7034lb # [ 172.556232] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7035worker1 # [ 172.561423] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=6247036worker1 # [ 172.564477] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047037worker1 # [ 172.570840] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7038worker1 # [ 172.574301] nix-daemon[868]: reaped child process 1940, status = succeeded7039worker1 # [ 172.578857] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7040worker1 # [ 173.594147] nix-daemon[868]: reaped child process 1949, status = succeeded7041worker1 # [ 173.667849] nix-daemon[868]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)7042worker1 # [ 173.680304] nix-daemon[868]: reaped child process 1968, status = succeeded7043lb # [ 173.683883] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes7044worker1 # [ 173.695954] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.943Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7045worker1 # [ 173.698142] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.944Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7046lb # [ 173.722143] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7047lb # [ 173.735234] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7048lb # [ 173.738663] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7049worker1 # [ 173.747170] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.994Z level=INFO msg="Uploading 1 narinfos"7050lb # [ 173.744246] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=75 count=17051lb # [ 173.750323] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7052lb # [ 173.762282] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/75/complete7053lb # [ 173.767043] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7054worker1 # [ 173.774331] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.021Z level=INFO msg="Upload complete. (188ms)"7055worker1 # [ 173.777027] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=17056worker1 # [ 173.781268] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047057worker2 # [ 173.802515] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:02Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17058worker1 # [ 173.788441] nix-daemon[868]: reaped child process 1947, status = succeeded7059worker1 # [ 173.790315] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047060worker2 # [ 173.807236] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:02Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07061worker1 # [ 173.795901] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.043Z level=INFO msg="All 1 paths already cached"7062worker1 # [ 173.799839] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047063worker1 # [ 173.804535] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7064worker1 # [ 173.886371] nix-daemon[868]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)7065worker1 # [ 173.899792] nix-daemon[868]: reaped child process 1975, status = succeeded7066lb # [ 173.900742] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes7067worker1 # [ 173.913024] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7068worker1 # [ 173.915065] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.162Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7069lb # [ 173.930693] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7070worker1 # [ 173.940101] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.187Z level=INFO msg="Uploading 1 narinfos"7071lb # [ 173.936470] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=76 count=17072lb # [ 173.944768] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7073lb # [ 173.955089] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7074lb # [ 173.961333] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/76/complete7075worker1 # [ 173.972670] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.220Z level=INFO msg="Upload complete. (163ms)"7076lb # [ 173.970628] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7077worker1 # [ 173.974846] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=6247078worker1 # [ 173.978601] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047079worker1 # [ 173.985123] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7080worker1 # [ 173.987457] nix-daemon[868]: reaped child process 1970, status = succeeded7081worker1 # [ 173.992554] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7082worker1 # [ 174.404859] nix-daemon[868]: reaped child process 1979, status = succeeded7083worker1 # [ 174.479252] nix-daemon[868]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7084worker1 # [ 174.492756] nix-daemon[868]: reaped child process 1998, status = succeeded7085lb # [ 174.493624] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes7086worker1 # [ 174.507069] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7087worker1 # [ 174.508476] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.753Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7088lb # [ 174.534344] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7089lb # [ 174.545164] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7090lb # [ 174.551188] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=77 count=17091worker1 # [ 174.558338] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.805Z level=INFO msg="Uploading 1 narinfos"7092lb # [ 174.559942] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7093lb # [ 174.568758] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7094lb # [ 174.576866] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/77/complete7095worker1 # [ 174.584910] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.832Z level=INFO msg="Upload complete. (187ms)"7096lb # [ 174.582624] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7097worker1 # [ 174.587250] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=17098worker1 # [ 174.591946] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047099worker2 # [ 174.609142] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:03Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17100worker1 # [ 174.598388] nix-daemon[868]: reaped child process 1977, status = succeeded7101worker1 # [ 174.600206] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047102worker2 # [ 174.618297] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:03Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07103worker1 # [ 174.604304] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.851Z level=INFO msg="All 1 paths already cached"7104worker1 # [ 174.608102] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047105worker1 # [ 174.613068] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7106worker1 # [ 174.695354] nix-daemon[868]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)7107worker1 # [ 174.709295] nix-daemon[868]: reaped child process 2005, status = succeeded7108lb # [ 174.710616] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received push request method=POST path=/api/pushes7109worker1 # [ 174.722478] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.969Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7110worker1 # [ 174.724354] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.971Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7111lb # [ 174.741457] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7112worker1 # [ 174.752158] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.999Z level=INFO msg="Uploading 1 narinfos"7113lb # [ 174.748463] niks3-server[1142]: 2026/09/23 16:15:03 INFO Signed narinfos id=78 count=17114lb # [ 174.756243] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7115lb # [ 174.762215] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7116lb # [ 174.772654] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received complete push request method=POST path=/api/pushes/78/complete7117worker1 # [ 174.785184] nix-grpc-daemon[1757]: time=2026-09-23T16:15:03.032Z level=INFO msg="Upload complete. (167ms)"7118lb # [ 174.782618] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7119worker1 # [ 174.787670] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=6247120worker1 # [ 174.790768] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047121worker1 # [ 174.798704] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7122worker1 # [ 174.800081] nix-daemon[868]: reaped child process 2000, status = succeeded7123worker1 # [ 174.805180] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7124worker1 # [ 175.820869] nix-daemon[868]: reaped child process 2009, status = succeeded7125worker1 # [ 175.894943] nix-daemon[868]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7126worker1 # [ 175.907348] nix-daemon[868]: reaped child process 2028, status = succeeded7127lb # [ 175.910919] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received push request method=POST path=/api/pushes7128worker1 # [ 175.921186] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.168Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7129worker1 # [ 175.922723] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.170Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7130lb # [ 175.946151] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7131lb # [ 175.960546] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7132lb # [ 175.967619] niks3-server[1142]: 2026/09/23 16:15:04 INFO Signed narinfos id=79 count=17133worker1 # [ 175.975095] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.222Z level=INFO msg="Uploading 1 narinfos"7134lb # [ 175.975351] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7135lb # [ 175.979282] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7136lb # [ 175.990451] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received complete push request method=POST path=/api/pushes/79/complete7137lb # [ 175.996921] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7138worker1 # [ 176.003574] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.251Z level=INFO msg="Upload complete. (192ms)"7139worker1 # [ 176.006202] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=17140worker1 # [ 176.009931] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047141worker2 # [ 176.027522] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17142worker1 # [ 176.016941] nix-daemon[868]: reaped child process 2007, status = succeeded7143worker1 # [ 176.018293] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047144worker2 # [ 176.037136] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07145worker1 # [ 176.023882] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.271Z level=INFO msg="All 2 paths already cached"7146worker1 # [ 176.028090] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047147worker1 # [ 176.032810] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7148worker1 # [ 176.114964] nix-daemon[868]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)7149worker1 # [ 176.127716] nix-daemon[868]: reaped child process 2035, status = succeeded7150lb # [ 176.130253] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received push request method=POST path=/api/pushes7151worker1 # [ 176.143149] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.390Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7152worker1 # [ 176.145216] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.392Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7153lb # [ 176.162258] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7154lb # [ 176.168060] niks3-server[1142]: 2026/09/23 16:15:04 INFO Signed narinfos id=80 count=17155worker1 # [ 176.175320] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.422Z level=INFO msg="Uploading 1 narinfos"7156lb # [ 176.177149] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7157lb # [ 176.186297] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7158lb # [ 176.194838] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received complete push request method=POST path=/api/pushes/80/complete7159lb # [ 176.199910] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7160worker1 # [ 176.208425] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.455Z level=INFO msg="Upload complete. (171ms)"7161worker1 # [ 176.210438] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=7527162worker1 # [ 176.213215] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047163worker1 # [ 176.221503] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7164worker1 # [ 176.223361] nix-daemon[868]: reaped child process 2030, status = succeeded7165worker1 # [ 176.228043] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7166worker1 # [ 176.640477] nix-daemon[868]: reaped child process 2039, status = succeeded7167worker1 # [ 176.716547] nix-daemon[868]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)7168worker1 # [ 176.727609] nix-daemon[868]: reaped child process 2058, status = succeeded7169lb # [ 176.730807] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes7170worker1 # [ 176.739096] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7171worker1 # [ 176.740622] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.988Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7172lb # [ 176.751634] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7173lb # [ 176.753133] niks3-server[1142]: 2026/09/23 16:15:05 INFO Signed narinfos id=81 count=17174worker1 # [ 176.758321] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.005Z level=INFO msg="Uploading 1 narinfos"7175lb # [ 176.757070] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7176lb # [ 176.760107] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7177lb # [ 176.763131] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7178lb # [ 176.767981] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/81/complete7179worker1 # [ 176.775488] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.022Z level=INFO msg="Upload complete. (140ms)"7180lb # [ 176.773273] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7181worker1 # [ 176.777777] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=17182worker1 # [ 176.782421] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047183worker2 # [ 176.799907] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17184worker1 # [ 176.788490] nix-daemon[868]: reaped child process 2037, status = succeeded7185worker1 # [ 176.789940] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047186worker1 # [ 176.795466] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.042Z level=INFO msg="All 2 paths already cached"7187worker1 # [ 176.799507] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047188worker1 # [ 176.804148] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7189worker1 # [ 176.886126] nix-daemon[868]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)7190worker1 # [ 176.899299] nix-daemon[868]: reaped child process 2065, status = succeeded7191lb # [ 176.900926] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes7192worker1 # [ 176.909696] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.156Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7193worker1 # [ 176.911073] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.156Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7194lb # [ 176.915251] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7195worker1 # [ 176.921524] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.168Z level=INFO msg="Uploading 1 narinfos"7196lb # [ 176.918460] niks3-server[1142]: 2026/09/23 16:15:05 INFO Signed narinfos id=82 count=17197lb # [ 176.919483] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7198lb # [ 176.926083] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7199lb # [ 176.927886] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/82/complete7200worker1 # [ 176.935177] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.182Z level=INFO msg="Upload complete. (126ms)"7201lb # [ 176.932876] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7202worker1 # [ 176.937256] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=7527203worker1 # [ 176.940865] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047204worker1 # [ 176.946938] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7205worker1 # [ 176.949102] nix-daemon[868]: reaped child process 2060, status = succeeded7206worker1 # [ 176.954332] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7207worker1 # [ 177.790378] nix-daemon[868]: reaped child process 2069, status = succeeded7208worker1 # [ 177.873088] nix-daemon[868]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)7209worker1 # [ 177.885077] nix-daemon[868]: reaped child process 2088, status = succeeded7210lb # [ 177.887555] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received push request method=POST path=/api/pushes7211worker1 # [ 177.899322] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.146Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7212worker1 # [ 177.900837] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.148Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7213lb # [ 177.924651] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7214lb # [ 177.934134] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7215worker1 # [ 177.943935] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.191Z level=INFO msg="Uploading 1 narinfos"7216lb # [ 177.939543] niks3-server[1142]: 2026/09/23 16:15:06 INFO Signed narinfos id=83 count=17217lb # [ 177.948059] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7218lb # [ 177.953644] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7219lb # [ 177.961357] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received complete push request method=POST path=/api/pushes/83/complete7220lb # [ 177.970728] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7221worker1 # [ 177.973957] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.221Z level=INFO msg="Upload complete. (212ms)"7222worker2 # [ 178.000418] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:07Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17223worker1 # [ 177.980068] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=17224worker1 # [ 177.989159] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047225worker1 # [ 178.000542] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:370047226worker1 # [ 178.007191] nix-daemon[868]: reaped child process 2067, status = succeeded7227worker1 # [ 178.013763] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.261Z level=INFO msg="All 2 paths already cached"7228worker1 # [ 178.020362] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:370047229worker1 # [ 178.026919] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7230worker1 # [ 178.112765] nix-daemon[868]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)7231worker1 # [ 178.125629] nix-daemon[868]: reaped child process 2095, status = succeeded7232lb # [ 178.127978] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received push request method=POST path=/api/pushes7233worker1 # [ 178.140823] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7234worker1 # [ 178.142329] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.389Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7235lb # [ 178.159966] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7236lb # [ 178.168290] niks3-server[1142]: 2026/09/23 16:15:06 INFO Signed narinfos id=84 count=17237worker1 # [ 178.175101] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.422Z level=INFO msg="Uploading 1 narinfos"7238lb # [ 178.174239] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7239lb # [ 178.180184] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7240lb # [ 178.191286] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received complete push request method=POST path=/api/pushes/84/complete7241worker1 # [ 178.203310] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.450Z level=INFO msg="Upload complete. (168ms)"7242lb # [ 178.202862] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7243worker1 # [ 178.207355] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=7527244worker1 # [ 178.213170] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:370047245worker1 # [ 178.224360] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7246worker1 # [ 178.226775] nix-daemon[868]: reaped child process 2090, status = succeeded7247worker1 # [ 178.231702] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7248worker1 # [ 179.447670] nix-daemon[868]: reaped child process 2099, status = succeeded7249worker1 # [ 179.527135] nix-daemon[868]: accepted connection from pid 2115, user nix-grpc-daemon (trusted)7250worker1 # [ 179.539928] nix-daemon[868]: reaped child process 2118, status = succeeded7251lb # [ 179.543118] niks3-server[1142]: 2026/09/23 16:15:07 INFO Received push request method=POST path=/api/pushes7252worker1 # [ 179.555374] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.802Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7253worker1 # [ 179.556894] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.804Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7254worker1 # [ 179.587246] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.834Z level=INFO msg="Uploading 1 narinfos"7255lb # [ 179.583176] niks3-server[1142]: 2026/09/23 16:15:07 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7256lb # [ 179.586589] niks3-server[1142]: 2026/09/23 16:15:07 INFO Signed narinfos id=85 count=17257lb # [ 179.597977] niks3-server[1142]: 2026/09/23 16:15:07 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7258lb # [ 179.603124] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7259lb # [ 179.613124] niks3-server[1142]: 2026/09/23 16:15:08 INFO Received complete push request method=POST path=/api/pushes/85/complete7260worker1 # [ 179.624865] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.872Z level=INFO msg="Upload complete. (186ms)"7261lb # [ 179.621918] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7262lb # [ 179.624489] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7263worker1 # [ 179.627843] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=17264worker1 # [ 179.635096] nix-daemon[868]: reaped child process 2097, status = succeeded7265client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 12.35 seconds)7266worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7267worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7268client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7269client # [ 179.762607] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7270client # Running as unit: gpu.service; invocation ID: 69eff944ceb040faa5efd13dc45be9177271client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7272lb: must succeed: curl -sf localhost:9901/clusters7273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7274lb: must succeed: curl -sf localhost:9901/clusters7275lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7276worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7277worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7278worker1 # [ 179.870334] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:370047279client # [ 179.896350] nix[1581]: don't know how to build these paths:7280client # [ 179.897182] nix[1581]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7281worker2 # [ 179.897460] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367347282client # [ 179.921408] nix[1581]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7283worker2 # [ 179.917595] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:09Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07284lb: must succeed: curl -sf localhost:9901/clusters7285lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7286lb: must succeed: curl -sf localhost:9901/clusters7287lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7288worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7289worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7290client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7291lb # [ 186.878956] envoy[921]: conn peer=[2001:db8:1::1]:59102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27047 tx=12197 ms=121777292client # [ 209.922730] nix[1581]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7293client # [ 209.931594] nix[1581]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7294client # [ 209.964549] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7295client # [ 209.967899] systemd[1]: gpu.service: Failed with result 'exit-code'.7296client # [ 209.972881] systemd[1]: gpu.service: Consumed 130ms CPU time over 30.208s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7297client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds)7298client: must succeed: journalctl -u gpu -o cat7299client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7300lb: must succeed: curl -sf localhost:9901/clusters7301lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7302lb: must succeed: curl -sf localhost:9901/clusters7303lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7304worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7305worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7306(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 43.05 seconds)7307subtest: low disk drains a worker and builds go to the other7308worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7309worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7310worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7311worker1 # [ 212.081290] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:40Z level=info event=unhealthy reason=min_free7312worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.06 seconds)7313client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27314client # don't know how to build these paths:7315client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7316worker2 # [ 212.507414] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367347317worker2 # [ 212.513635] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367347318worker2 # [ 212.529824] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07319worker2 # [ 212.535310] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17320worker2 # [ 212.540836] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347321worker2 # [ 212.553879] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367347322client # copying 1 paths to 'lb:50051'...7323client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7324worker2 # [ 212.565073] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367347325worker2 # [ 212.573893] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7326worker2 # [ 212.670856] nix-daemon[860]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7327worker2 # [ 212.683389] nix-daemon[860]: reaped child process 2038, status = succeeded7328lb # [ 212.668582] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes7329worker2 # [ 212.701271] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7330worker2 # [ 212.702723] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.874Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7331lb # [ 212.696169] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7332worker2 # [ 212.727701] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.901Z level=INFO msg="Uploading 1 narinfos"7333lb # [ 212.704539] niks3-server[1142]: 2026/09/23 16:15:41 INFO Signed narinfos id=86 count=17334lb # [ 212.718557] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7335lb # [ 212.727227] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7336lb # [ 212.736758] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/86/complete7337worker2 # [ 212.765208] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.938Z level=INFO msg="Upload complete. (178ms)"7338lb # [ 212.746557] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7339worker2 # [ 212.770544] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=5127340worker2 # [ 212.776629] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347341worker2 # [ 212.788140] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7342worker2 # [ 212.798130] nix-daemon[860]: reaped child process 2033, status = succeeded7343worker2 # [ 212.801047] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7344client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7345client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7346client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7347client # farm-a> LOG-farm-a-drain7348worker2 # [ 214.035839] nix-daemon[860]: reaped child process 2042, status = succeeded7349worker2 # [ 214.120744] nix-daemon[860]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)7350worker2 # [ 214.132567] nix-daemon[860]: reaped child process 2061, status = succeeded7351lb # [ 214.115717] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received push request method=POST path=/api/pushes7352worker2 # [ 214.148307] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7353worker2 # [ 214.150926] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.323Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7354lb # [ 214.153506] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7355worker2 # [ 214.183989] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.357Z level=INFO msg="Uploading 1 narinfos"7356lb # [ 214.160366] niks3-server[1142]: 2026/09/23 16:15:42 INFO Signed narinfos id=87 count=17357lb # [ 214.168431] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7358lb # [ 214.172681] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7359lb # [ 214.177145] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7360lb # [ 214.182660] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received complete push request method=POST path=/api/pushes/87/complete7361worker2 # [ 214.210225] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.383Z level=INFO msg="Upload complete. (201ms)"7362lb # [ 214.189523] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7363worker2 # [ 214.218409] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=17364worker2 # [ 214.227308] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07365worker2 # [ 214.231378] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17366worker2 # [ 214.235128] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347367worker2 # [ 214.243348] nix-daemon[860]: reaped child process 2040, status = succeeded7368worker2 # [ 214.244679] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367347369worker2 # [ 214.250902] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.424Z level=INFO msg="All 1 paths already cached"7370client # copying 1 paths to 'lb:50051'...7371worker2 # [ 214.255914] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367347372client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7373worker2 # [ 214.261836] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7374worker2 # [ 214.343418] nix-daemon[860]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)7375worker2 # [ 214.355388] nix-daemon[860]: reaped child process 2070, status = succeeded7376lb # [ 214.340116] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received push request method=POST path=/api/pushes7377worker2 # [ 214.372534] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.546Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7378worker2 # [ 214.374634] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.547Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7379lb # [ 214.371186] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7380worker2 # [ 214.399173] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.572Z level=INFO msg="Uploading 1 narinfos"7381lb # [ 214.376116] niks3-server[1142]: 2026/09/23 16:15:42 INFO Signed narinfos id=88 count=17382lb # [ 214.381061] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7383lb # [ 214.386164] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7384lb # [ 214.394054] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received complete push request method=POST path=/api/pushes/88/complete7385lb # [ 214.397838] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7386worker2 # [ 214.424513] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.598Z level=INFO msg="Upload complete. (158ms)"7387worker2 # [ 214.427565] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=6247388worker2 # [ 214.432257] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347389worker2 # [ 214.441492] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7390worker2 # [ 214.443144] nix-daemon[860]: reaped child process 2065, status = succeeded7391worker2 # [ 214.448149] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7392client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7393client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7394client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7395client # farm-b> LOG-farm-b-drain7396worker2 # [ 215.884448] nix-daemon[860]: reaped child process 2074, status = succeeded7397worker2 # [ 215.968980] nix-daemon[860]: accepted connection from pid 2090, user nix-grpc-daemon (trusted)7398worker2 # [ 215.981811] nix-daemon[860]: reaped child process 2093, status = succeeded7399lb # [ 215.965955] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received push request method=POST path=/api/pushes7400worker2 # [ 215.997497] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.171Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7401worker2 # [ 215.999294] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.172Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7402lb # [ 216.009224] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7403worker2 # [ 216.037596] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.211Z level=INFO msg="Uploading 1 narinfos"7404lb # [ 216.015316] niks3-server[1142]: 2026/09/23 16:15:44 INFO Signed narinfos id=89 count=17405lb # [ 216.023422] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7406lb # [ 216.028839] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7407lb # [ 216.036726] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7408lb # [ 216.047756] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received complete push request method=POST path=/api/pushes/89/complete7409lb # [ 216.054678] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7410worker2 # [ 216.080993] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.254Z level=INFO msg="Upload complete. (222ms)"7411worker2 # [ 216.091265] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=17412worker2 # [ 216.099765] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07413worker2 # [ 216.105538] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17414worker2 # [ 216.110169] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347415worker2 # [ 216.121566] nix-daemon[860]: reaped child process 2072, status = succeeded7416worker2 # [ 216.123227] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367347417worker2 # [ 216.130911] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.304Z level=INFO msg="All 2 paths already cached"7418client # copying 1 paths to 'lb:50051'...7419worker2 # [ 216.136461] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367347420client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7421worker2 # [ 216.141380] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7422worker2 # [ 216.228878] nix-daemon[860]: accepted connection from pid 2099, user nix-grpc-daemon (trusted)7423worker2 # [ 216.244050] nix-daemon[860]: reaped child process 2102, status = succeeded7424lb # [ 216.225900] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received push request method=POST path=/api/pushes7425worker2 # [ 216.256947] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.430Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7426worker2 # [ 216.258720] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.432Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7427lb # [ 216.255974] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7428lb # [ 216.260452] niks3-server[1142]: 2026/09/23 16:15:44 INFO Signed narinfos id=90 count=17429worker2 # [ 216.287364] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.460Z level=INFO msg="Uploading 1 narinfos"7430lb # [ 216.269763] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7431lb # [ 216.275872] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7432lb # [ 216.289113] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received complete push request method=POST path=/api/pushes/90/complete7433lb # [ 216.295307] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7434worker2 # [ 216.324239] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.497Z level=INFO msg="Upload complete. (176ms)"7435worker2 # [ 216.328117] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=7527436worker2 # [ 216.333704] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367347437worker2 # [ 216.346084] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7438worker2 # [ 216.352924] nix-daemon[860]: reaped child process 2097, status = succeeded7439worker2 # [ 216.357801] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7440client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7441client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7442lb # [ 216.879615] envoy[921]: conn peer=[2001:db8:1::1]:36714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300517443client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7444client # farm-top> LOG-farm-top-drain7445worker2 # [ 217.371808] nix-daemon[860]: reaped child process 2106, status = succeeded7446worker2 # [ 217.447065] nix-daemon[860]: accepted connection from pid 2122, user nix-grpc-daemon (trusted)7447worker2 # [ 217.458623] nix-daemon[860]: reaped child process 2125, status = succeeded7448lb # [ 217.442936] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes7449worker2 # [ 217.474366] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.647Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7450worker2 # [ 217.476363] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.649Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7451lb # [ 217.484953] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7452worker2 # [ 217.513891] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.687Z level=INFO msg="Uploading 1 narinfos"7453lb # [ 217.491144] niks3-server[1142]: 2026/09/23 16:15:45 INFO Signed narinfos id=91 count=17454lb # [ 217.499920] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7455lb # [ 217.504046] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7456lb # [ 217.511178] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7457lb # [ 217.517570] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/91/complete7458lb # [ 217.526409] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7459worker2 # [ 217.548142] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.721Z level=INFO msg="Upload complete. (183ms)"7460worker2 # [ 217.554654] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=17461worker2 # [ 217.568909] nix-daemon[860]: reaped child process 2104, status = succeeded7462client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.23 seconds)7463client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7464client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7465worker1: must succeed: rm /nix/.rw-store/fill7466worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.09 seconds)7467worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7468worker1 # [ 218.084186] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:46Z level=info event=healthy reason=min_free7469worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7470(finished: subtest: low disk drains a worker and builds go to the other, in 8.54 seconds)7471(finished: run the VM test script, in 219.81 seconds)7472test script finished in 219.98s7473cleanup7474kill QemuMachine (pid 49)7475client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7476client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7477client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7478client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7479kill QemuMachine (pid 50)7480lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7481lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7482lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7483lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7484kill QemuMachine (pid 52)7485worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7486worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7487worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7488worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down7489kill QemuMachine (pid 51)7490worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7491worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down7492worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down7493worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down7494(finished: cleanup, in 1.00 seconds)7495additionally exposed symbols:7496 client, lb, worker1, worker2,7497 vlan1,7498 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh