vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #512
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 49)15lb # Formatting '/build/vm-state-lb/tmp.buYwVSYIH9', fmt=raw size=107374182416client: QEMU running (pid 50)17worker2 # Disk image does not exist, creating the virtualisation disk image...18client # Formatting '/build/vm-state-client/tmp.7qL4Dc2xYc', fmt=raw size=107374182419lb # mke2fs 1.47.4 (6-Mar-2025)20worker2: QEMU running (pid 52)21worker2 # Formatting '/build/vm-state-worker2/tmp.uXJEC9kI4L', fmt=raw size=107374182422client # mke2fs 1.47.4 (6-Mar-2025)23worker2 # mke2fs 1.47.4 (6-Mar-2025)24client # Discarding device blocks: 0/262144 done25worker2 # Discarding device blocks: 0/262144 done26client # Creating filesystem with 262144 4k blocks and 65536 inodes27worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes28client # Filesystem UUID: 935c7864-4bec-451b-9c6d-f9811e037d8e29worker2 # Filesystem UUID: 267390b9-6407-422c-a794-9f1b4b2bdf9f30lb # Discarding device blocks: 0/262144 done31client # Superblock backups stored on blocks:32worker2 # Superblock backups stored on blocks:33lb # Creating filesystem with 262144 4k blocks and 65536 inodes34client # 32768, 98304, 163840, 22937635worker2 # 32768, 98304, 163840, 22937636lb # Filesystem UUID: 342e5c28-2b4a-4c5a-9a1d-76ced8e4ca0837worker1: QEMU running (pid 51)38worker2 # 39lb # Superblock backups stored on blocks:40client # 41worker1 # Disk image does not exist, creating the virtualisation disk image...42worker2 # Allocating group tables: 0/8 done43client # Allocating group tables: 0/8 done44worker1 # Formatting '/build/vm-state-worker1/tmp.I5xV9Cn87C', fmt=raw size=107374182445worker2 # Writing inode tables: 0/8 done46client # Writing inode tables: 0/8 done47worker1 # mke2fs 1.47.4 (6-Mar-2025)48worker2 # Creating journal (8192 blocks): done49client # Creating journal (8192 blocks): done50worker1 # Discarding device blocks: 0/262144 done51worker2 # Writing superblocks and filesystem accounting information: 0/8 done52client # Writing superblocks and filesystem accounting information: 0/8 done53worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes54worker2 # 55client # 56worker1 # Filesystem UUID: 661a4feb-6ef4-4235-acef-9ac0c6c3851b57worker2 # Virtualisation disk image created.58client # Virtualisation disk image created.59worker1 # Superblock backups stored on blocks:60worker2 # Starting virtiofs daemons...61client # Starting virtiofs daemons...62worker1 # 32768, 98304, 163840, 22937663worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)65worker1 # 66worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether67client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether68client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...69worker1 # Allocating group tables: 0/8 done70(finished: start all VMs, in 0.23 seconds)71worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...72client: waiting for unit network-addresses-eth1.service73worker1 # Writing inode tables: 0/8 done74worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client: waiting for the VM to finish booting76worker1 # Creating journal (8192 blocks): done77worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79lb # 32768, 98304, 163840, 22937680worker1 # Writing superblocks and filesystem accounting information: 0/8 done81worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...82client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83lb # 84worker1 # 85worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...87lb # Allocating group tables: 0/8 done88worker1 # Virtualisation disk image created.89worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91lb # Writing inode tables: 0/8 done92worker1 # Starting virtiofs daemons...93worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...94client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95lb # Creating journal (8192 blocks): done96worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...99lb # Writing superblocks and filesystem accounting information: 0/8 done100worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests102client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests103lb # 104worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests106client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests107lb # Virtualisation disk image created.108client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests109lb # Starting virtiofs daemons...110worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)111lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)112worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether113lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether114worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...115lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...116worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)117lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)118worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether119lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether120worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...121lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests123lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)124worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests125lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests127lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests131lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135lb # 136lb # 137client # 138client # 139lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142lb # 143lb # 144lb # 145lb # 146client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147client # 148client # 149client # 150client # 151lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00152client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00153worker1 # 154worker1 # 155lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...156worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00157lb # 158lb # 159client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...160client # 161client # 162worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...163worker1 # 164worker1 # 165worker1 # 166worker1 # 167worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker2 # 169worker2 # 170worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00171worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...172worker1 # 173worker1 # 174worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175worker2 # 176worker2 # 177worker2 # 178worker2 # 179worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183lb # Booting from ROM...184client # Booting from ROM...185worker1 # Booting from ROM...186worker2 # Booting from ROM...187lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0190lb # [ 0.000000] BIOS-provided physical RAM map:191lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable192lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test/init regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration console=ttyS0,115200n8 console=tty0194client # [ 0.000000] BIOS-provided physical RAM map:195lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved196lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable197client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable198lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved200lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved201client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved202lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved203client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable204lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved205client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved206lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved207client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved208worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026209lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved210client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved211lb # [ 0.000000] NX (Execute Disable) protection: active212client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved213lb # [ 0.000000] APIC: Static calls initialized214lb # [ 0.000000] SMBIOS 2.8 present.215client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved216client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved217lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014218lb # [ 0.000000] DMI: Memory slots populated: 1/1219client # [ 0.000000] NX (Execute Disable) protection: active220worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mmcv7i4v7dgji0gb0y4pksgkj6hhjkk-nixos-system-worker1-test/init regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration console=ttyS0,115200n8 console=tty0221lb # [ 0.000000] Hypervisor detected: KVM222client # [ 0.000000] APIC: Static calls initialized223worker1 # [ 0.000000] BIOS-provided physical RAM map:224client # [ 0.000000] SMBIOS 2.8 present.225lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000226worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable227lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00228client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014229lb # [ 0.000001] kvm-clock: using sched offset of 459552412 cycles230worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved231client # [ 0.000000] DMI: Memory slots populated: 1/1232client # [ 0.000000] Hypervisor detected: KVM233worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved234lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns235client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000236worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable237lb # [ 0.000005] tsc: Detected 2400.012 MHz processor238client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00239lb # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000240worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved241client # [ 0.000000] kvm-clock: using sched offset of 456535148 cycles242worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved243lb # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs244client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns245worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved246lb # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT247client # [ 0.000005] tsc: Detected 2400.012 MHz processor248lb # [ 0.002749] found SMP MP-table at [mem 0x000f5450-0x000f545f]249worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved250client # [ 0.000823] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000251lb # [ 0.002761] Using GB pages for direct mapping252worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved253lb # [ 0.002900] RAMDISK: [mem 0x5e36e000-0x5ffcffff]254client # [ 0.000850] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs255worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved256lb # [ 0.002907] ACPI: Early table checksum verification disabled257client # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT258worker1 # [ 0.000000] NX (Execute Disable) protection: active259lb # [ 0.002911] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )260client # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]261worker1 # [ 0.000000] APIC: Static calls initialized262worker1 # [ 0.000000] SMBIOS 2.8 present.263client # [ 0.002762] Using GB pages for direct mapping264lb # [ 0.002914] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.002854] RAMDISK: [mem 0x5e370000-0x5ffcffff]266worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014267client # [ 0.002861] ACPI: Early table checksum verification disabled268lb # [ 0.002919] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)269worker1 # [ 0.000000] DMI: Memory slots populated: 1/1270worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026271client # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )272worker1 # [ 0.000000] Hypervisor detected: KVM273lb # [ 0.002926] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)274worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275lb # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040276client # [ 0.002869] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)277worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00278lb # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)279client # [ 0.002874] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)280worker1 # [ 0.000000] kvm-clock: using sched offset of 455925500 cycles281lb # [ 0.002931] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)282client # [ 0.002880] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)283worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns284client # [ 0.002882] ACPI: FACS 0x000000005FFE0000 000040285worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zqb8x0sddns70mbblwgjmcf92snp08cl-nixos-system-worker2-test/init regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration console=ttyS0,115200n8 console=tty0286lb # [ 0.002933] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)287worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor288worker2 # [ 0.000000] BIOS-provided physical RAM map:289worker1 # [ 0.000818] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000290client # [ 0.002884] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)291worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable292lb # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)293client # [ 0.002886] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs295worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved296lb # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]297worker1 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT298worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved299lb # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]300client # [ 0.002887] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)301worker1 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]302worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable303lb # [ 0.002937] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]304worker1 # [ 0.002747] Using GB pages for direct mapping305client # [ 0.002889] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)306worker1 # [ 0.002819] RAMDISK: [mem 0x5e370000-0x5ffcffff]307worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved308lb # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]309client # [ 0.002890] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]310worker1 # [ 0.002827] ACPI: Early table checksum verification disabled311worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved312lb # [ 0.002938] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]313client # [ 0.002891] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]314worker1 # [ 0.002830] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )315worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved316lb # [ 0.002938] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]317client # [ 0.002892] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]318worker1 # [ 0.002834] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)319worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved320lb # [ 0.002939] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]321client # [ 0.002892] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]322lb # [ 0.003167] No NUMA configuration found323worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved324worker1 # [ 0.002838] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)325client # [ 0.002893] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]326lb # [ 0.003168] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]327worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved328worker1 # [ 0.002844] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)329client # [ 0.002893] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]330lb # [ 0.003170] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]331worker2 # [ 0.000000] NX (Execute Disable) protection: active332lb # [ 0.005438] Zone ranges:333worker1 # [ 0.002846] ACPI: FACS 0x000000005FFE0000 000040334worker2 # [ 0.000000] APIC: Static calls initialized335client # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]336worker2 # [ 0.000000] SMBIOS 2.8 present.337lb # [ 0.005439] DMA [mem 0x0000000000001000-0x0000000000ffffff]338client # [ 0.003117] No NUMA configuration found339worker1 # [ 0.002848] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)340lb # [ 0.005441] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]341client # [ 0.003118] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]342lb # [ 0.005442] Normal empty343worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014344worker1 # [ 0.002849] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)345lb # [ 0.005443] Device empty346client # [ 0.003121] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]347worker2 # [ 0.000000] DMI: Memory slots populated: 1/1348lb # [ 0.005443] Movable zone start for each node349client # [ 0.005768] Zone ranges:350worker2 # [ 0.000000] Hypervisor detected: KVM351worker1 # [ 0.002851] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)352lb # [ 0.005444] Early memory node ranges353client # [ 0.005769] DMA [mem 0x0000000000001000-0x0000000000ffffff]354worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000355lb # [ 0.005444] node 0: [mem 0x0000000000001000-0x000000000009efff]356worker1 # [ 0.002852] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)357worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00358client # [ 0.005771] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]359client # [ 0.005772] Normal empty360lb # [ 0.005445] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]361worker2 # [ 0.000001] kvm-clock: using sched offset of 480198118 cycles362client # [ 0.005773] Device empty363worker1 # [ 0.002854] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]364client # [ 0.005774] Movable zone start for each node365lb # [ 0.005446] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]366worker1 # [ 0.002855] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]367client # [ 0.005775] Early memory node ranges368lb # [ 0.005467] On node 0, zone DMA: 1 pages in unavailable ranges369worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns370client # [ 0.005775] node 0: [mem 0x0000000000001000-0x000000000009efff]371worker1 # [ 0.002855] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]372worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor373lb # [ 0.005746] On node 0, zone DMA: 97 pages in unavailable ranges374client # [ 0.005776] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]375worker2 # [ 0.000807] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000376worker1 # [ 0.002856] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]377lb # [ 0.033306] On node 0, zone DMA32: 40 pages in unavailable ranges378lb # [ 0.033782] ACPI: PM-Timer IO Port: 0x608379client # [ 0.005777] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]380worker1 # [ 0.002856] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]381worker2 # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs382lb # [ 0.033795] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])383client # [ 0.005797] On node 0, zone DMA: 1 pages in unavailable ranges384worker2 # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT385worker1 # [ 0.002857] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]386lb # [ 0.033824] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23387client # [ 0.006058] On node 0, zone DMA: 97 pages in unavailable ranges388worker2 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f]389worker1 # [ 0.002857] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]390client # [ 0.034162] On node 0, zone DMA32: 40 pages in unavailable ranges391worker2 # [ 0.002741] Using GB pages for direct mapping392lb # [ 0.033827] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)393worker1 # [ 0.003078] No NUMA configuration found394client # [ 0.034638] ACPI: PM-Timer IO Port: 0x608395worker2 # [ 0.002833] RAMDISK: [mem 0x5e36f000-0x5ffcffff]396lb # [ 0.033828] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)397worker1 # [ 0.003079] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]398client # [ 0.034652] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])399worker2 # [ 0.002841] ACPI: Early table checksum verification disabled400lb # [ 0.033830] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)401worker1 # [ 0.003082] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]402worker1 # [ 0.005367] Zone ranges:403worker2 # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )404client # [ 0.034679] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23405lb # [ 0.033830] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)406worker1 # [ 0.005368] DMA [mem 0x0000000000001000-0x0000000000ffffff]407client # [ 0.034682] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)408worker2 # [ 0.002848] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)409lb # [ 0.033831] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)410worker1 # [ 0.005370] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]411client # [ 0.034684] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)412worker1 # [ 0.005371] Normal empty413lb # [ 0.033834] ACPI: Using ACPI (MADT) for SMP configuration information414worker2 # [ 0.002853] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)415worker1 # [ 0.005372] Device empty416client # [ 0.034685] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)417lb # [ 0.033835] ACPI: HPET id: 0x8086a201 base: 0xfed00000418worker1 # [ 0.005372] Movable zone start for each node419worker1 # [ 0.005373] Early memory node ranges420worker2 # [ 0.002860] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)421client # [ 0.034686] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)422lb # [ 0.033840] TSC deadline timer available423worker2 # [ 0.002862] ACPI: FACS 0x000000005FFE0000 000040424lb # [ 0.033844] CPU topo: Max. logical packages: 1425worker1 # [ 0.005374] node 0: [mem 0x0000000000001000-0x000000000009efff]426client # [ 0.034687] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)427lb # [ 0.033845] CPU topo: Max. logical dies: 1428worker1 # [ 0.005375] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]429worker2 # [ 0.002864] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)430lb # [ 0.033845] CPU topo: Max. dies per package: 1431client # [ 0.034689] ACPI: Using ACPI (MADT) for SMP configuration information432lb # [ 0.033848] CPU topo: Max. threads per core: 1433client # [ 0.034690] ACPI: HPET id: 0x8086a201 base: 0xfed00000434worker1 # [ 0.005375] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]435worker2 # [ 0.002865] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)436client # [ 0.034695] TSC deadline timer available437lb # [ 0.033849] CPU topo: Num. cores per package: 1438worker1 # [ 0.005395] On node 0, zone DMA: 1 pages in unavailable ranges439client # [ 0.034700] CPU topo: Max. logical packages: 1440lb # [ 0.033849] CPU topo: Num. threads per package: 1441worker2 # [ 0.002867] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)442worker1 # [ 0.005670] On node 0, zone DMA: 97 pages in unavailable ranges443client # [ 0.034700] CPU topo: Max. logical dies: 1444lb # [ 0.033849] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs445client # [ 0.034701] CPU topo: Max. dies per package: 1446worker1 # [ 0.033085] On node 0, zone DMA32: 40 pages in unavailable ranges447worker2 # [ 0.002868] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)448worker1 # [ 0.033547] ACPI: PM-Timer IO Port: 0x608449lb # [ 0.033870] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()450client # [ 0.034704] CPU topo: Max. threads per core: 1451worker2 # [ 0.002870] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]452client # [ 0.034704] CPU topo: Num. cores per package: 1453worker1 # [ 0.033558] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])454lb # [ 0.033904] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]455client # [ 0.034705] CPU topo: Num. threads per package: 1456worker2 # [ 0.002871] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]457worker1 # [ 0.033585] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23458lb # [ 0.033906] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]459client # [ 0.034705] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs460worker2 # [ 0.002871] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]461worker1 # [ 0.033588] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)462lb # [ 0.033907] [mem 0x60000000-0xafffffff] available for PCI devices463client # [ 0.034730] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()464worker1 # [ 0.033589] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)465lb # [ 0.033909] Booting paravirtualized kernel on KVM466worker2 # [ 0.002872] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]467client # [ 0.034764] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]468worker1 # [ 0.033590] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)469worker2 # [ 0.002872] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]470lb # [ 0.033912] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns471client # [ 0.034766] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]472worker1 # [ 0.033592] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)473worker2 # [ 0.002873] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]474client # [ 0.034767] [mem 0x60000000-0xafffffff] available for PCI devices475lb # [ 0.038443] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1476worker1 # [ 0.033592] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)477worker2 # [ 0.002873] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]478client # [ 0.034769] Booting paravirtualized kernel on KVM479lb # [ 0.041235] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152480worker2 # [ 0.003106] No NUMA configuration found481worker1 # [ 0.033595] ACPI: Using ACPI (MADT) for SMP configuration information482lb # [ 0.041279] kvm-guest: PV spinlocks disabled, single CPU483worker2 # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]484worker1 # [ 0.033596] ACPI: HPET id: 0x8086a201 base: 0xfed00000485client # [ 0.034773] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns486worker1 # [ 0.033599] TSC deadline timer available487worker2 # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]488worker2 # [ 0.005446] Zone ranges:489worker1 # [ 0.033603] CPU topo: Max. logical packages: 1490client # [ 0.039284] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1491worker1 # [ 0.033604] CPU topo: Max. logical dies: 1492worker2 # [ 0.005447] DMA [mem 0x0000000000001000-0x0000000000ffffff]493client # [ 0.041516] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152494worker1 # [ 0.033604] CPU topo: Max. dies per package: 1495worker2 # [ 0.005449] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]496client # [ 0.041561] kvm-guest: PV spinlocks disabled, single CPU497worker1 # [ 0.033608] CPU topo: Max. threads per core: 1498worker2 # [ 0.005450] Normal empty499worker2 # [ 0.005451] Device empty500worker1 # [ 0.033608] CPU topo: Num. cores per package: 1501lb # [ 0.041280] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0502worker1 # [ 0.033609] CPU topo: Num. threads per package: 1503worker2 # [ 0.005452] Movable zone start for each node504worker2 # [ 0.005452] Early memory node ranges505worker1 # [ 0.033609] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs506worker2 # [ 0.005453] node 0: [mem 0x0000000000001000-0x000000000009efff]507worker1 # [ 0.033627] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()508lb # [ 0.041374] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.509worker2 # [ 0.005454] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]510lb # [ 0.041387] random: crng init done511worker1 # [ 0.033657] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]512client # [ 0.041563] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test/init regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration console=ttyS0,115200n8 console=tty0513worker2 # [ 0.005455] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]514lb # [ 0.041388] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes515worker2 # [ 0.005475] On node 0, zone DMA: 1 pages in unavailable ranges516worker1 # [ 0.033659] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]517lb # [ 0.043551] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)518worker2 # [ 0.005745] On node 0, zone DMA: 97 pages in unavailable ranges519worker1 # [ 0.033660] [mem 0x60000000-0xafffffff] available for PCI devices520client # [ 0.041664] Unknown kernel command line parameters "regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration", will be passed to user space.521client # [ 0.041677] random: crng init done522worker1 # [ 0.033661] Booting paravirtualized kernel on KVM523worker2 # [ 0.033537] On node 0, zone DMA32: 40 pages in unavailable ranges524lb # [ 0.044647] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)525lb # [ 0.044691] Fallback order for Node 0: 0526worker2 # [ 0.034021] ACPI: PM-Timer IO Port: 0x608527client # [ 0.041678] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes528worker1 # [ 0.033665] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns529worker2 # [ 0.034035] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])530lb # [ 0.044694] Built 1 zonelists, mobility grouping on. Total pages: 393078531client # [ 0.043879] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)532lb # [ 0.044695] Policy zone: DMA32533worker2 # [ 0.034067] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23534worker1 # [ 0.038160] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1535lb # [ 0.047253] mem auto-init: stack:all(zero), heap alloc:on, heap free:off536client # [ 0.044985] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)537worker2 # [ 0.034069] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)538worker1 # [ 0.040363] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152539client # [ 0.045020] Fallback order for Node 0: 0540lb # [ 0.050600] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1541worker1 # [ 0.040406] kvm-guest: PV spinlocks disabled, single CPU542worker2 # [ 0.034071] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)543lb # [ 0.054341] allocated 3145728 bytes of page_ext544client # [ 0.045023] Built 1 zonelists, mobility grouping on. Total pages: 393078545client # [ 0.045024] Policy zone: DMA32546worker2 # [ 0.034072] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)547lb # [ 0.064803] ftrace: allocating 48787 entries in 192 pages548client # [ 0.048482] mem auto-init: stack:all(zero), heap alloc:on, heap free:off549lb # [ 0.064805] ftrace: allocated 192 pages with 2 groups550worker2 # [ 0.034073] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)551lb # [ 0.065709] Dynamic Preempt: lazy552client # [ 0.051834] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1553worker2 # [ 0.034074] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)554lb # [ 0.065845] rcu: Preemptible hierarchical RCU implementation.555client # [ 0.055222] allocated 3145728 bytes of page_ext556worker2 # [ 0.034076] ACPI: Using ACPI (MADT) for SMP configuration information557lb # [ 0.065845] rcu: RCU event tracing is enabled.558client # [ 0.065640] ftrace: allocating 48787 entries in 192 pages559worker1 # [ 0.040408] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mmcv7i4v7dgji0gb0y4pksgkj6hhjkk-nixos-system-worker1-test/init regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration console=ttyS0,115200n8 console=tty0560worker2 # [ 0.034077] ACPI: HPET id: 0x8086a201 base: 0xfed00000561lb # [ 0.065846] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.562client # [ 0.065642] ftrace: allocated 192 pages with 2 groups563worker2 # [ 0.034083] TSC deadline timer available564client # [ 0.066509] Dynamic Preempt: lazy565lb # [ 0.065847] Trampoline variant of Tasks RCU enabled.566worker2 # [ 0.034087] CPU topo: Max. logical packages: 1567client # [ 0.066671] rcu: Preemptible hierarchical RCU implementation.568lb # [ 0.065848] Rude variant of Tasks RCU enabled.569worker2 # [ 0.034088] CPU topo: Max. logical dies: 1570worker1 # [ 0.040510] Unknown kernel command line parameters "regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration", will be passed to user space.571client # [ 0.066671] rcu: RCU event tracing is enabled.572worker1 # [ 0.040524] random: crng init done573lb # [ 0.065848] Tracing variant of Tasks RCU enabled.574worker2 # [ 0.034088] CPU topo: Max. dies per package: 1575client # [ 0.066672] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.576worker2 # [ 0.034091] CPU topo: Max. threads per core: 1577worker1 # [ 0.040525] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes578lb # [ 0.065849] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.579worker2 # [ 0.034092] CPU topo: Num. cores per package: 1580client # [ 0.066673] Trampoline variant of Tasks RCU enabled.581lb # [ 0.065850] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1582worker1 # [ 0.043012] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)583client # [ 0.066674] Rude variant of Tasks RCU enabled.584worker2 # [ 0.034093] CPU topo: Num. threads per package: 1585client # [ 0.066674] Tracing variant of Tasks RCU enabled.586worker1 # [ 0.044107] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)587lb # [ 0.065906] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588worker2 # [ 0.034093] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs589worker1 # [ 0.044140] Fallback order for Node 0: 0590client # [ 0.066675] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.591worker2 # [ 0.034117] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()592lb # [ 0.065908] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.593worker1 # [ 0.044143] Built 1 zonelists, mobility grouping on. Total pages: 393078594client # [ 0.066676] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1595worker1 # [ 0.044144] Policy zone: DMA32596worker2 # [ 0.034153] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]597lb # [ 0.065909] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.598worker1 # [ 0.046698] mem auto-init: stack:all(zero), heap alloc:on, heap free:off599client # [ 0.066698] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.600worker2 # [ 0.034155] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]601lb # [ 0.070334] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16602worker1 # [ 0.050060] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1603worker2 # [ 0.034157] [mem 0x60000000-0xafffffff] available for PCI devices604client # [ 0.066700] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.605lb # [ 0.070620] rcu: srcu_init: Setting srcu_struct sizes based on contention.606worker1 # [ 0.053489] allocated 3145728 bytes of page_ext607worker2 # [ 0.034158] Booting paravirtualized kernel on KVM608worker1 # [ 0.064026] ftrace: allocating 48787 entries in 192 pages609client # [ 0.066701] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.610lb # [ 0.070626] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns611worker1 # [ 0.064027] ftrace: allocated 192 pages with 2 groups612worker2 # [ 0.034162] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns613client # [ 0.071161] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16614worker1 # [ 0.064886] Dynamic Preempt: lazy615worker1 # [ 0.065030] rcu: Preemptible hierarchical RCU implementation.616lb # [ 0.070742] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)617client # [ 0.071448] rcu: srcu_init: Setting srcu_struct sizes based on contention.618worker2 # [ 0.038706] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1619lb # [ 0.074332] Console: colour VGA+ 80x25620worker1 # [ 0.065031] rcu: RCU event tracing is enabled.621worker2 # [ 0.040953] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152622lb # [ 0.074336] printk: legacy console [tty0] enabled623client # [ 0.071454] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns624worker1 # [ 0.065032] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.625worker2 # [ 0.041003] kvm-guest: PV spinlocks disabled, single CPU626lb # [ 0.103961] printk: legacy console [ttyS0] enabled627worker1 # [ 0.065033] Trampoline variant of Tasks RCU enabled.628client # [ 0.071566] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)629lb # [ 0.200382] ACPI: Core revision 20250807630worker1 # [ 0.065034] Rude variant of Tasks RCU enabled.631client # [ 0.075058] Console: colour VGA+ 80x25632worker1 # [ 0.065034] Tracing variant of Tasks RCU enabled.633client # [ 0.075062] printk: legacy console [tty0] enabled634worker1 # [ 0.065035] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.635client # [ 0.104659] printk: legacy console [ttyS0] enabled636lb # [ 0.201251] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns637worker1 # [ 0.065035] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1638client # [ 0.201345] ACPI: Core revision 20250807639lb # [ 0.202745] APIC: Switch to symmetric I/O mode setup640worker2 # [ 0.041005] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zqb8x0sddns70mbblwgjmcf92snp08cl-nixos-system-worker2-test/init regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration console=ttyS0,115200n8 console=tty0641lb # [ 0.203718] x2apic enabled642worker1 # [ 0.065090] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.643client # [ 0.202193] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns644lb # [ 0.204484] APIC: Switched APIC routing to: physical x2apic645worker1 # [ 0.065092] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.646worker2 # [ 0.041101] Unknown kernel command line parameters "regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration", will be passed to user space.647client # [ 0.203692] APIC: Switch to symmetric I/O mode setup648worker2 # [ 0.041114] random: crng init done649client # [ 0.204638] x2apic enabled650worker1 # [ 0.065093] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.651worker2 # [ 0.041115] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes652worker1 # [ 0.069500] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16653lb # [ 0.206285] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1654client # [ 0.205387] APIC: Switched APIC routing to: physical x2apic655worker2 # [ 0.043304] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)656worker1 # [ 0.069792] rcu: srcu_init: Setting srcu_struct sizes based on contention.657worker2 # [ 0.044710] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)658lb # [ 0.207270] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns659worker2 # [ 0.044744] Fallback order for Node 0: 0660worker1 # [ 0.069799] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns661client # [ 0.207191] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1662worker2 # [ 0.044747] Built 1 zonelists, mobility grouping on. Total pages: 393078663worker2 # [ 0.044748] Policy zone: DMA32664lb # [ 0.208942] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)665worker1 # [ 0.069903] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)666worker1 # [ 0.073433] Console: colour VGA+ 80x25667worker2 # [ 0.047322] mem auto-init: stack:all(zero), heap alloc:on, heap free:off668worker1 # [ 0.073436] printk: legacy console [tty0] enabled669client # [ 0.208173] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns670lb # [ 0.210025] x86/cpu: User Mode Instruction Prevention (UMIP) activated671worker2 # [ 0.050813] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1672worker1 # [ 0.102964] printk: legacy console [ttyS0] enabled673worker2 # [ 0.054227] allocated 3145728 bytes of page_ext674lb # [ 0.211078] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127675worker1 # [ 0.198724] ACPI: Core revision 20250807676client # [ 0.209958] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)677worker2 # [ 0.064572] ftrace: allocating 48787 entries in 192 pages678lb # [ 0.211940] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0679worker2 # [ 0.064575] ftrace: allocated 192 pages with 2 groups680client # [ 0.211042] x86/cpu: User Mode Instruction Prevention (UMIP) activated681worker2 # [ 0.065498] Dynamic Preempt: lazy682worker1 # [ 0.199579] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns683worker2 # [ 0.065640] rcu: Preemptible hierarchical RCU implementation.684client # [ 0.212841] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127685worker1 # [ 0.201047] APIC: Switch to symmetric I/O mode setup686lb # [ 0.212943] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto687worker2 # [ 0.065640] rcu: RCU event tracing is enabled.688worker1 # [ 0.202014] x2apic enabled689client # [ 0.213646] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0690worker2 # [ 0.065641] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.691lb # [ 0.214939] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl692worker2 # [ 0.065643] Trampoline variant of Tasks RCU enabled.693worker1 # [ 0.202747] APIC: Switched APIC routing to: physical x2apic694worker2 # [ 0.065643] Rude variant of Tasks RCU enabled.695lb # [ 0.215940] Transient Scheduler Attacks: Vulnerable: No microcode696client # [ 0.214713] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto697worker2 # [ 0.065643] Tracing variant of Tasks RCU enabled.698lb # [ 0.216939] Spectre V2 : Mitigation: Enhanced / Automatic IBRS699worker2 # [ 0.065644] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.700worker1 # [ 0.204535] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1701client # [ 0.215955] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl702lb # [ 0.217940] Speculative Return Stack Overflow: Mitigation: Safe RET703worker2 # [ 0.065645] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1704client # [ 0.216955] Transient Scheduler Attacks: Vulnerable: No microcode705worker1 # [ 0.205508] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns706worker2 # [ 0.065702] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.707client # [ 0.218672] Spectre V2 : Mitigation: Enhanced / Automatic IBRS708worker2 # [ 0.065704] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.709worker1 # [ 0.207155] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)710client # [ 0.219667] Speculative Return Stack Overflow: Mitigation: Safe RET711worker2 # [ 0.065705] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.712worker1 # [ 0.208239] x86/cpu: User Mode Instruction Prevention (UMIP) activated713lb # [ 0.218939] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization714worker2 # [ 0.070502] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16715worker1 # [ 0.209290] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127716lb # [ 0.220947] Spectre V2 : Enabling IBPB for BPF717worker2 # [ 0.070787] rcu: srcu_init: Setting srcu_struct sizes based on contention.718worker1 # [ 0.210837] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0719lb # [ 0.221940] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier720worker2 # [ 0.070794] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns721client # [ 0.219955] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization722lb # [ 0.222940] active return thunk: srso_alias_return_thunk723client # [ 0.221962] Spectre V2 : Enabling IBPB for BPF724worker1 # [ 0.211156] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto725worker2 # [ 0.070906] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)726worker2 # [ 0.074450] Console: colour VGA+ 80x25727lb # [ 0.223960] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'728worker2 # [ 0.074454] printk: legacy console [tty0] enabled729client # [ 0.222955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier730worker1 # [ 0.213152] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl731worker2 # [ 0.103919] printk: legacy console [ttyS0] enabled732lb # [ 0.224940] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'733client # [ 0.223955] active return thunk: srso_alias_return_thunk734worker1 # [ 0.214153] Transient Scheduler Attacks: Vulnerable: No microcode735worker2 # [ 0.200595] ACPI: Core revision 20250807736lb # [ 0.226651] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'737worker1 # [ 0.215152] Spectre V2 : Mitigation: Enhanced / Automatic IBRS738client # [ 0.224982] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'739lb # [ 0.226939] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'740worker2 # [ 0.201451] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns741worker1 # [ 0.216152] Speculative Return Stack Overflow: Mitigation: Safe RET742client # [ 0.225955] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'743worker2 # [ 0.202927] APIC: Switch to symmetric I/O mode setup744lb # [ 0.227939] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'745client # [ 0.227672] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'746worker2 # [ 0.203865] x2apic enabled747lb # [ 0.228939] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'748client # [ 0.228720] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'749worker2 # [ 0.204623] APIC: Switched APIC routing to: physical x2apic750lb # [ 0.229939] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'751client # [ 0.229742] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'752lb # [ 0.231835] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256753worker1 # [ 0.217152] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization754client # [ 0.230723] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'755worker2 # [ 0.206509] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1756worker1 # [ 0.219159] Spectre V2 : Enabling IBPB for BPF757lb # [ 0.232648] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64758client # [ 0.231730] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'759lb # [ 0.233641] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512760worker1 # [ 0.220153] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier761worker2 # [ 0.207487] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns762client # [ 0.232955] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256763lb # [ 0.234633] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024764worker1 # [ 0.221152] active return thunk: srso_alias_return_thunk765client # [ 0.233955] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64766worker2 # [ 0.209170] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)767lb # [ 0.234939] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8768worker1 # [ 0.222181] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'769client # [ 0.234866] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512770worker2 # [ 0.210253] x86/cpu: User Mode Instruction Prevention (UMIP) activated771worker1 # [ 0.223152] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'772client # [ 0.235687] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024773lb # [ 0.235939] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.774worker2 # [ 0.211304] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127775worker1 # [ 0.224152] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'776client # [ 0.236668] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8777worker2 # [ 0.212844] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0778worker1 # [ 0.225152] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'779client # [ 0.237697] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.780worker1 # [ 0.226152] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'781worker2 # [ 0.213171] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto782worker1 # [ 0.227152] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'783worker2 # [ 0.215167] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl784worker1 # [ 0.228152] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'785worker2 # [ 0.216167] Transient Scheduler Attacks: Vulnerable: No microcode786worker1 # [ 0.229153] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256787worker2 # [ 0.217166] Spectre V2 : Mitigation: Enhanced / Automatic IBRS788worker1 # [ 0.230870] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64789worker2 # [ 0.218112] Speculative Return Stack Overflow: Mitigation: Safe RET790worker1 # [ 0.231869] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512791worker1 # [ 0.232152] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024792worker1 # [ 0.233152] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8793worker1 # [ 0.234152] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.794worker2 # [ 0.219166] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization795worker2 # [ 0.221174] Spectre V2 : Enabling IBPB for BPF796worker2 # [ 0.221940] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier797worker2 # [ 0.223167] active return thunk: srso_alias_return_thunk798worker2 # [ 0.224188] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799worker2 # [ 0.225167] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800worker2 # [ 0.226166] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801worker2 # [ 0.227167] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802worker2 # [ 0.228971] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803worker2 # [ 0.229984] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804worker2 # [ 0.231167] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805worker2 # [ 0.232167] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806worker2 # [ 0.233167] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker2 # [ 0.234167] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker2 # [ 0.235166] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker2 # [ 0.236166] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker2 # [ 0.237167] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811lb # [ 0.271546] Freeing SMP alternatives memory: 44K812lb # [ 0.271943] pid_max: default: 32768 minimum: 301813lb # [ 0.273019] LSM: initializing lsm=capability,landlock,yama,bpf,ima814lb # [ 0.274052] landlock: Up and running.815lb # [ 0.274940] Yama: becoming mindful.816client # [ 0.273323] Freeing SMP alternatives memory: 44K817client # [ 0.273958] pid_max: default: 32768 minimum: 301818lb # [ 0.275796] LSM support for eBPF active819client # [ 0.275047] LSM: initializing lsm=capability,landlock,yama,bpf,ima820client # [ 0.276077] landlock: Up and running.821lb # [ 0.277118] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)822client # [ 0.276956] Yama: becoming mindful.823client # [ 0.278149] LSM support for eBPF active824lb # [ 0.278000] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)825client # [ 0.278903] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)826lb # [ 0.281185] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)827client # [ 0.279992] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828worker1 # [ 0.269222] Freeing SMP alternatives memory: 44K829lb # [ 0.282497] Performance Events: Fam17h+ core perfctr, AMD PMU driver.830worker1 # [ 0.270005] pid_max: default: 32768 minimum: 301831lb # [ 0.282949] ... version: 2832client # [ 0.282847] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)833lb # [ 0.283822] ... bit width: 48834worker1 # [ 0.270851] LSM: initializing lsm=capability,landlock,yama,bpf,ima835lb # [ 0.284480] ... generic counters: 6836worker1 # [ 0.271978] landlock: Up and running.837worker1 # [ 0.272598] Yama: becoming mindful.838lb # [ 0.284942] ... generic bitmap: 000000000000003f839client # [ 0.283500] Performance Events: Fam17h+ core perfctr, AMD PMU driver.840lb # [ 0.285839] ... fixed-purpose counters: 0841worker1 # [ 0.273365] LSM support for eBPF active842client # [ 0.283962] ... version: 2843lb # [ 0.286528] ... fixed-purpose bitmap: 0000000000000000844client # [ 0.284780] ... bit width: 48845worker1 # [ 0.274239] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)846client # [ 0.285611] ... generic counters: 6847lb # [ 0.286942] ... value mask: 0000ffffffffffff848worker1 # [ 0.275191] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849lb # [ 0.287833] ... max period: 00007fffffffffff850client # [ 0.286021] ... generic bitmap: 000000000000003f851client # [ 0.286959] ... fixed-purpose counters: 0852lb # [ 0.288656] ... global_ctrl mask: 000000000000003f853lb # [ 0.289044] signal: max sigframe size: 3376854client # [ 0.287741] ... fixed-purpose bitmap: 0000000000000000855worker1 # [ 0.277688] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)856lb # [ 0.289821] rcu: Hierarchical SRCU implementation.857client # [ 0.287959] ... value mask: 0000ffffffffffff858lb # [ 0.290534] rcu: Max phase no-delay instances is 400.859client # [ 0.288959] ... max period: 00007fffffffffff860worker1 # [ 0.278727] Performance Events: Fam17h+ core perfctr, AMD PMU driver.861worker1 # [ 0.279157] ... version: 2862client # [ 0.289954] ... global_ctrl mask: 000000000000003f863worker1 # [ 0.279854] ... bit width: 48864client # [ 0.291084] signal: max sigframe size: 3376865worker1 # [ 0.280155] ... generic counters: 6866client # [ 0.291966] rcu: Hierarchical SRCU implementation.867worker1 # [ 0.280850] ... generic bitmap: 000000000000003f868worker1 # [ 0.281160] ... fixed-purpose counters: 0869client # [ 0.292847] rcu: Max phase no-delay instances is 400.870worker1 # [ 0.281864] ... fixed-purpose bitmap: 0000000000000000871lb # [ 0.295644] smp: Bringing up secondary CPUs ...872worker1 # [ 0.282155] ... value mask: 0000ffffffffffff873lb # [ 0.295956] smp: Brought up 1 node, 1 CPU874worker1 # [ 0.283032] ... max period: 00007fffffffffff875lb # [ 0.296624] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)876worker1 # [ 0.283803] ... global_ctrl mask: 000000000000003f877worker1 # [ 0.284249] signal: max sigframe size: 3376878worker2 # [ 0.274410] Freeing SMP alternatives memory: 44K879worker1 # [ 0.285020] rcu: Hierarchical SRCU implementation.880lb # [ 0.297092] Memory: 1454624K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)881client # [ 0.298268] smp: Bringing up secondary CPUs ...882worker2 # [ 0.275159] pid_max: default: 32768 minimum: 301883worker1 # [ 0.285732] rcu: Max phase no-delay instances is 400.884client # [ 0.298976] smp: Brought up 1 node, 1 CPU885lb # [ 0.298215] devtmpfs: initialized886worker2 # [ 0.275784] LSM: initializing lsm=capability,landlock,yama,bpf,ima887lb # [ 0.299029] x86/mm: Memory block size: 128MB888client # [ 0.299738] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)889worker2 # [ 0.276995] landlock: Up and running.890worker2 # [ 0.277609] Yama: becoming mindful.891worker2 # [ 0.278383] LSM support for eBPF active892lb # [ 0.300846] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)893client # [ 0.301041] Memory: 1454636K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)894worker2 # [ 0.279291] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)895client # [ 0.302288] devtmpfs: initialized896lb # [ 0.301912] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).897worker1 # [ 0.290761] smp: Bringing up secondary CPUs ...898lb # [ 0.303017] pinctrl core: initialized pinctrl subsystem899worker2 # [ 0.280205] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)900client # [ 0.303196] x86/mm: Memory block size: 128MB901worker1 # [ 0.291168] smp: Brought up 1 node, 1 CPU902worker1 # [ 0.291829] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)903lb # [ 0.304145] PM: RTC time: 04:41:17, date: 2026-09-23904worker2 # [ 0.283064] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)905client # [ 0.305087] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)906worker1 # [ 0.292381] Memory: 1454580K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)907worker1 # [ 0.293390] devtmpfs: initialized908client # [ 0.305998] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).909worker2 # [ 0.283727] Performance Events: Fam17h+ core perfctr, AMD PMU driver.910lb # [ 0.307630] NET: Registered PF_NETLINK/PF_ROUTE protocol family911worker2 # [ 0.284172] ... version: 2912worker1 # [ 0.294200] x86/mm: Memory block size: 128MB913client # [ 0.307066] pinctrl core: initialized pinctrl subsystem914worker2 # [ 0.284847] ... bit width: 48915client # [ 0.308292] PM: RTC time: 04:41:17, date: 2026-09-23916lb # [ 0.308434] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations917worker2 # [ 0.285169] ... generic counters: 6918worker1 # [ 0.296066] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)919worker2 # [ 0.286027] ... generic bitmap: 000000000000003f920lb # [ 0.308964] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations921worker2 # [ 0.286986] ... fixed-purpose counters: 0922worker1 # [ 0.297151] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).923lb # [ 0.310236] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations924worker2 # [ 0.287733] ... fixed-purpose bitmap: 0000000000000000925worker1 # [ 0.298270] pinctrl core: initialized pinctrl subsystem926lb # [ 0.310955] audit: initializing netlink subsys (disabled)927worker2 # [ 0.288171] ... value mask: 0000ffffffffffff928client # [ 0.312117] NET: Registered PF_NETLINK/PF_ROUTE protocol family929worker1 # [ 0.299474] PM: RTC time: 04:41:17, date: 2026-09-23930worker2 # [ 0.289172] ... max period: 00007fffffffffff931lb # [ 0.312291] thermal_sys: Registered thermal governor 'fair_share'932client # [ 0.313576] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations933worker2 # [ 0.290171] ... global_ctrl mask: 000000000000003f934lb # [ 0.312294] thermal_sys: Registered thermal governor 'bang_bang'935lb # [ 0.312944] thermal_sys: Registered thermal governor 'step_wise'936client # [ 0.313986] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations937worker2 # [ 0.291305] signal: max sigframe size: 3376938worker2 # [ 0.292294] rcu: Hierarchical SRCU implementation.939lb # [ 0.313946] audit: type=2000 audit(1790138478.464:1): state=initialized audit_enabled=0 res=1940worker1 # [ 0.303196] NET: Registered PF_NETLINK/PF_ROUTE protocol family941client # [ 0.315254] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations942worker2 # [ 0.293174] rcu: Max phase no-delay instances is 400.943lb # [ 0.315944] thermal_sys: Registered thermal governor 'user_space'944client # [ 0.315970] audit: initializing netlink subsys (disabled)945worker1 # [ 0.304654] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations946lb # [ 0.315945] thermal_sys: Registered thermal governor 'power_allocator'947client # [ 0.317189] thermal_sys: Registered thermal governor 'fair_share'948lb # [ 0.316945] cpuidle: using governor menu949worker1 # [ 0.305178] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations950client # [ 0.317191] thermal_sys: Registered thermal governor 'bang_bang'951lb # [ 0.319147] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5952worker1 # [ 0.306438] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations953client # [ 0.317962] audit: type=2000 audit(1790138478.470:1): state=initialized audit_enabled=0 res=1954worker1 # [ 0.307164] audit: initializing netlink subsys (disabled)955lb # [ 0.320170] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]956client # [ 0.319959] thermal_sys: Registered thermal governor 'step_wise'957worker1 # [ 0.308299] thermal_sys: Registered thermal governor 'fair_share'958worker2 # [ 0.299298] smp: Bringing up secondary CPUs ...959client # [ 0.319960] thermal_sys: Registered thermal governor 'user_space'960lb # [ 0.320945] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry961worker2 # [ 0.300163] smp: Brought up 1 node, 1 CPU962worker1 # [ 0.308301] thermal_sys: Registered thermal governor 'bang_bang'963lb # [ 0.321956] PCI: Using configuration type 1 for base access964client # [ 0.320926] thermal_sys: Registered thermal governor 'power_allocator'965worker1 # [ 0.309156] thermal_sys: Registered thermal governor 'step_wise'966client # [ 0.321770] cpuidle: using governor menu967worker2 # [ 0.300941] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)968lb # [ 0.322987] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.969worker1 # [ 0.310083] thermal_sys: Registered thermal governor 'user_space'970client # [ 0.323512] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5971worker1 # [ 0.310863] audit: type=2000 audit(1790138478.473:1): state=initialized audit_enabled=0 res=1972worker2 # [ 0.302284] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)973client # [ 0.324207] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]974worker1 # [ 0.312158] thermal_sys: Registered thermal governor 'power_allocator'975worker2 # [ 0.303412] devtmpfs: initialized976worker1 # [ 0.312173] cpuidle: using governor menu977client # [ 0.324961] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978worker2 # [ 0.304297] x86/mm: Memory block size: 128MB979client # [ 0.325970] PCI: Using configuration type 1 for base access980worker1 # [ 0.314365] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5981lb # [ 0.328211] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages982client # [ 0.327074] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.983worker2 # [ 0.306149] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)984worker1 # [ 0.315434] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]985lb # [ 0.328943] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page986lb # [ 0.329911] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages987worker1 # [ 0.316158] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry988worker2 # [ 0.307208] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).989worker1 # [ 0.317167] PCI: Using configuration type 1 for base access990worker2 # [ 0.308254] pinctrl core: initialized pinctrl subsystem991worker1 # [ 0.318251] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.992worker2 # [ 0.309367] PM: RTC time: 04:41:17, date: 2026-09-23993client # [ 0.334044] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages994worker2 # [ 0.312984] NET: Registered PF_NETLINK/PF_ROUTE protocol family995client # [ 0.334958] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page996lb # [ 0.336943] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page997worker1 # [ 0.323432] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages998worker2 # [ 0.314504] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations999worker1 # [ 0.324156] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1000worker2 # [ 0.315197] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1001worker2 # [ 0.316451] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1002client # [ 0.339959] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1003worker2 # [ 0.317184] audit: initializing netlink subsys (disabled)1004client # [ 0.340958] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1005worker2 # [ 0.318522] thermal_sys: Registered thermal governor 'fair_share'1006worker1 # [ 0.329155] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1007worker2 # [ 0.318525] thermal_sys: Registered thermal governor 'bang_bang'1008worker1 # [ 0.330155] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1009lb # [ 0.345317] ACPI: Added _OSI(Module Device)1010worker2 # [ 0.319175] audit: type=2000 audit(1790138478.498:1): state=initialized audit_enabled=0 res=11011lb # [ 0.345944] ACPI: Added _OSI(Processor Device)1012lb # [ 0.346704] ACPI: Added _OSI(Processor Aggregator Device)1013worker2 # [ 0.321174] thermal_sys: Registered thermal governor 'step_wise'1014worker2 # [ 0.321176] thermal_sys: Registered thermal governor 'user_space'1015worker2 # [ 0.322112] thermal_sys: Registered thermal governor 'power_allocator'1016worker2 # [ 0.322921] cpuidle: using governor menu1017worker2 # [ 0.324389] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51018worker2 # [ 0.325457] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1019client # [ 0.351439] ACPI: Added _OSI(Module Device)1020client # [ 0.351960] ACPI: Added _OSI(Processor Device)1021worker2 # [ 0.326175] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1022client # [ 0.352690] ACPI: Added _OSI(Processor Aggregator Device)1023worker1 # [ 0.340536] ACPI: Added _OSI(Module Device)1024worker2 # [ 0.327186] PCI: Using configuration type 1 for base access1025worker1 # [ 0.341157] ACPI: Added _OSI(Processor Device)1026worker1 # [ 0.341937] ACPI: Added _OSI(Processor Aggregator Device)1027lb # [ 0.355736] ACPI: 1 ACPI AML tables successfully acquired and loaded1028worker2 # [ 0.328411] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1029lb # [ 0.359801] ACPI: Interpreter enabled1030lb # [ 0.360424] ACPI: PM: (supports S0 S3 S4 S5)1031lb # [ 0.360944] ACPI: Using IOAPIC for interrupt routing1032client # [ 0.360860] ACPI: 1 ACPI AML tables successfully acquired and loaded1033worker2 # [ 0.335493] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1034lb # [ 0.363951] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1035worker2 # [ 0.336171] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1036lb # [ 0.364943] PCI: Using E820 reservations for host bridge windows1037worker1 # [ 0.351949] ACPI: 1 ACPI AML tables successfully acquired and loaded1038worker2 # [ 0.337171] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1039client # [ 0.364928] ACPI: Interpreter enabled1040client # [ 0.365442] ACPI: PM: (supports S0 S3 S4 S5)1041client # [ 0.365960] ACPI: Using IOAPIC for interrupt routing1042lb # [ 0.368079] ACPI: Enabled 2 GPEs in block 00 to 3F1043worker1 # [ 0.355815] ACPI: Interpreter enabled1044client # [ 0.366869] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1045worker1 # [ 0.356172] ACPI: PM: (supports S0 S3 S4 S5)1046worker1 # [ 0.356856] ACPI: Using IOAPIC for interrupt routing1047worker2 # [ 0.344170] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1048worker1 # [ 0.359204] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049client # [ 0.371958] PCI: Using E820 reservations for host bridge windows1050worker1 # [ 0.360155] PCI: Using E820 reservations for host bridge windows1051client # [ 0.373129] ACPI: Enabled 2 GPEs in block 00 to 3F1052lb # [ 0.375711] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1053worker1 # [ 0.363203] ACPI: Enabled 2 GPEs in block 00 to 3F1054lb # [ 0.376851] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1055lb # [ 0.378029] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1056lb # [ 0.379085] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1057worker2 # [ 0.352418] ACPI: Added _OSI(Module Device)1058lb # [ 0.380364] PCI host bridge to bus 0000:001059worker2 # [ 0.353171] ACPI: Added _OSI(Processor Device)1060worker2 # [ 0.353877] ACPI: Added _OSI(Processor Aggregator Device)1061client # [ 0.380462] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1062lb # [ 0.380948] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1063lb # [ 0.381943] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1064client # [ 0.380965] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1065lb # [ 0.382943] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1066client # [ 0.382041] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1067worker1 # [ 0.371488] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1068client # [ 0.383082] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1069lb # [ 0.383943] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1070worker1 # [ 0.372161] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1071client # [ 0.384319] PCI host bridge to bus 0000:001072lb # [ 0.384943] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1073worker1 # [ 0.373235] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1074client # [ 0.384962] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1075lb # [ 0.385943] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1076worker1 # [ 0.374277] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1077client # [ 0.385958] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1078lb # [ 0.386943] pci_bus 0000:00: root bus resource [bus 00-ff]1079worker1 # [ 0.375524] PCI host bridge to bus 0000:001080client # [ 0.386958] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1081worker2 # [ 0.361776] ACPI: 1 ACPI AML tables successfully acquired and loaded1082worker1 # [ 0.376161] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1083lb # [ 0.388076] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1084client # [ 0.387958] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1085worker1 # [ 0.377155] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1086client # [ 0.388958] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1087lb # [ 0.389425] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1088worker1 # [ 0.378155] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1089client # [ 0.389963] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1090worker1 # [ 0.379159] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1091worker2 # [ 0.365570] ACPI: Interpreter enabled1092client # [ 0.391005] pci_bus 0000:00: root bus resource [bus 00-ff]1093worker2 # [ 0.366185] ACPI: PM: (supports S0 S3 S4 S5)1094worker1 # [ 0.380156] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1095worker2 # [ 0.366916] ACPI: Using IOAPIC for interrupt routing1096client # [ 0.391988] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1097worker1 # [ 0.381181] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1098worker1 # [ 0.382156] pci_bus 0000:00: root bus resource [bus 00-ff]1099worker2 # [ 0.367205] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1100client # [ 0.393410] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1101worker2 # [ 0.368170] PCI: Using E820 reservations for host bridge windows1102worker1 # [ 0.383143] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1103worker1 # [ 0.384579] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1104worker2 # [ 0.371270] ACPI: Enabled 2 GPEs in block 00 to 3F1105lb # [ 0.392035] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1106lb # [ 0.392954] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1107lb # [ 0.393907] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1108lb # [ 0.395229] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1109client # [ 0.396024] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1110lb # [ 0.396667] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1111client # [ 0.396970] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1112client # [ 0.397948] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1113worker1 # [ 0.387222] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1114worker2 # [ 0.379982] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1115lb # [ 0.398979] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1116client # [ 0.399136] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1117worker1 # [ 0.388168] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1118lb # [ 0.399822] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1119worker2 # [ 0.380942] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1120worker1 # [ 0.389106] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1121lb # [ 0.400681] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1122client # [ 0.400700] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1123lb # [ 0.400948] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1124worker1 # [ 0.390340] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1125worker2 # [ 0.383808] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1126client # [ 0.402976] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1127lb # [ 0.402719] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128worker2 # [ 0.384293] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1129worker1 # [ 0.391885] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1130client # [ 0.403864] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1131worker2 # [ 0.385515] PCI host bridge to bus 0000:001132client # [ 0.404718] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1133worker2 # [ 0.386173] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1134lb # [ 0.404981] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1135client # [ 0.404964] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1136worker1 # [ 0.394170] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1137worker2 # [ 0.387169] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1138lb # [ 0.405833] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1139worker1 # [ 0.394986] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1140lb # [ 0.406638] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1141worker2 # [ 0.388170] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1142worker1 # [ 0.395870] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1143client # [ 0.406737] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1144worker2 # [ 0.389174] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1145worker1 # [ 0.396161] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1146lb # [ 0.407550] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1147worker2 # [ 0.390195] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1148client # [ 0.408975] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1149worker1 # [ 0.397999] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1150worker2 # [ 0.391169] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1151client # [ 0.409833] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1152worker2 # [ 0.392170] pci_bus 0000:00: root bus resource [bus 00-ff]1153client # [ 0.410712] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1154lb # [ 0.409949] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1155worker1 # [ 0.400162] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1156worker2 # [ 0.393181] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1157lb # [ 0.410949] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1158worker1 # [ 0.401003] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1159client # [ 0.411525] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1160lb # [ 0.411853] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1161worker2 # [ 0.394620] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1162worker1 # [ 0.401852] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1163lb # [ 0.413307] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164worker1 # [ 0.402701] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1165client # [ 0.414125] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1166lb # [ 0.414978] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1167client # [ 0.414964] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1168lb # [ 0.415831] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1169client # [ 0.415968] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1170lb # [ 0.416684] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1171worker1 # [ 0.405163] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1172lb # [ 0.416948] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1173worker1 # [ 0.406162] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1174client # [ 0.417523] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1175worker1 # [ 0.407091] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1176worker2 # [ 0.397265] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1177lb # [ 0.418928] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1178client # [ 0.418965] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1179worker1 # [ 0.408542] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1180worker2 # [ 0.398182] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1181client # [ 0.419823] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1182lb # [ 0.420823] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1183worker2 # [ 0.399159] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1184client # [ 0.420734] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1185worker1 # [ 0.410163] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1186lb # [ 0.421688] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1187worker2 # [ 0.400351] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1188worker1 # [ 0.410985] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1189client # [ 0.421821] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1190lb # [ 0.422497] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1191worker1 # [ 0.411861] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1192worker2 # [ 0.401898] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1193worker1 # [ 0.412161] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1194client # [ 0.423499] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1195lb # [ 0.423986] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1196client # [ 0.424823] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1197lb # [ 0.424964] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1198worker1 # [ 0.414135] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199worker2 # [ 0.404210] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1200client # [ 0.425691] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1201worker2 # [ 0.405152] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1202worker1 # [ 0.415976] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1203lb # [ 0.426913] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1204client # [ 0.426521] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205worker2 # [ 0.406045] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1206worker1 # [ 0.416887] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1207lb # [ 0.428714] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208worker2 # [ 0.407017] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1209worker1 # [ 0.417725] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210client # [ 0.427981] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1211lb # [ 0.428965] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1212client # [ 0.428942] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1213worker2 # [ 0.408759] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1214worker1 # [ 0.419167] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1215lb # [ 0.430513] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker1 # [ 0.420094] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1217client # [ 0.430723] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1218worker2 # [ 0.410211] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1219lb # [ 0.431924] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1220worker2 # [ 0.411067] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1221lb # [ 0.432686] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1222client # [ 0.432836] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1223worker1 # [ 0.421540] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224worker2 # [ 0.411917] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1225client # [ 0.433683] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1226worker1 # [ 0.422902] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1227lb # [ 0.433524] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1228worker2 # [ 0.412709] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1229worker1 # [ 0.423178] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1230client # [ 0.434517] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1231lb # [ 0.435986] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1232worker1 # [ 0.424713] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1233client # [ 0.435938] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1234lb # [ 0.436797] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1235worker2 # [ 0.415178] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1236client # [ 0.436725] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1237lb # [ 0.437642] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1238worker1 # [ 0.426167] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1239worker2 # [ 0.416012] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1240worker1 # [ 0.427087] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1241worker2 # [ 0.416918] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1242lb # [ 0.438516] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1243client # [ 0.438803] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1244worker2 # [ 0.417764] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1245worker1 # [ 0.428489] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1246lb # [ 0.439979] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1247client # [ 0.440977] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1248client # [ 0.441850] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1249lb # [ 0.440831] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1250worker1 # [ 0.430181] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1251worker2 # [ 0.419215] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1252client # [ 0.442753] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1253worker1 # [ 0.431015] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1254lb # [ 0.441963] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1255worker2 # [ 0.420079] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1256worker1 # [ 0.431875] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1257client # [ 0.444307] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1258lb # [ 0.443556] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1259worker2 # [ 0.420971] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1260worker1 # [ 0.432713] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1261worker2 # [ 0.422176] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1262lb # [ 0.444526] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1263client # [ 0.445976] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1264lb # [ 0.445138] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1265worker1 # [ 0.434170] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1266worker2 # [ 0.423707] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1267client # [ 0.446965] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1268worker1 # [ 0.434989] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1269lb # [ 0.446508] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1270client # [ 0.447923] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1271worker2 # [ 0.424995] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1272worker1 # [ 0.435870] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1273lb # [ 0.447134] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1274worker2 # [ 0.425898] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1275client # [ 0.449377] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1276worker1 # [ 0.436730] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1277lb # [ 0.448557] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1278worker2 # [ 0.426791] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1279client # [ 0.450515] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1280worker1 # [ 0.437696] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1281client # [ 0.451155] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1282worker1 # [ 0.438373] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283worker2 # [ 0.428217] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1284client # [ 0.452530] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1285worker2 # [ 0.429193] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1286worker1 # [ 0.439682] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1287client # [ 0.453157] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1288worker1 # [ 0.440346] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1289worker2 # [ 0.430799] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1290client # [ 0.454509] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1291worker1 # [ 0.441700] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1292worker2 # [ 0.431975] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1293worker2 # [ 0.432994] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1294worker2 # [ 0.434692] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1295lb # [ 0.449814] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1296worker2 # [ 0.436121] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1297lb # [ 0.451863] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1298worker2 # [ 0.437191] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1299lb # [ 0.452970] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1300worker2 # [ 0.438710] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1301lb # [ 0.454278] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1302worker2 # [ 0.440184] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1303client # [ 0.455190] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1304lb # [ 0.455185] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1305worker2 # [ 0.441085] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1306worker1 # [ 0.442369] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1307worker2 # [ 0.441904] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1308client # [ 0.457525] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1309lb # [ 0.457729] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1310client # [ 0.458263] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1311lb # [ 0.458559] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1312worker2 # [ 0.442748] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1313worker1 # [ 0.445730] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1314client # [ 0.459243] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1315lb # [ 0.459488] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1316worker2 # [ 0.444727] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1317worker1 # [ 0.446446] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1318client # [ 0.460202] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1319lb # [ 0.460477] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1320worker2 # [ 0.445177] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1321worker1 # [ 0.447456] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1322worker2 # [ 0.446174] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1323client # [ 0.461987] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1324client # [ 0.462882] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1325worker1 # [ 0.448413] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1326worker2 # [ 0.447730] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1327lb # [ 0.463974] ACPI: PCI: Interrupt link LNKA configured for IRQ 101328client # [ 0.464060] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1329worker2 # [ 0.448725] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1330lb # [ 0.465011] ACPI: PCI: Interrupt link LNKB configured for IRQ 101331worker1 # [ 0.450940] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1332lb # [ 0.466028] ACPI: PCI: Interrupt link LNKC configured for IRQ 111333client # [ 0.465590] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1334worker1 # [ 0.451162] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1335worker2 # [ 0.449376] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1336lb # [ 0.467039] ACPI: PCI: Interrupt link LNKD configured for IRQ 111337lb # [ 0.468038] ACPI: PCI: Interrupt link LNKE configured for IRQ 101338worker1 # [ 0.452541] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1339worker2 # [ 0.450815] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1340lb # [ 0.469060] ACPI: PCI: Interrupt link LNKF configured for IRQ 101341worker1 # [ 0.453817] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1342worker2 # [ 0.451623] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1343lb # [ 0.470050] ACPI: PCI: Interrupt link LNKG configured for IRQ 111344client # [ 0.469025] ACPI: PCI: Interrupt link LNKA configured for IRQ 101345lb # [ 0.471078] ACPI: PCI: Interrupt link LNKH configured for IRQ 111346worker2 # [ 0.452721] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1347client # [ 0.470066] ACPI: PCI: Interrupt link LNKB configured for IRQ 101348lb # [ 0.471976] ACPI: PCI: Interrupt link GSIA configured for IRQ 161349client # [ 0.471062] ACPI: PCI: Interrupt link LNKC configured for IRQ 111350lb # [ 0.472935] ACPI: PCI: Interrupt link GSIB configured for IRQ 171351worker1 # [ 0.457230] ACPI: PCI: Interrupt link LNKA configured for IRQ 101352client # [ 0.472054] ACPI: PCI: Interrupt link LNKD configured for IRQ 111353lb # [ 0.473872] ACPI: PCI: Interrupt link GSIC configured for IRQ 181354worker1 # [ 0.458257] ACPI: PCI: Interrupt link LNKB configured for IRQ 101355lb # [ 0.474647] ACPI: PCI: Interrupt link GSID configured for IRQ 191356client # [ 0.473057] ACPI: PCI: Interrupt link LNKE configured for IRQ 101357worker1 # [ 0.459255] ACPI: PCI: Interrupt link LNKC configured for IRQ 111358lb # [ 0.474953] ACPI: PCI: Interrupt link GSIE configured for IRQ 201359client # [ 0.474060] ACPI: PCI: Interrupt link LNKF configured for IRQ 101360worker1 # [ 0.460253] ACPI: PCI: Interrupt link LNKD configured for IRQ 111361lb # [ 0.475875] ACPI: PCI: Interrupt link GSIF configured for IRQ 211362client # [ 0.475057] ACPI: PCI: Interrupt link LNKG configured for IRQ 111363lb # [ 0.476672] ACPI: PCI: Interrupt link GSIG configured for IRQ 221364worker1 # [ 0.461251] ACPI: PCI: Interrupt link LNKE configured for IRQ 101365client # [ 0.476054] ACPI: PCI: Interrupt link LNKH configured for IRQ 111366lb # [ 0.476957] ACPI: PCI: Interrupt link GSIH configured for IRQ 231367worker1 # [ 0.462252] ACPI: PCI: Interrupt link LNKF configured for IRQ 101368client # [ 0.476993] ACPI: PCI: Interrupt link GSIA configured for IRQ 161369worker1 # [ 0.463261] ACPI: PCI: Interrupt link LNKG configured for IRQ 111370client # [ 0.477971] ACPI: PCI: Interrupt link GSIB configured for IRQ 171371lb # [ 0.478993] iommu: Default domain type: Translated1372worker1 # [ 0.464259] ACPI: PCI: Interrupt link LNKH configured for IRQ 111373client # [ 0.478969] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374lb # [ 0.479762] iommu: DMA domain TLB invalidation policy: lazy mode1375worker1 # [ 0.465189] ACPI: PCI: Interrupt link GSIA configured for IRQ 161376client # [ 0.479947] ACPI: PCI: Interrupt link GSID configured for IRQ 191377lb # [ 0.480946] ACPI: bus type USB registered1378worker1 # [ 0.466114] ACPI: PCI: Interrupt link GSIB configured for IRQ 171379client # [ 0.480701] ACPI: PCI: Interrupt link GSIE configured for IRQ 201380lb # [ 0.481680] usbcore: registered new interface driver usbfs1381worker1 # [ 0.466868] ACPI: PCI: Interrupt link GSIC configured for IRQ 181382worker2 # [ 0.453386] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1383client # [ 0.480970] ACPI: PCI: Interrupt link GSIF configured for IRQ 211384lb # [ 0.481959] usbcore: registered new interface driver hub1385worker1 # [ 0.467171] ACPI: PCI: Interrupt link GSID configured for IRQ 191386client # [ 0.481902] ACPI: PCI: Interrupt link GSIG configured for IRQ 221387lb # [ 0.482810] usbcore: registered new device driver usb1388worker2 # [ 0.455684] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1389worker1 # [ 0.468099] ACPI: PCI: Interrupt link GSIE configured for IRQ 201390client # [ 0.482684] ACPI: PCI: Interrupt link GSIH configured for IRQ 231391worker1 # [ 0.468879] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392lb # [ 0.484474] NetLabel: Initializing1393worker2 # [ 0.456471] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1394lb # [ 0.484943] NetLabel: domain hash size = 1281395worker1 # [ 0.469171] ACPI: PCI: Interrupt link GSIG configured for IRQ 221396client # [ 0.484077] iommu: Default domain type: Translated1397lb # [ 0.485682] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1398worker1 # [ 0.470171] ACPI: PCI: Interrupt link GSIH configured for IRQ 231399client # [ 0.484866] iommu: DMA domain TLB invalidation policy: lazy mode1400worker2 # [ 0.457474] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1401lb # [ 0.485983] NetLabel: unlabeled traffic allowed by default1402client # [ 0.486002] ACPI: bus type USB registered1403lb # [ 0.486958] PCI: Using ACPI for IRQ routing1404worker2 # [ 0.458449] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1405client # [ 0.486864] usbcore: registered new interface driver usbfs1406worker1 # [ 0.473252] iommu: Default domain type: Translated1407client # [ 0.487692] usbcore: registered new interface driver hub1408worker2 # [ 0.460089] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1409worker1 # [ 0.474167] iommu: DMA domain TLB invalidation policy: lazy mode1410client # [ 0.487969] usbcore: registered new device driver usb1411worker2 # [ 0.460817] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1412worker1 # [ 0.475414] ACPI: bus type USB registered1413client # [ 0.489667] NetLabel: Initializing1414worker1 # [ 0.476146] usbcore: registered new interface driver usbfs1415client # [ 0.489965] NetLabel: domain hash size = 1281416worker2 # [ 0.461525] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1417worker1 # [ 0.476835] usbcore: registered new interface driver hub1418client # [ 0.490718] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1419worker1 # [ 0.477167] usbcore: registered new device driver usb1420worker2 # [ 0.462735] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1421client # [ 0.490997] NetLabel: unlabeled traffic allowed by default1422worker1 # [ 0.478874] NetLabel: Initializing1423client # [ 0.491965] PCI: Using ACPI for IRQ routing1424worker1 # [ 0.479162] NetLabel: domain hash size = 1281425worker1 # [ 0.479970] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker1 # [ 0.480963] NetLabel: unlabeled traffic allowed by default1427worker1 # [ 0.481876] PCI: Using ACPI for IRQ routing1428worker2 # [ 0.466839] ACPI: PCI: Interrupt link LNKA configured for IRQ 101429worker2 # [ 0.467280] ACPI: PCI: Interrupt link LNKB configured for IRQ 101430worker2 # [ 0.468273] ACPI: PCI: Interrupt link LNKC configured for IRQ 111431worker2 # [ 0.469268] ACPI: PCI: Interrupt link LNKD configured for IRQ 111432worker2 # [ 0.470279] ACPI: PCI: Interrupt link LNKE configured for IRQ 101433worker2 # [ 0.471266] ACPI: PCI: Interrupt link LNKF configured for IRQ 101434worker2 # [ 0.472270] ACPI: PCI: Interrupt link LNKG configured for IRQ 111435worker2 # [ 0.473266] ACPI: PCI: Interrupt link LNKH configured for IRQ 111436worker2 # [ 0.474379] ACPI: PCI: Interrupt link GSIA configured for IRQ 161437worker2 # [ 0.475187] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438worker2 # [ 0.476184] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439worker2 # [ 0.477181] ACPI: PCI: Interrupt link GSID configured for IRQ 191440worker2 # [ 0.478190] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441worker2 # [ 0.479186] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442worker2 # [ 0.480182] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443worker2 # [ 0.481182] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker2 # [ 0.483458] iommu: Default domain type: Translated1445worker2 # [ 0.484182] iommu: DMA domain TLB invalidation policy: lazy mode1446worker2 # [ 0.485442] ACPI: bus type USB registered1447worker2 # [ 0.486226] usbcore: registered new interface driver usbfs1448worker2 # [ 0.487170] usbcore: registered new interface driver hub1449worker2 # [ 0.488080] usbcore: registered new device driver usb1450worker2 # [ 0.489812] NetLabel: Initializing1451worker2 # [ 0.490170] NetLabel: domain hash size = 1281452worker2 # [ 0.490919] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker2 # [ 0.491225] NetLabel: unlabeled traffic allowed by default1454worker2 # [ 0.492181] PCI: Using ACPI for IRQ routing1455lb # [ 0.531507] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456lb # [ 0.531938] pci 0000:00:01.0: vgaarb: bridge control possible1457lb # [ 0.531938] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458lb # [ 0.531946] vgaarb: loaded1459lb # [ 0.532666] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460lb # [ 0.532943] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461lb # [ 0.536024] clocksource: Switched to clocksource kvm-clock1462client # [ 0.537054] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463client # [ 0.537954] pci 0000:00:01.0: vgaarb: bridge control possible1464worker1 # [ 0.525735] pci 0000:00:01.0: vgaarb: setting as boot VGA device1465client # [ 0.537954] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1466client # [ 0.537962] vgaarb: loaded1467lb # [ 0.539519] VFS: Disk quotas dquot_6.6.01468worker1 # [ 0.526151] pci 0000:00:01.0: vgaarb: bridge control possible1469client # [ 0.538710] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470lb # [ 0.540181] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1471worker1 # [ 0.526151] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1472lb # [ 0.541511] pnp: PnP ACPI init1473client # [ 0.538958] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker1 # [ 0.526160] vgaarb: loaded1475lb # [ 0.542252] ACPI: IRQ 4 override to edge(!), high(!)1476worker1 # [ 0.526909] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01477worker1 # [ 0.527157] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1478lb # [ 0.543205] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1479lb # [ 0.544705] pnp: PnP ACPI: found 5 devices1480client # [ 0.543047] clocksource: Switched to clocksource kvm-clock1481worker1 # [ 0.531251] clocksource: Switched to clocksource kvm-clock1482client # [ 0.546657] VFS: Disk quotas dquot_6.6.01483client # [ 0.547393] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1484client # [ 0.548808] pnp: PnP ACPI init1485worker1 # [ 0.534846] VFS: Disk quotas dquot_6.6.01486lb # [ 0.552234] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1487client # [ 0.549656] ACPI: IRQ 4 override to edge(!), high(!)1488worker1 # [ 0.535548] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1489lb # [ 0.553677] clocksource: Switched to clocksource acpi_pm1490worker1 # [ 0.536863] pnp: PnP ACPI init1491client # [ 0.550626] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1492lb # [ 0.554674] NET: Registered PF_INET protocol family1493worker1 # [ 0.537614] ACPI: IRQ 4 override to edge(!), high(!)1494client # [ 0.552221] pnp: PnP ACPI: found 5 devices1495worker1 # [ 0.538532] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1496lb # [ 0.555785] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1497worker1 # [ 0.540008] pnp: PnP ACPI: found 5 devices1498client # [ 0.559899] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1499client # [ 0.561452] clocksource: Switched to clocksource acpi_pm1500worker1 # [ 0.547612] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1501client # [ 0.562456] NET: Registered PF_INET protocol family1502worker1 # [ 0.549005] clocksource: Switched to clocksource acpi_pm1503worker1 # [ 0.549980] NET: Registered PF_INET protocol family1504worker2 # [ 0.536101] pci 0000:00:01.0: vgaarb: setting as boot VGA device1505client # [ 0.563673] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1506worker2 # [ 0.536166] pci 0000:00:01.0: vgaarb: bridge control possible1507worker1 # [ 0.551142] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1508worker2 # [ 0.536166] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1509worker2 # [ 0.536173] vgaarb: loaded1510worker2 # [ 0.536930] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01511worker2 # [ 0.537170] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1512lb # [ 0.571937] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1513lb # [ 0.573447] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1514lb # [ 0.574712] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1515worker2 # [ 0.541258] clocksource: Switched to clocksource kvm-clock1516lb # [ 0.576025] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1517lb # [ 0.577240] TCP: Hash tables configured (established 16384 bind 16384)1518lb # [ 0.578352] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1519lb # [ 0.579639] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1520worker2 # [ 0.545215] VFS: Disk quotas dquot_6.6.01521lb # [ 0.580767] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1522worker2 # [ 0.545939] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1523lb # [ 0.581954] NET: Registered PF_UNIX/PF_LOCAL protocol family1524worker2 # [ 0.547341] pnp: PnP ACPI init1525lb # [ 0.582918] NET: Registered PF_XDP protocol family1526worker2 # [ 0.548148] ACPI: IRQ 4 override to edge(!), high(!)1527lb # [ 0.583757] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1528worker2 # [ 0.549177] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1529client # [ 0.580816] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1530lb # [ 0.584744] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1531worker1 # [ 0.567756] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1532worker2 # [ 0.550731] pnp: PnP ACPI: found 5 devices1533lb # [ 0.585732] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1534client # [ 0.582408] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1535worker1 # [ 0.569262] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1536lb # [ 0.586800] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537client # [ 0.583712] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1538lb # [ 0.587878] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539worker1 # [ 0.570593] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1540client # [ 0.585082] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1541lb # [ 0.588958] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1542worker1 # [ 0.571937] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1543client # [ 0.586365] TCP: Hash tables configured (established 16384 bind 16384)1544lb # [ 0.590635] ACPI: \_SB_.GSIA: Enabled at IRQ 161545worker1 # [ 0.573209] TCP: Hash tables configured (established 16384 bind 16384)1546client # [ 0.587495] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1547worker1 # [ 0.574321] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1548client # [ 0.588815] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1549lb # [ 0.592775] ACPI: \_SB_.GSIB: Enabled at IRQ 171550worker1 # [ 0.575611] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1551client # [ 0.589956] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1552worker2 # [ 0.558416] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1553worker1 # [ 0.576693] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1554client # [ 0.591219] NET: Registered PF_UNIX/PF_LOCAL protocol family1555lb # [ 0.594792] ACPI: \_SB_.GSIC: Enabled at IRQ 181556worker2 # [ 0.559837] clocksource: Switched to clocksource acpi_pm1557worker1 # [ 0.577887] NET: Registered PF_UNIX/PF_LOCAL protocol family1558client # [ 0.592195] NET: Registered PF_XDP protocol family1559worker2 # [ 0.560833] NET: Registered PF_INET protocol family1560worker1 # [ 0.578830] NET: Registered PF_XDP protocol family1561client # [ 0.593102] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1562lb # [ 0.596688] ACPI: \_SB_.GSID: Enabled at IRQ 191563worker1 # [ 0.579669] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1564worker2 # [ 0.562054] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1565client # [ 0.594156] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566worker1 # [ 0.580663] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1567lb # [ 0.598294] PCI: CLS 0 bytes, default 641568client # [ 0.595168] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1569worker1 # [ 0.581682] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1570client # [ 0.596317] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1571lb # [ 0.599236] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1572worker1 # [ 0.582791] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1573client # [ 0.597405] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1574lb # [ 0.600938] Trying to unpack rootfs image as initramfs...1575worker1 # [ 0.583888] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1576client # [ 0.598492] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1577worker1 # [ 0.584971] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1578client # [ 0.600388] ACPI: \_SB_.GSIA: Enabled at IRQ 161579worker1 # [ 0.586703] ACPI: \_SB_.GSIA: Enabled at IRQ 161580client # [ 0.602726] ACPI: \_SB_.GSIB: Enabled at IRQ 171581worker1 # [ 0.589114] ACPI: \_SB_.GSIB: Enabled at IRQ 171582worker1 # [ 0.591014] ACPI: \_SB_.GSIC: Enabled at IRQ 181583client # [ 0.604995] ACPI: \_SB_.GSIC: Enabled at IRQ 181584worker1 # [ 0.592998] ACPI: \_SB_.GSID: Enabled at IRQ 191585client # [ 0.607005] ACPI: \_SB_.GSID: Enabled at IRQ 191586worker1 # [ 0.594741] PCI: CLS 0 bytes, default 641587client # [ 0.608717] PCI: CLS 0 bytes, default 641588worker1 # [ 0.595640] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1589client # [ 0.609667] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1590worker2 # [ 0.579044] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1591worker1 # [ 0.597403] Trying to unpack rootfs image as initramfs...1592client # [ 0.611406] Trying to unpack rootfs image as initramfs...1593worker2 # [ 0.580590] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1594worker2 # [ 0.581931] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1595worker2 # [ 0.583358] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1596worker2 # [ 0.584728] TCP: Hash tables configured (established 16384 bind 16384)1597worker2 # [ 0.585899] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1598worker2 # [ 0.587217] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1599worker2 # [ 0.588393] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1600worker2 # [ 0.589658] NET: Registered PF_UNIX/PF_LOCAL protocol family1601worker2 # [ 0.590666] NET: Registered PF_XDP protocol family1602worker2 # [ 0.591548] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1603worker2 # [ 0.592591] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1604worker2 # [ 0.593649] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1605worker2 # [ 0.594797] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1606worker2 # [ 0.595937] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607worker2 # [ 0.597084] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker2 # [ 0.598953] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker2 # [ 0.601337] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker2 # [ 0.603561] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker2 # [ 0.605573] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker2 # [ 0.607294] PCI: CLS 0 bytes, default 641613lb # [ 0.643830] Initialise system trusted keyrings1614worker2 # [ 0.608288] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1615worker2 # [ 0.610090] Trying to unpack rootfs image as initramfs...1616lb # [ 0.646398] workingset: timestamp_bits=40 max_order=19 bucket_order=01617client # [ 0.655084] Initialise system trusted keyrings1618worker1 # [ 0.643760] Initialise system trusted keyrings1619client # [ 0.657578] workingset: timestamp_bits=40 max_order=19 bucket_order=01620worker1 # [ 0.644695] workingset: timestamp_bits=40 max_order=19 bucket_order=01621lb # [ 0.667294] Key type asymmetric registered1622lb # [ 0.670347] Asymmetric key parser 'x509' registered1623lb # [ 0.671155] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1624lb # [ 0.674467] io scheduler mq-deadline registered1625lb # [ 0.675207] io scheduler kyber registered1626lb # [ 0.679353] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1627lb # [ 0.680760] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1628client # [ 0.682569] Key type asymmetric registered1629worker1 # [ 0.669172] Key type asymmetric registered1630client # [ 0.683313] Asymmetric key parser 'x509' registered1631worker1 # [ 0.669980] Asymmetric key parser 'x509' registered1632lb # [ 0.687535] Linux agpgart interface v0.1031633lb # [ 0.688263] ACPI: bus type drm_connector registered1634client # [ 0.686592] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1635worker2 # [ 0.655560] Initialise system trusted keyrings1636lb # [ 0.691680] usbcore: registered new interface driver usbserial_generic1637worker1 # [ 0.674444] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638client # [ 0.689572] io scheduler mq-deadline registered1639lb # [ 0.692791] usbserial: USB Serial support registered for generic1640client # [ 0.690359] io scheduler kyber registered1641worker2 # [ 0.658567] workingset: timestamp_bits=40 max_order=19 bucket_order=01642worker1 # [ 0.678452] io scheduler mq-deadline registered1643lb # [ 0.695350] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1644worker1 # [ 0.679428] io scheduler kyber registered1645client # [ 0.693218] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1646lb # [ 0.695350] Please enable it if your BIOS has the CPPC option.1647lb # [ 0.697690] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1648client # [ 0.696651] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1649worker1 # [ 0.683074] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1650lb # [ 0.702496] drop_monitor: Initializing network drop monitor service1651lb # [ 0.703737] NET: Registered PF_INET6 protocol family1652worker1 # [ 0.686732] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1653client # [ 0.701815] Linux agpgart interface v0.1031654client # [ 0.702596] ACPI: bus type drm_connector registered1655lb # [ 0.706853] Segment Routing with IPv61656lb # [ 0.709373] In-situ OAM (IOAM) with IPv61657client # [ 0.705908] usbcore: registered new interface driver usbserial_generic1658worker1 # [ 0.692604] Linux agpgart interface v0.1031659lb # [ 0.710467] IPI shorthand broadcast: enabled1660client # [ 0.707035] usbserial: USB Serial support registered for generic1661worker1 # [ 0.693574] ACPI: bus type drm_connector registered1662client # [ 0.709533] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1663client # [ 0.709533] Please enable it if your BIOS has the CPPC option.1664worker1 # [ 0.696981] usbcore: registered new interface driver usbserial_generic1665worker2 # [ 0.680458] Key type asymmetric registered1666client # [ 0.711827] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1667worker2 # [ 0.681216] Asymmetric key parser 'x509' registered1668worker2 # [ 0.682111] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1669lb # [ 0.718769] sched_clock: Marking stable (584014458, 134312978)->(788620747, -70293311)1670worker1 # [ 0.698347] usbserial: USB Serial support registered for generic1671client # [ 0.716691] drop_monitor: Initializing network drop monitor service1672client # [ 0.717917] NET: Registered PF_INET6 protocol family1673worker1 # [ 0.703644] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1674worker2 # [ 0.687510] io scheduler mq-deadline registered1675worker1 # [ 0.703644] Please enable it if your BIOS has the CPPC option.1676worker2 # [ 0.688271] io scheduler kyber registered1677lb # [ 0.723552] registered taskstats version 11678client # [ 0.720987] Segment Routing with IPv61679lb # [ 0.724570] Loading compiled-in X.509 certificates1680worker2 # [ 0.692414] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1681client # [ 0.724546] In-situ OAM (IOAM) with IPv61682client # [ 0.725617] IPI shorthand broadcast: enabled1683worker2 # [ 0.693740] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1684worker1 # [ 0.711396] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1685worker1 # [ 0.713076] drop_monitor: Initializing network drop monitor service1686worker2 # [ 0.698818] Linux agpgart interface v0.1031687worker1 # [ 0.716545] NET: Registered PF_INET6 protocol family1688worker2 # [ 0.699641] ACPI: bus type drm_connector registered1689worker1 # [ 0.719703] Segment Routing with IPv61690worker1 # [ 0.720436] In-situ OAM (IOAM) with IPv61691client # [ 0.733811] sched_clock: Marking stable (599013206, 134500728)->(805725211, -72211277)1692worker2 # [ 0.702910] usbcore: registered new interface driver usbserial_generic1693worker2 # [ 0.704026] usbserial: USB Serial support registered for generic1694worker1 # [ 0.723663] IPI shorthand broadcast: enabled1695client # [ 0.738723] registered taskstats version 11696client # [ 0.739714] Loading compiled-in X.509 certificates1697worker2 # [ 0.707405] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1698lb # [ 0.743341] Demotion targets for Node 0: null1699worker2 # [ 0.707405] Please enable it if your BIOS has the CPPC option.1700lb # [ 0.744229] Key type .fscrypt registered1701worker2 # [ 0.709764] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1702lb # [ 0.747331] Key type fscrypt-provisioning registered1703lb # [ 0.748290] ima: No TPM chip found, activating TPM-bypass!1704lb # [ 0.749227] ima: Allocated hash algorithm: sha11705worker1 # [ 0.731962] sched_clock: Marking stable (598013882, 133352743)->(805681532, -74314907)1706worker2 # [ 0.714547] drop_monitor: Initializing network drop monitor service1707lb # [ 0.750022] ima: No architecture policies found1708worker2 # [ 0.715719] NET: Registered PF_INET6 protocol family1709lb # [ 0.752339] PM: Magic number: 10:302:6671710worker1 # [ 0.735500] registered taskstats version 11711worker1 # [ 0.736529] Loading compiled-in X.509 certificates1712worker2 # [ 0.718896] Segment Routing with IPv61713lb # [ 0.753974] RAS: Correctable Errors collector initialized.1714worker2 # [ 0.719600] In-situ OAM (IOAM) with IPv61715worker2 # [ 0.722706] IPI shorthand broadcast: enabled1716client # [ 0.758524] Demotion targets for Node 0: null1717client # [ 0.759462] Key type .fscrypt registered1718lb # [ 0.763003] clk: Disabling unused clocks1719worker2 # [ 0.728865] sched_clock: Marking stable (594014623, 134372398)->(805979190, -77592169)1720client # [ 0.761529] Key type fscrypt-provisioning registered1721client # [ 0.762494] ima: No TPM chip found, activating TPM-bypass!1722lb # [ 0.766365] PM: genpd: Disabling unused power domains1723client # [ 0.763448] ima: Allocated hash algorithm: sha11724worker2 # [ 0.735613] registered taskstats version 11725client # [ 0.767564] ima: No architecture policies found1726worker1 # [ 0.754386] Demotion targets for Node 0: null1727worker2 # [ 0.736619] Loading compiled-in X.509 certificates1728client # [ 0.768578] PM: Magic number: 10:302:6671729client # [ 0.770254] RAS: Correctable Errors collector initialized.1730worker1 # [ 0.757424] Key type .fscrypt registered1731worker1 # [ 0.758119] Key type fscrypt-provisioning registered1732worker1 # [ 0.759179] ima: No TPM chip found, activating TPM-bypass!1733worker1 # [ 0.761397] ima: Allocated hash algorithm: sha11734worker1 # [ 0.762305] ima: No architecture policies found1735worker1 # [ 0.764382] PM: Magic number: 10:302:6671736worker1 # [ 0.766141] RAS: Correctable Errors collector initialized.1737client # [ 0.782212] clk: Disabling unused clocks1738client # [ 0.782983] PM: genpd: Disabling unused power domains1739worker2 # [ 0.753062] Demotion targets for Node 0: null1740worker2 # [ 0.756443] Key type .fscrypt registered1741worker2 # [ 0.757134] Key type fscrypt-provisioning registered1742worker2 # [ 0.758126] ima: No TPM chip found, activating TPM-bypass!1743worker1 # [ 0.777209] clk: Disabling unused clocks1744worker2 # [ 0.760404] ima: Allocated hash algorithm: sha11745worker2 # [ 0.761207] ima: No architecture policies found1746worker1 # [ 0.779388] PM: genpd: Disabling unused power domains1747worker2 # [ 0.763399] PM: Magic number: 10:302:6671748worker2 # [ 0.765028] RAS: Correctable Errors collector initialized.1749worker2 # [ 0.774075] clk: Disabling unused clocks1750worker2 # [ 0.778420] PM: genpd: Disabling unused power domains1751lb # [ 0.940281] Freeing initrd memory: 29064K1752lb # [ 0.943590] Freeing unused decrypted memory: 2028K1753lb # [ 0.946224] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.947345] Write protecting the kernel read-only data: 32768k1755lb # [ 0.949210] Freeing unused kernel image (text/rodata gap) memory: 1184K1756lb # [ 0.950797] Freeing unused kernel image (rodata/data gap) memory: 720K1757client # [ 0.954554] Freeing initrd memory: 29056K1758client # [ 0.957860] Freeing unused decrypted memory: 2028K1759client # [ 0.960634] Freeing unused kernel image (initmem) memory: 3652K1760client # [ 0.961769] Write protecting the kernel read-only data: 32768k1761worker1 # [ 0.949231] Freeing initrd memory: 29056K1762client # [ 0.963721] Freeing unused kernel image (text/rodata gap) memory: 1184K1763client # [ 0.965297] Freeing unused kernel image (rodata/data gap) memory: 720K1764worker1 # [ 0.952465] Freeing unused decrypted memory: 2028K1765worker1 # [ 0.955114] Freeing unused kernel image (initmem) memory: 3652K1766worker1 # [ 0.956191] Write protecting the kernel read-only data: 32768k1767worker1 # [ 0.958061] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker1 # [ 0.959558] Freeing unused kernel image (rodata/data gap) memory: 720K1769worker2 # [ 0.946273] Freeing initrd memory: 29060K1770worker2 # [ 0.949527] Freeing unused decrypted memory: 2028K1771worker2 # [ 0.952192] Freeing unused kernel image (initmem) memory: 3652K1772worker2 # [ 0.953293] Write protecting the kernel read-only data: 32768k1773worker2 # [ 0.955207] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker2 # [ 0.956746] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 1.001859] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 1.002985] Run /init as init process1777lb # [ 1.013929] systemd[1]: Inserted module 'autofs4'1778client # [ 1.016316] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779client # [ 1.017483] Run /init as init process1780worker1 # [ 1.010323] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781worker1 # [ 1.011437] Run /init as init process1782client # [ 1.028442] systemd[1]: Inserted module 'autofs4'1783lb # [ 1.033821] fuse: init (API version 7.45)1784worker1 # [ 1.022421] systemd[1]: Inserted module 'autofs4'1785lb # [ 1.041176] ACPI: \_SB_.GSIG: Enabled at IRQ 221786worker2 # [ 1.007551] x86/mm: Checked W+X mappings: passed, no W+X pages found.1787worker2 # [ 1.008737] Run /init as init process1788lb # [ 1.043798] ACPI: \_SB_.GSIH: Enabled at IRQ 231789lb # [ 1.046787] ACPI: \_SB_.GSIE: Enabled at IRQ 201790lb # [ 1.049111] ACPI: \_SB_.GSIF: Enabled at IRQ 211791client # [ 1.048611] fuse: init (API version 7.45)1792lb # [ 1.054687] virtiofs virtio5: discovered new tag: nix-store1793worker2 # [ 1.019917] systemd[1]: Inserted module 'autofs4'1794lb # [ 1.056260] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1795client # [ 1.056184] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker1 # [ 1.042537] fuse: init (API version 7.45)1797client # [ 1.058684] ACPI: \_SB_.GSIH: Enabled at IRQ 231798lb # [ 1.064823] virtiofs virtio6: discovered new tag: shared1799client # [ 1.061803] ACPI: \_SB_.GSIE: Enabled at IRQ 201800lb # [ 1.066596] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1801client # [ 1.063820] ACPI: \_SB_.GSIF: Enabled at IRQ 211802worker1 # [ 1.050033] ACPI: \_SB_.GSIG: Enabled at IRQ 221803worker1 # [ 1.052548] ACPI: \_SB_.GSIH: Enabled at IRQ 231804lb # [ 1.070078] virtiofs virtio7: discovered new tag: xchg1805lb # [ 1.071654] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1806worker1 # [ 1.055537] ACPI: \_SB_.GSIE: Enabled at IRQ 201807client # [ 1.069379] virtiofs virtio5: discovered new tag: nix-store1808client # [ 1.071052] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1809worker1 # [ 1.057527] ACPI: \_SB_.GSIF: Enabled at IRQ 211810worker2 # [ 1.039930] fuse: init (API version 7.45)1811worker1 # [ 1.062837] virtiofs virtio5: discovered new tag: nix-store1812client # [ 1.077922] virtiofs virtio6: discovered new tag: shared1813worker1 # [ 1.064426] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1814worker2 # [ 1.047306] ACPI: \_SB_.GSIG: Enabled at IRQ 221815client # [ 1.079810] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1816worker2 # [ 1.049889] ACPI: \_SB_.GSIH: Enabled at IRQ 231817client # [ 1.083979] virtiofs virtio7: discovered new tag: xchg1818worker2 # [ 1.052805] ACPI: \_SB_.GSIE: Enabled at IRQ 201819client # [ 1.085443] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1820worker2 # [ 1.054773] ACPI: \_SB_.GSIF: Enabled at IRQ 211821worker1 # [ 1.072345] virtiofs virtio6: discovered new tag: shared1822worker1 # [ 1.074002] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823worker1 # [ 1.077170] virtiofs virtio7: discovered new tag: xchg1824worker2 # [ 1.059826] virtiofs virtio5: discovered new tag: nix-store1825worker1 # [ 1.078645] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1826worker2 # [ 1.061444] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1827lb # [ 1.096555] systemd[1]: Successfully made /usr/ read-only.1828worker2 # [ 1.068759] virtiofs virtio6: discovered new tag: shared1829worker2 # [ 1.070515] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830worker2 # [ 1.075594] virtiofs virtio7: discovered new tag: xchg1831client # [ 1.108296] systemd[1]: Successfully made /usr/ read-only.1832worker2 # [ 1.077205] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833worker1 # [ 1.102655] systemd[1]: Successfully made /usr/ read-only.1834worker2 # [ 1.102689] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.433290] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.448817] systemd[1]: Detected virtualization kvm.1837lb # [ 1.451669] systemd[1]: Detected architecture x86-64.1838lb # [ 1.454426] systemd[1]: Running in initrd.1839client # [ 1.444614] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840lb # [ 1.457635] systemd[1]: Initializing machine ID from random generator.1841client # [ 1.456714] systemd[1]: Detected virtualization kvm.1842worker1 # [ 1.438428] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1843lb # [ 1.461517] systemd[1]: Hostname set to <lb>.1844client # [ 1.458854] systemd[1]: Detected architecture x86-64.1845worker1 # [ 1.445260] systemd[1]: Detected virtualization kvm.1846client # [ 1.461173] systemd[1]: Running in initrd.1847worker1 # [ 1.447437] systemd[1]: Detected architecture x86-64.1848client # [ 1.462346] systemd[1]: Initializing machine ID from random generator.1849client # [ 1.463575] systemd[1]: Hostname set to <client>.1850worker1 # [ 1.449531] systemd[1]: Running in initrd.1851worker1 # [ 1.452044] systemd[1]: Initializing machine ID from random generator.1852worker1 # [ 1.454963] systemd[1]: Hostname set to <worker1>.1853worker2 # [ 1.439289] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker2 # [ 1.451331] systemd[1]: Detected virtualization kvm.1855worker2 # [ 1.453730] systemd[1]: Detected architecture x86-64.1856worker2 # [ 1.455866] systemd[1]: Running in initrd.1857worker2 # [ 1.458327] systemd[1]: Initializing machine ID from random generator.1858worker2 # [ 1.461225] systemd[1]: Hostname set to <worker2>.1859client # [ 1.657007] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.671953] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.695164] systemd[1]: Queued start job for default target Initrd Default Target.1862worker1 # [ 1.684063] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.699834] systemd[1]: Created slice Slice /system/modprobe.1864client # [ 1.701038] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865client # [ 1.702492] systemd[1]: Expecting device /dev/disk/by-label/nixos...1866worker2 # [ 1.671606] systemd[1]: bpf-restrict-fs: LSM BPF program attached1867client # [ 1.703724] systemd[1]: Reached target Path Units.1868client # [ 1.704605] systemd[1]: Reached target Slice Units.1869client # [ 1.705481] systemd[1]: Reached target Swaps.1870client # [ 1.706303] systemd[1]: Reached target Timer Units.1871client # [ 1.707298] systemd[1]: Listening on D-Bus System Message Bus Socket.1872client # [ 1.708540] systemd[1]: Listening on Journal Socket (/dev/log).1873lb # [ 1.711948] systemd[1]: Queued start job for default target Initrd Default Target.1874client # [ 1.709687] systemd[1]: Listening on Journal Sockets.1875client # [ 1.710697] systemd[1]: Listening on udev Control Socket.1876client # [ 1.711759] systemd[1]: Listening on udev Kernel Socket.1877lb # [ 1.715734] systemd[1]: Created slice Slice /system/modprobe.1878client # [ 1.712781] systemd[1]: Reached target Socket Units.1879lb # [ 1.716910] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1880client # [ 1.714633] systemd[1]: Starting Create List of Static Device Nodes...1881lb # [ 1.718380] systemd[1]: Expecting device /dev/disk/by-label/nixos...1882lb # [ 1.719511] systemd[1]: Reached target Path Units.1883lb # [ 1.720424] systemd[1]: Reached target Slice Units.1884lb # [ 1.721286] systemd[1]: Reached target Swaps.1885client # [ 1.718337] systemd[1]: Starting Load Kernel Module configfs...1886lb # [ 1.722098] systemd[1]: Reached target Timer Units.1887lb # [ 1.723068] systemd[1]: Listening on D-Bus System Message Bus Socket.1888lb # [ 1.724339] systemd[1]: Listening on Journal Socket (/dev/log).1889lb # [ 1.725532] systemd[1]: Listening on Journal Sockets.1890lb # [ 1.726513] systemd[1]: Listening on udev Control Socket.1891lb # [ 1.727572] systemd[1]: Listening on udev Kernel Socket.1892lb # [ 1.728641] systemd[1]: Reached target Socket Units.1893lb # [ 1.730557] systemd[1]: Starting Create List of Static Device Nodes...1894lb # [ 1.734410] systemd[1]: Starting Load Kernel Module configfs...1895client # [ 1.727987] systemd[1]: Starting Journal Service...1896worker2 # [ 1.709050] systemd[1]: Queued start job for default target Initrd Default Target.1897worker2 # [ 1.713693] systemd[1]: Created slice Slice /system/modprobe.1898worker2 # [ 1.714919] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1899worker2 # [ 1.716404] systemd[1]: Expecting device /dev/disk/by-label/nixos...1900worker2 # [ 1.717536] systemd[1]: Reached target Path Units.1901worker2 # [ 1.718447] systemd[1]: Reached target Slice Units.1902worker2 # [ 1.719299] systemd[1]: Reached target Swaps.1903lb # [ 1.743524] systemd[1]: Starting Journal Service...1904worker2 # [ 1.720129] systemd[1]: Reached target Timer Units.1905worker2 # [ 1.721093] systemd[1]: Listening on D-Bus System Message Bus Socket.1906worker1 # [ 1.739312] systemd[1]: Queued start job for default target Initrd Default Target.1907client # [ 1.753691] systemd[1]: Starting Load Kernel Modules...1908worker2 # [ 1.722429] systemd[1]: Listening on Journal Socket (/dev/log).1909worker2 # [ 1.723556] systemd[1]: Listening on Journal Sockets.1910worker2 # [ 1.724569] systemd[1]: Listening on udev Control Socket.1911worker2 # [ 1.725618] systemd[1]: Listening on udev Kernel Socket.1912worker1 # [ 1.743741] systemd[1]: Created slice Slice /system/modprobe.1913worker2 # [ 1.726592] systemd[1]: Reached target Socket Units.1914client # [ 1.757621] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1915worker1 # [ 1.745021] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1916worker2 # [ 1.728267] systemd[1]: Starting Create List of Static Device Nodes...1917worker1 # [ 1.746520] systemd[1]: Expecting device /dev/disk/by-label/nixos...1918worker1 # [ 1.747765] systemd[1]: Reached target Path Units.1919worker1 # [ 1.748679] systemd[1]: Reached target Slice Units.1920worker1 # [ 1.749567] systemd[1]: Reached target Swaps.1921worker1 # [ 1.750414] systemd[1]: Reached target Timer Units.1922worker2 # [ 1.732503] systemd[1]: Starting Load Kernel Module configfs...1923client # [ 1.764296] systemd[1]: Starting Coldplug All udev Devices...1924worker1 # [ 1.751407] systemd[1]: Listening on D-Bus System Message Bus Socket.1925lb # [ 1.769520] systemd[1]: Starting Load Kernel Modules...1926worker1 # [ 1.752667] systemd[1]: Listening on Journal Socket (/dev/log).1927worker1 # [ 1.753815] systemd[1]: Listening on Journal Sockets.1928worker1 # [ 1.754828] systemd[1]: Listening on udev Control Socket.1929worker1 # [ 1.755922] systemd[1]: Listening on udev Kernel Socket.1930lb # [ 1.773441] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1931worker1 # [ 1.756943] systemd[1]: Reached target Socket Units.1932worker1 # [ 1.758875] systemd[1]: Starting Create List of Static Device Nodes...1933client # [ 1.777023] systemd-journald[66]: Collecting audit messages is disabled.1934lb # [ 1.780411] systemd[1]: Starting Coldplug All udev Devices...1935worker1 # [ 1.763447] systemd[1]: Starting Load Kernel Module configfs...1936client # [ 1.778474] systemd[1]: Finished Create List of Static Device Nodes.1937worker2 # [ 1.741997] systemd[1]: Starting Journal Service...1938client # [ 1.784967] systemd[1]: modprobe@configfs.service: Deactivated successfully.1939lb # [ 1.793572] systemd-journald[66]: Collecting audit messages is disabled.1940client # [ 1.791919] systemd[1]: Finished Load Kernel Module configfs.1941lb # [ 1.795487] systemd[1]: Finished Create List of Static Device Nodes.1942worker2 # [ 1.766575] systemd[1]: Starting Load Kernel Modules...1943client # [ 1.797920] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1944worker1 # [ 1.773460] systemd[1]: Starting Journal Service...1945lb # [ 1.803002] systemd[1]: modprobe@configfs.service: Deactivated successfully.1946worker2 # [ 1.770485] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1947lb # [ 1.808780] systemd[1]: Finished Load Kernel Module configfs.1948worker2 # [ 1.777159] systemd[1]: Starting Coldplug All udev Devices...1949client # [ 1.809597] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1950lb # [ 1.814780] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1951worker1 # [ 1.799013] systemd[1]: Starting Load Kernel Modules...1952worker1 # [ 1.803503] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1953worker2 # [ 1.789942] systemd-journald[66]: Collecting audit messages is disabled.1954client # [ 1.821682] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1955worker2 # [ 1.791098] systemd[1]: Finished Create List of Static Device Nodes.1956lb # [ 1.826419] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1957worker2 # [ 1.795238] systemd[1]: modprobe@configfs.service: Deactivated successfully.1958worker1 # [ 1.813602] systemd[1]: Starting Coldplug All udev Devices...1959client # [ 1.829602] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1960worker2 # [ 1.802765] systemd[1]: Finished Load Kernel Module configfs.1961lb # [ 1.837505] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1962worker1 # [ 1.825921] systemd-journald[66]: Collecting audit messages is disabled.1963worker2 # [ 1.808782] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1964worker1 # [ 1.827882] systemd[1]: Finished Create List of Static Device Nodes.1965lb # [ 1.848552] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966worker1 # [ 1.833054] systemd[1]: modprobe@configfs.service: Deactivated successfully.1967client # [ 1.849048] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1968worker2 # [ 1.818450] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1969worker1 # [ 1.840786] systemd[1]: Finished Load Kernel Module configfs.1970client # [ 1.858708] systemd[1]: Starting Create Static Device Nodes in /dev...1971worker1 # [ 1.845823] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1972worker2 # [ 1.831112] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1973lb # [ 1.869931] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1974client # [ 1.870038] systemd[1]: Finished Load Kernel Modules.1975worker1 # [ 1.858451] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1976worker2 # [ 1.840559] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1977lb # [ 1.878548] systemd[1]: Starting Create Static Device Nodes in /dev...1978client # [ 1.879615] systemd[1]: Starting Apply Kernel Variables...1979worker1 # [ 1.869884] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1980lb # [ 1.890404] systemd[1]: Finished Load Kernel Modules.1981worker2 # [ 1.855911] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1982worker1 # [ 1.879404] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1983lb # [ 1.899540] systemd[1]: Starting Apply Kernel Variables...1984worker2 # [ 1.864639] systemd[1]: Starting Create Static Device Nodes in /dev...1985client # [ 1.898827] systemd[1]: Finished Create Static Device Nodes in /dev.1986client # [ 1.766324] systemd-modules-load[68]: Inserted module 'dm_mod'1987client # [ 1.768379] systemd-modules-load[68]: Inserted module 'virtio_balloon'1988client # [ 1.904742] systemd[1]: Started Journal Service.1989client # [ 1.771297] systemd-modules-load[68]: Inserted module 'virtio_gpu'1990worker2 # [ 1.881484] systemd[1]: Finished Load Kernel Modules.1991lb # [ 1.783331] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.918892] systemd[1]: Started Journal Service.1992client # [ 1.782188] systemd[1]: Finished Apply Kernel Variables.1993worker1 # [ 1.902448] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1994lb # 1995lb # [ 1.788358] systemd-modules-load[68]: Inserted module 'virtio_balloon'1996client # [ 1.785073] systemd[1]: Reached target Preparation for Local File Systems.1997client # [ 1.786040] systemd[1]: Reached target Local File Systems.1998lb # [ 1.789426] systemd-modules-load[68]: Inserted module 'virtio_gpu'1999worker2 # [ 1.889583] systemd[1]: Starting Apply Kernel Variables...2000client # [ 1.788975] systemd[1]: Starting Create System Files and Directories...
loading 2,870 hidden lines…
4871worker1 # [ 43.415675] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4872worker1 # [ 43.417114] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.495Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4873lb # [ 43.460295] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4874lb # [ 43.464597] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4875lb # [ 43.468697] niks3-server[970]: 2026/09/23 04:42:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4876lb # [ 43.472786] niks3-server[970]: 2026/09/23 04:42:01 INFO Signed narinfos id=10 count=14877worker1 # [ 43.459187] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.537Z level=INFO msg="Uploading 1 narinfos"4878lb # [ 43.481435] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4879lb # [ 43.485413] niks3-server[970]: 2026/09/23 04:42:01 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4880lb # [ 43.489578] niks3-server[970]: 2026/09/23 04:42:01 INFO Completed upload id=104881worker1 # [ 43.473450] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.551Z level=INFO msg="Upload complete. (195ms)"4882lb # [ 43.491070] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4883worker1 # [ 43.477353] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14884worker2 # [ 43.465392] nix-grpc-daemon[783]: ts=2026-09-23T04:42:01Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04885worker1 # [ 43.487504] nix-daemon[846]: reaped child process 892, status = succeeded4886worker2 # [ 43.469438] nix-grpc-daemon[783]: ts=2026-09-23T04:42:01Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14887worker1 # [ 43.526963] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566944888worker1 # [ 43.537396] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566944889worker1 # [ 43.546591] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.624Z level=INFO msg="All 1 paths already cached"4890client # copying 1 paths to 'lb:50051'...4891client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4892worker1 # [ 43.554453] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566944893worker1 # [ 43.564316] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4894worker1 # [ 43.653577] nix-daemon[846]: accepted connection from pid 917, user nix-grpc-daemon (trusted)4895worker1 # [ 43.668209] nix-daemon[846]: reaped child process 920, status = succeeded4896lb # [ 43.689226] niks3-server[970]: 2026/09/23 04:42:01 INFO Received uploads request method=POST path=/api/pending_closures4897worker1 # [ 43.680463] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4898worker1 # [ 43.681960] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.760Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4899lb # [ 43.718220] niks3-server[970]: 2026/09/23 04:42:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4900lb # [ 43.723563] niks3-server[970]: 2026/09/23 04:42:01 INFO Signed narinfos id=11 count=14901worker1 # [ 43.711084] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.789Z level=INFO msg="Uploading 1 narinfos"4902lb # [ 43.734131] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4903lb # [ 43.741148] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4904lb # [ 43.751489] niks3-server[970]: 2026/09/23 04:42:01 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4905worker1 # [ 43.741960] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.820Z level=INFO msg="Upload complete. (168ms)"4906lb # [ 43.758120] niks3-server[970]: 2026/09/23 04:42:01 INFO Completed upload id=114907worker1 # [ 43.744294] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=6244908worker1 # [ 43.747339] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566944909lb # [ 43.764284] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4910worker1 # [ 43.754522] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4911worker1 # [ 43.756945] nix-daemon[846]: reaped child process 915, status = succeeded4912client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4913worker1 # [ 43.762177] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4914client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4915client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4916client # farm-b> LOG-farm-b-sig4917worker1 # [ 45.181585] nix-daemon[846]: reaped child process 924, status = succeeded4918worker1 # [ 45.248978] nix-daemon[846]: accepted connection from pid 940, user nix-grpc-daemon (trusted)4919worker1 # [ 45.261625] nix-daemon[846]: reaped child process 943, status = succeeded4920lb # [ 45.283780] niks3-server[970]: 2026/09/23 04:42:03 INFO Received uploads request method=POST path=/api/pending_closures4921worker1 # [ 45.275141] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.353Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4922worker1 # [ 45.276862] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.354Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4923lb # [ 45.315582] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4924lb # [ 45.331730] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4925lb # [ 45.337890] niks3-server[970]: 2026/09/23 04:42:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4926worker1 # [ 45.330957] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.409Z level=INFO msg="Uploading 1 narinfos"4927lb # [ 45.346924] niks3-server[970]: 2026/09/23 04:42:03 INFO Signed narinfos id=12 count=14928lb # [ 45.351961] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4929lb # [ 45.362187] niks3-server[970]: 2026/09/23 04:42:03 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4930lb # [ 45.366264] niks3-server[970]: 2026/09/23 04:42:03 INFO Completed upload id=124931lb # [ 45.367941] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4932worker1 # [ 45.350297] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.428Z level=INFO msg="Upload complete. (181ms)"4933worker1 # [ 45.356799] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14934worker2 # [ 45.344433] nix-grpc-daemon[783]: ts=2026-09-23T04:42:03Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04935worker1 # [ 45.366914] nix-daemon[846]: reaped child process 922, status = succeeded4936worker2 # [ 45.348859] nix-grpc-daemon[783]: ts=2026-09-23T04:42:03Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14937worker1 # [ 45.405591] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566944938worker1 # [ 45.415913] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566944939worker1 # [ 45.426920] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.505Z level=INFO msg="All 2 paths already cached"4940client # copying 1 paths to 'lb:50051'...4941client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4942worker1 # [ 45.433736] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566944943worker1 # [ 45.443159] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4944worker1 # [ 45.534148] nix-daemon[846]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4945worker1 # [ 45.547101] nix-daemon[846]: reaped child process 950, status = succeeded4946lb # [ 45.568830] niks3-server[970]: 2026/09/23 04:42:03 INFO Received uploads request method=POST path=/api/pending_closures4947worker1 # [ 45.560387] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4948worker1 # [ 45.561853] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.640Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4949lb # [ 45.600082] niks3-server[970]: 2026/09/23 04:42:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4950lb # [ 45.604372] niks3-server[970]: 2026/09/23 04:42:03 INFO Signed narinfos id=13 count=14951worker1 # [ 45.590730] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.669Z level=INFO msg="Uploading 1 narinfos"4952lb # [ 45.610756] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4953lb # [ 45.614904] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4954lb # [ 45.625577] niks3-server[970]: 2026/09/23 04:42:03 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4955lb # [ 45.629485] niks3-server[970]: 2026/09/23 04:42:03 INFO Completed upload id=134956worker1 # [ 45.613254] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.691Z level=INFO msg="Upload complete. (160ms)"4957lb # [ 45.631378] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4958worker1 # [ 45.617091] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=7524959worker1 # [ 45.621756] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566944960worker1 # [ 45.631247] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4961worker1 # [ 45.636126] nix-daemon[846]: reaped child process 945, status = succeeded4962client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4963worker1 # [ 45.639344] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4964client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4965client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4966client # farm-top> LOG-farm-top-sig4967worker1 # [ 46.253482] nix-daemon[846]: reaped child process 954, status = succeeded4968worker1 # [ 46.324391] nix-daemon[846]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4969worker1 # [ 46.338133] nix-daemon[846]: reaped child process 973, status = succeeded4970lb # [ 46.358593] niks3-server[970]: 2026/09/23 04:42:04 INFO Received uploads request method=POST path=/api/pending_closures4971worker1 # [ 46.348311] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.426Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4972worker1 # [ 46.350170] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.428Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4973lb # [ 46.382476] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4974lb # [ 46.384384] niks3-server[970]: 2026/09/23 04:42:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4975lb # [ 46.386411] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4976lb # [ 46.389691] niks3-server[970]: 2026/09/23 04:42:04 INFO Signed narinfos id=14 count=14977worker1 # [ 46.373621] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.451Z level=INFO msg="Uploading 1 narinfos"4978lb # [ 46.392587] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4979lb # [ 46.397480] niks3-server[970]: 2026/09/23 04:42:04 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4980lb # [ 46.400728] niks3-server[970]: 2026/09/23 04:42:04 INFO Completed upload id=144981worker1 # [ 46.384413] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.462Z level=INFO msg="Upload complete. (139ms)"4982worker1 # [ 46.386964] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14983lb # [ 46.405357] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4984worker1 # [ 46.394841] nix-daemon[846]: reaped child process 952, status = succeeded4985client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.74 seconds)4986client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4987client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4988client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4989worker1 # [ 46.679367] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:556464990worker1 # [ 46.685590] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:556464991client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.16 seconds)4992client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4993worker2 # [ 46.818826] nix-grpc-daemon[783]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:477584994client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4995client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4996client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4997client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4998worker2 # [ 46.840365] nix-grpc-daemon[783]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:477584999worker2 # [ 46.844115] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5000worker2 # [ 46.854159] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5001worker2 # [ 46.862449] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5002client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.16 seconds)5003client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5004client # finding garbage collector roots...5005lb # [ 47.008901] envoy[927]: conn peer=[2001:db8:1::1]:45764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=4388 ms=51105006lb # [ 47.012679] envoy[927]: conn peer=[2001:db8:1::1]:45768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=445007lb # [ 47.016173] envoy[927]: conn peer=[2001:db8:1::1]:45782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1879 ms=15575008lb # [ 47.019603] envoy[927]: conn peer=[2001:db8:1::1]:45796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4351 ms=45955009client # removing stale temporary roots file "/nix/var/nix/temproots/825"5010client # removing stale temporary roots file "/nix/var/nix/temproots/808"5011client # removing stale temporary roots file "/nix/var/nix/temproots/888"5012client # removing stale temporary roots file "/nix/var/nix/temproots/846"5013client # removing stale temporary roots file "/nix/var/nix/temproots/858"5014client # removing stale temporary roots file "/nix/var/nix/temproots/903"5015client # removing stale temporary roots file "/nix/var/nix/temproots/866"5016client # removing stale temporary roots file "/nix/var/nix/temproots/880"5017client # deleting specified paths...5018client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5019client # deleting unused links...5020client # note: hard linking is currently saving 0.0 KiB5021worker2 # [ 47.122672] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:600405022worker2 # [ 47.130167] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405023worker2 # [ 47.138643] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.270Z level=INFO msg="All 1 paths already cached"5024worker2 # [ 47.143542] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405025worker1 # [ 47.167983] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566945026client # copying 3 paths...5027worker2 # [ 47.158759] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405028client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5029worker1 # [ 47.190646] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566945030worker2 # [ 47.175973] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600405031worker1 # [ 47.196435] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=2325032worker2 # [ 47.183396] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=1765033worker2 # [ 47.187733] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600405034worker2 # [ 47.191343] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=1285035client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5036client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5037client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.39 seconds)5038(finished: subtest: outputs read back through any worker carry the cache signature, in 5.56 seconds)5039subtest: two clients wanting the same drv build it once5040client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5041client # [ 47.322869] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5042client # Running as unit: dedup1.service; invocation ID: 64265fa6e39145dfb90f7da1db5a5f745043client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5044client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5045client # [ 47.446894] nix[956]: don't know how to build these paths:5046client # [ 47.447768] nix[956]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5047worker1 # [ 47.433729] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566945048worker2 # [ 47.422137] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05049worker2 # [ 47.427268] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15050worker2 # [ 47.431828] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405051worker2 # [ 47.441351] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405052client # [ 47.482670] nix[956]: copying 1 paths to 'lb:50051'...5053client # [ 47.483553] nix[956]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5054worker2 # [ 47.452312] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405055worker2 # [ 47.462674] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5056worker2 # [ 47.567736] nix-daemon[869]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)5057worker2 # [ 47.582075] nix-daemon[869]: reaped child process 1068, status = succeeded5058lb # [ 47.624529] niks3-server[970]: 2026/09/23 04:42:05 INFO Received uploads request method=POST path=/api/pending_closures5059worker2 # [ 47.599691] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5060worker2 # [ 47.601149] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.731Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5061lb # [ 47.656523] niks3-server[970]: 2026/09/23 04:42:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5062lb # [ 47.665342] niks3-server[970]: 2026/09/23 04:42:05 INFO Signed narinfos id=15 count=15063worker2 # [ 47.635169] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.767Z level=INFO msg="Uploading 1 narinfos"5064lb # [ 47.670333] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5065lb # [ 47.682004] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5066lb # [ 47.692374] niks3-server[970]: 2026/09/23 04:42:05 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5067lb # [ 47.698586] niks3-server[970]: 2026/09/23 04:42:05 INFO Completed upload id=155068worker2 # [ 47.663875] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.795Z level=INFO msg="Upload complete. (189ms)"5069lb # [ 47.702218] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5070worker2 # [ 47.668833] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=4325071worker2 # [ 47.674377] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405072worker2 # [ 47.686834] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5073worker2 # [ 47.696138] nix-daemon[869]: reaped child process 1063, status = succeeded5074worker2 # [ 47.699291] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5075client # [ 47.730665] nix[956]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5076client # [ 48.375834] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5077client # Running as unit: dedup2.service; invocation ID: 8e28dfff5c94458582796e2bfa826dd95078client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5079worker2 # [ 48.507601] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:600405080client # [ 48.545180] nix[972]: this derivation will be built:5081worker1 # [ 48.534551] nix-grpc-daemon[781]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566945082worker1 # [ 48.542555] nix-grpc-daemon[781]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...5083client # [ 48.568147] nix[972]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5084worker2 # [ 48.540500] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05085worker2 # [ 48.544375] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405086client # [ 48.576443] nix[972]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5087client: must succeed: sleep 25088client: (finished: must succeed: sleep 2, in 2.04 seconds)5089client: waiting for success: ! systemctl is-active dedup1 dedup25090worker2 # [ 51.718840] nix-daemon[869]: reaped child process 1072, status = succeeded5091worker2 # [ 51.793364] nix-daemon[869]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)5092worker2 # [ 51.807090] nix-daemon[869]: reaped child process 1107, status = succeeded5093lb # [ 51.850169] niks3-server[970]: 2026/09/23 04:42:09 INFO Received uploads request method=POST path=/api/pending_closures5094worker2 # [ 51.824053] nix-grpc-daemon[883]: time=2026-09-23T04:42:09.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5095worker2 # [ 51.827057] nix-grpc-daemon[883]: time=2026-09-23T04:42:09.957Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5096lb # [ 51.892092] niks3-server[970]: 2026/09/23 04:42:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5097lb # [ 51.898455] niks3-server[970]: 2026/09/23 04:42:09 INFO Signed narinfos id=16 count=15098worker2 # [ 51.870472] nix-grpc-daemon[883]: time=2026-09-23T04:42:10.002Z level=INFO msg="Uploading 1 narinfos"5099lb # [ 51.907089] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5100lb # [ 51.912847] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5101lb # [ 51.918763] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5102lb # [ 51.930139] niks3-server[970]: 2026/09/23 04:42:09 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5103lb # [ 51.937409] niks3-server[970]: 2026/09/23 04:42:09 INFO Completed upload id=165104worker2 # [ 51.903927] nix-grpc-daemon[883]: time=2026-09-23T04:42:10.035Z level=INFO msg="Upload complete. (195ms)"5105worker2 # [ 51.909995] nix-grpc-daemon[783]: ts=2026-09-23T04:42:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15106worker2 # [ 51.918310] nix-grpc-daemon[783]: ts=2026-09-23T04:42:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15107worker2 # [ 51.930577] nix-daemon[869]: reaped child process 1070, status = succeeded5108client # [ 51.972278] systemd[1]: dedup2.service: Deactivated successfully.5109client # [ 51.975286] systemd[1]: dedup2.service: Consumed 141ms CPU time over 3.596s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic.5110lb # [ 51.994844] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5111client # [ 52.011120] systemd[1]: dedup1.service: Deactivated successfully.5112client # [ 52.013898] systemd[1]: dedup1.service: Consumed 112ms CPU time over 4.690s wall clock time, 43.5M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.5113client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5114client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5115client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5116worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5117worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.02 seconds)5118worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5119worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)5120worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5121worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5122worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5123worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5124(finished: subtest: two clients wanting the same drv build it once, in 5.63 seconds)5125subtest: --timeout and --max-silent-time reach the builder5126worker2 # [ 53.020214] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:600405127worker2 # [ 53.030216] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05128worker2 # [ 53.034818] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15129worker1 # [ 53.056844] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945130worker1 # [ 53.069935] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945131worker1 # [ 53.081729] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566945132worker1 # [ 53.090986] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5133worker1 # [ 53.195366] nix-daemon[846]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5134worker1 # [ 53.209993] nix-daemon[846]: reaped child process 1002, status = succeeded5135lb # [ 53.232760] niks3-server[970]: 2026/09/23 04:42:11 INFO Received uploads request method=POST path=/api/pending_closures5136worker1 # [ 53.224626] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5137worker1 # [ 53.226787] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.304Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5138lb # [ 53.263355] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5139lb # [ 53.272953] niks3-server[970]: 2026/09/23 04:42:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5140worker1 # [ 53.262432] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.340Z level=INFO msg="Uploading 1 narinfos"5141lb # [ 53.278668] niks3-server[970]: 2026/09/23 04:42:11 INFO Signed narinfos id=17 count=15142lb # [ 53.287545] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5143lb # [ 53.298881] niks3-server[970]: 2026/09/23 04:42:11 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5144lb # [ 53.306795] niks3-server[970]: 2026/09/23 04:42:11 INFO Completed upload id=175145worker1 # [ 53.290416] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.368Z level=INFO msg="Upload complete. (185ms)"5146lb # [ 53.308787] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5147worker1 # [ 53.295268] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=4245148worker1 # [ 53.300573] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945149worker1 # [ 53.312693] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5150worker1 # [ 53.318136] nix-daemon[846]: reaped child process 997, status = succeeded5151worker1 # [ 53.322046] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5152lb # [ 57.008901] envoy[927]: conn peer=[2001:db8:1::1]:42652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1375153lb # [ 57.012735] envoy[927]: conn peer=[2001:db8:1::1]:42670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=785154lb # [ 57.016413] envoy[927]: conn peer=[2001:db8:1::1]:42664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=835155lb # [ 57.020073] envoy[927]: conn peer=[2001:db8:1::1]:42660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=885156lb # [ 57.023642] envoy[927]: conn peer=[2001:db8:1::1]:42682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1598 ms=34165157lb # [ 57.027776] envoy[927]: conn peer=[2001:db8:1::1]:42678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1545 ms=45485158worker1 # [ 58.933421] nix-grpc-daemon[781]: ts=2026-09-23T04:42:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05159worker1 # [ 58.957444] nix-daemon[846]: reaped child process 1004, status = succeeded5160worker1 # [ 58.959427] nix-daemon[846]: reaped child process 1006, status = succeeded5161worker1 # [ 59.110139] nix-grpc-daemon[781]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566945162worker2 # [ 59.103557] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05163worker2 # [ 59.108303] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15164worker2 # [ 59.112721] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405165worker2 # [ 59.123949] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405166worker2 # [ 59.134691] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405167worker2 # [ 59.145702] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5168worker2 # [ 59.247031] nix-daemon[869]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)5169worker2 # [ 59.261996] nix-daemon[869]: reaped child process 1127, status = succeeded5170lb # [ 59.305715] niks3-server[970]: 2026/09/23 04:42:17 INFO Received uploads request method=POST path=/api/pending_closures5171worker2 # [ 59.276953] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.408Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5172worker2 # [ 59.278444] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.408Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5173lb # [ 59.332180] niks3-server[970]: 2026/09/23 04:42:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5174worker2 # [ 59.305740] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.437Z level=INFO msg="Uploading 1 narinfos"5175lb # [ 59.340443] niks3-server[970]: 2026/09/23 04:42:17 INFO Signed narinfos id=18 count=15176lb # [ 59.347670] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5177lb # [ 59.358617] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5178lb # [ 59.370342] niks3-server[970]: 2026/09/23 04:42:17 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5179lb # [ 59.375802] niks3-server[970]: 2026/09/23 04:42:17 INFO Completed upload id=185180worker2 # [ 59.340971] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.472Z level=INFO msg="Upload complete. (184ms)"5181lb # [ 59.380587] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5182worker2 # [ 59.345122] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=4245183worker2 # [ 59.351431] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405184worker2 # [ 59.362110] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5185worker2 # [ 59.366951] nix-daemon[869]: reaped child process 1122, status = succeeded5186worker2 # [ 59.371696] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5187worker2 # [ 65.788140] nix-grpc-daemon[783]: ts=2026-09-23T04:42:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05188worker2 # [ 65.812266] nix-daemon[869]: reaped child process 1129, status = succeeded5189worker2 # [ 65.815591] nix-daemon[869]: reaped child process 1131, status = succeeded5190(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.02 seconds)5191subtest: the first client leaving does not fail the second5192client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5193client # [ 65.992545] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5194client # Running as unit: share1.service; invocation ID: 32488c1e4eec489d8c15c6beda8987a35195client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.05 seconds)5196client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5197worker1 # [ 66.114726] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566945198client # [ 66.133293] nix[1045]: don't know how to build these paths:5199client # [ 66.134175] nix[1045]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5200worker2 # [ 66.107826] nix-grpc-daemon[783]: ts=2026-09-23T04:42:24Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05201worker1 # [ 66.128803] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945202worker2 # [ 66.112354] nix-grpc-daemon[783]: ts=2026-09-23T04:42:24Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15203worker1 # [ 66.139248] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945204client # [ 66.162024] nix[1045]: copying 1 paths to 'lb:50051'...5205client # [ 66.162895] nix[1045]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5206worker1 # [ 66.150125] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566945207worker1 # [ 66.160952] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5208worker1 # [ 66.262924] nix-daemon[846]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5209worker1 # [ 66.277182] nix-daemon[846]: reaped child process 1032, status = succeeded5210lb # [ 66.298922] niks3-server[970]: 2026/09/23 04:42:24 INFO Received uploads request method=POST path=/api/pending_closures5211worker1 # [ 66.287913] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.365Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5212worker1 # [ 66.289215] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.365Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5213lb # [ 66.324142] niks3-server[970]: 2026/09/23 04:42:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5214lb # [ 66.329262] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5215worker1 # [ 66.320404] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.398Z level=INFO msg="Uploading 1 narinfos"5216lb # [ 66.336142] niks3-server[970]: 2026/09/23 04:42:24 INFO Signed narinfos id=19 count=15217lb # [ 66.345847] niks3-server[970]: 2026/09/23 04:42:24 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5218lb # [ 66.365187] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5219lb # [ 66.370306] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5220lb # [ 66.373633] niks3-server[970]: 2026/09/23 04:42:24 INFO Completed upload id=195221worker1 # [ 66.357764] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.435Z level=INFO msg="Upload complete. (185ms)"5222worker1 # [ 66.361857] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=4325223worker1 # [ 66.367864] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945224worker1 # [ 66.378132] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5225worker1 # [ 66.388909] nix-daemon[846]: reaped child process 1027, status = succeeded5226worker1 # [ 66.392402] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5227client # [ 66.405131] nix[1045]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5228lb # [ 67.009224] envoy[927]: conn peer=[2001:db8:1::1]:53586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=59475229lb # [ 67.012953] envoy[927]: conn peer=[2001:db8:1::1]:53590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=67075230client # [ 67.051745] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5231client # Running as unit: share2.service; invocation ID: 0846223b70e64b71b43fd0f4b3d8bd6f5232client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5233client: must succeed: sleep 2; systemctl kill -s INT share15234worker1 # [ 67.195161] nix-grpc-daemon[781]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566945235client # [ 67.216114] nix[1061]: this derivation will be built:5236worker2 # [ 67.187211] nix-grpc-daemon[783]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405237worker2 # [ 67.198349] nix-grpc-daemon[783]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'...5238client # [ 67.244158] nix[1061]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5239worker2 # [ 67.219249] nix-grpc-daemon[783]: ts=2026-09-23T04:42:25Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05240worker1 # [ 67.245142] nix-grpc-daemon[781]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945241client # [ 67.260559] nix[1061]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5242client # [ 69.201651] systemd[1]: share1.service: Sent signal SIGINT to main process 1045 (nix) on client request.5243client # [ 69.208654] nix[1045]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5244client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.07 seconds)5245client: waiting for success: ! systemctl is-active share15246client # [ 69.215677] nix[1045]: error: interrupted by the user5247client # [ 69.257218] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5248client # [ 69.258846] systemd[1]: share1.service: Failed with result 'exit-code'.5249client # [ 69.260821] systemd[1]: share1.service: Consumed 115ms CPU time over 3.266s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5250client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5251client: waiting for success: ! systemctl is-active share25252worker1 # [ 70.635420] nix-daemon[846]: reaped child process 1036, status = succeeded5253worker1 # [ 70.713712] nix-daemon[846]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)5254worker1 # [ 70.725086] nix-daemon[846]: reaped child process 1069, status = succeeded5255lb # [ 70.748897] niks3-server[970]: 2026/09/23 04:42:28 INFO Received uploads request method=POST path=/api/pending_closures5256worker1 # [ 70.742109] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5257worker1 # [ 70.743979] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.821Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5258lb # [ 70.777824] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5259lb # [ 70.779450] niks3-server[970]: 2026/09/23 04:42:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5260lb # [ 70.781868] niks3-server[970]: 2026/09/23 04:42:28 INFO Signed narinfos id=20 count=15261worker1 # [ 70.765761] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.843Z level=INFO msg="Uploading 1 narinfos"5262lb # [ 70.785260] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5263lb # [ 70.789822] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5264lb # [ 70.793184] niks3-server[970]: 2026/09/23 04:42:28 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5265lb # [ 70.795890] niks3-server[970]: 2026/09/23 04:42:28 INFO Completed upload id=205266worker1 # [ 70.781033] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.859Z level=INFO msg="Upload complete. (178ms)"5267lb # [ 70.800907] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5268worker1 # [ 70.783762] nix-grpc-daemon[781]: ts=2026-09-23T04:42:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15269worker1 # [ 70.787129] nix-grpc-daemon[781]: ts=2026-09-23T04:42:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15270worker1 # [ 70.792787] nix-daemon[846]: reaped child process 1034, status = succeeded5271client # [ 70.867187] systemd[1]: share2.service: Deactivated successfully.5272client # [ 70.870749] systemd[1]: share2.service: Consumed 121ms CPU time over 3.818s wall clock time, 33.4M memory peak, 6.4K incoming IP traffic, 6K outgoing IP traffic.5273client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5274client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5275client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5276(finished: subtest: the first client leaving does not fail the second, in 5.63 seconds)5277subtest: build hook: nix-daemon with builders = grpc://lb5278client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5279client # [ 71.669425] systemd[1]: Started Nix Daemon.5280client # [ 71.759812] nix-daemon[1112]: accepted connection from pid 1109, user root (trusted)5281worker2 # [ 71.842861] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600405282worker1 # [ 71.866550] nix-grpc-daemon[781]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945283worker2 # [ 71.856535] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05284worker2 # [ 71.858734] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15285worker2 # [ 71.861208] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405286worker2 # [ 71.869704] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405287worker2 # [ 71.877143] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405288worker2 # [ 71.880917] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5289worker2 # [ 71.966518] nix-daemon[869]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5290worker2 # [ 71.979525] nix-daemon[869]: reaped child process 1176, status = succeeded5291lb # [ 72.022439] niks3-server[970]: 2026/09/23 04:42:29 INFO Received uploads request method=POST path=/api/pending_closures5292worker2 # [ 71.997215] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5293worker2 # [ 71.999240] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.130Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5294lb # [ 72.052772] niks3-server[970]: 2026/09/23 04:42:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5295worker2 # [ 72.025596] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.157Z level=INFO msg="Uploading 1 narinfos"5296lb # [ 72.060664] niks3-server[970]: 2026/09/23 04:42:30 INFO Signed narinfos id=21 count=15297lb # [ 72.066569] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5298lb # [ 72.070352] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5299lb # [ 72.079812] niks3-server[970]: 2026/09/23 04:42:30 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5300lb # [ 72.083148] niks3-server[970]: 2026/09/23 04:42:30 INFO Completed upload id=215301worker2 # [ 72.048259] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.180Z level=INFO msg="Upload complete. (160ms)"5302worker2 # [ 72.050302] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=5125303lb # [ 72.086297] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5304worker2 # [ 72.053426] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405305worker2 # [ 72.060721] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5306worker2 # [ 72.063043] nix-daemon[869]: reaped child process 1171, status = succeeded5307worker2 # [ 72.068110] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5308worker2 # [ 73.305696] nix-daemon[869]: reaped child process 1180, status = succeeded5309worker2 # [ 73.391089] nix-daemon[869]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5310worker2 # [ 73.405337] nix-daemon[869]: reaped child process 1199, status = succeeded5311lb # [ 73.448591] niks3-server[970]: 2026/09/23 04:42:31 INFO Received uploads request method=POST path=/api/pending_closures5312worker2 # [ 73.617721] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.748Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5313worker2 # [ 73.630706] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.748Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5314lb # [ 73.698979] niks3-server[970]: 2026/09/23 04:42:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5315lb # [ 73.709076] niks3-server[970]: 2026/09/23 04:42:31 INFO Signed narinfos id=22 count=15316worker2 # [ 73.677665] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.809Z level=INFO msg="Uploading 1 narinfos"5317lb # [ 73.718087] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5318lb # [ 73.726504] niks3-server[970]: 2026/09/23 04:42:31 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5319lb # [ 73.748138] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5320lb # [ 73.757562] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5321lb # [ 73.781221] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5322lb # [ 73.787219] niks3-server[970]: 2026/09/23 04:42:31 INFO Completed upload id=225323worker2 # [ 73.752493] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.884Z level=INFO msg="Upload complete. (475ms)"5324worker2 # [ 73.760118] nix-grpc-daemon[783]: ts=2026-09-23T04:42:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15325worker2 # [ 73.776193] nix-daemon[869]: reaped child process 1178, status = succeeded5326worker2 # [ 73.806086] nix-grpc-daemon[783]: ts=2026-09-23T04:42:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405327worker1 # [ 73.846777] nix-grpc-daemon[781]: ts=2026-09-23T04:42:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566945328worker1 # [ 73.858106] nix-grpc-daemon[781]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5329worker1 # [ 73.872348] nix-grpc-daemon[781]: ts=2026-09-23T04:42:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=1285330worker1 # [ 74.087875] nix-grpc-daemon[781]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:566945331worker1 # [ 74.094111] nix-grpc-daemon[781]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945332worker1 # [ 74.099704] nix-grpc-daemon[875]: time=2026-09-23T04:42:32.176Z level=INFO msg="All 1 paths already cached"5333worker2 # [ 74.093583] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05334worker2 # [ 74.099225] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15335worker2 # [ 74.104137] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405336worker2 # [ 74.115444] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405337worker2 # [ 74.127781] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.259Z level=INFO msg="All 1 paths already cached"5338worker2 # [ 74.135133] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405339worker2 # [ 74.145267] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5340worker2 # [ 74.254935] nix-daemon[869]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5341worker2 # [ 74.272075] nix-daemon[869]: reaped child process 1208, status = succeeded5342lb # [ 74.312145] niks3-server[970]: 2026/09/23 04:42:32 INFO Received uploads request method=POST path=/api/pending_closures5343worker2 # [ 74.284944] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.417Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5344worker2 # [ 74.286475] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.418Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5345lb # [ 74.346561] niks3-server[970]: 2026/09/23 04:42:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5346lb # [ 74.351633] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5347worker2 # [ 74.324487] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.456Z level=INFO msg="Uploading 1 narinfos"5348lb # [ 74.359266] niks3-server[970]: 2026/09/23 04:42:32 INFO Signed narinfos id=23 count=15349lb # [ 74.365871] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5350lb # [ 74.377134] niks3-server[970]: 2026/09/23 04:42:32 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5351lb # [ 74.386324] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5352lb # [ 74.388928] niks3-server[970]: 2026/09/23 04:42:32 INFO Completed upload id=235353worker2 # [ 74.354378] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.486Z level=INFO msg="Upload complete. (196ms)"5354worker2 # [ 74.359109] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=6245355worker2 # [ 74.364944] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405356worker2 # [ 74.378337] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5357worker2 # [ 74.384544] nix-daemon[869]: reaped child process 1203, status = succeeded5358worker2 # [ 74.388911] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5359worker2 # [ 75.839428] nix-daemon[869]: reaped child process 1212, status = succeeded5360worker2 # [ 75.932110] nix-daemon[869]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5361worker2 # [ 75.949236] nix-daemon[869]: reaped child process 1231, status = succeeded5362lb # [ 75.988698] niks3-server[970]: 2026/09/23 04:42:33 INFO Received uploads request method=POST path=/api/pending_closures5363worker2 # [ 76.009441] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5364worker2 # [ 76.012097] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.143Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5365lb # [ 76.080573] niks3-server[970]: 2026/09/23 04:42:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5366lb # [ 76.085375] niks3-server[970]: 2026/09/23 04:42:34 INFO Signed narinfos id=24 count=15367worker2 # [ 76.053779] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.185Z level=INFO msg="Uploading 1 narinfos"5368lb # [ 76.098805] niks3-server[970]: 2026/09/23 04:42:34 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5369lb # [ 76.144328] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5370lb # [ 76.150457] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5371lb # [ 76.159356] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5372lb # [ 76.165995] niks3-server[970]: 2026/09/23 04:42:34 INFO Completed upload id=245373worker2 # [ 76.134669] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.266Z level=INFO msg="Upload complete. (336ms)"5374lb # [ 76.171118] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5375worker2 # [ 76.142532] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15376worker2 # [ 76.158343] nix-daemon[869]: reaped child process 1210, status = succeeded5377worker1 # [ 76.207368] nix-grpc-daemon[781]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566945378worker1 # [ 76.218225] nix-grpc-daemon[781]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5379worker2 # [ 76.225946] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600405380worker2 # [ 76.231884] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=1765381worker2 # [ 76.396412] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600405382worker1 # [ 76.418085] nix-grpc-daemon[781]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945383worker1 # [ 76.427841] nix-grpc-daemon[875]: time=2026-09-23T04:42:34.505Z level=INFO msg="All 2 paths already cached"5384worker2 # [ 76.423256] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05385worker2 # [ 76.427336] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15386worker2 # [ 76.434793] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405387worker2 # [ 76.446861] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405388worker2 # [ 76.459464] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.591Z level=INFO msg="All 2 paths already cached"5389worker2 # [ 76.467601] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405390worker2 # [ 76.478180] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5391worker2 # [ 76.586089] nix-daemon[869]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5392worker2 # [ 76.602131] nix-daemon[869]: reaped child process 1240, status = succeeded5393lb # [ 76.644560] niks3-server[970]: 2026/09/23 04:42:34 INFO Received uploads request method=POST path=/api/pending_closures5394worker2 # [ 76.617072] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5395worker2 # [ 76.618598] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.750Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5396lb # [ 76.678604] niks3-server[970]: 2026/09/23 04:42:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5397worker2 # [ 76.652650] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.784Z level=INFO msg="Uploading 1 narinfos"5398lb # [ 76.687670] niks3-server[970]: 2026/09/23 04:42:34 INFO Signed narinfos id=25 count=15399lb # [ 76.692605] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5400lb # [ 76.699242] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5401lb # [ 76.711081] niks3-server[970]: 2026/09/23 04:42:34 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5402lb # [ 76.717945] niks3-server[970]: 2026/09/23 04:42:34 INFO Completed upload id=255403worker2 # [ 76.683235] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.815Z level=INFO msg="Upload complete. (192ms)"5404lb # [ 76.724294] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5405worker2 # [ 76.687373] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=7525406worker2 # [ 76.692618] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405407worker2 # [ 76.705065] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5408worker2 # [ 76.709587] nix-daemon[869]: reaped child process 1235, status = succeeded5409worker2 # [ 76.714678] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5410lb # [ 77.010318] envoy[927]: conn peer=[2001:db8:1::1]:59662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=767 ms=30975411lb # [ 77.013917] envoy[927]: conn peer=[2001:db8:1::1]:59664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1602 tx=1598 ms=36425412lb # [ 77.017440] envoy[927]: conn peer=[2001:db8:1::1]:45004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=815413lb # [ 77.020973] envoy[927]: conn peer=[2001:db8:1::1]:59670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1962 ms=20645414lb # [ 77.024590] envoy[927]: conn peer=[2001:db8:1::1]:45016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=175415lb # [ 77.028155] envoy[927]: conn peer=[2001:db8:1::1]:45012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2141 ms=21745416worker2 # [ 77.755816] nix-daemon[869]: reaped child process 1244, status = succeeded5417worker2 # [ 77.844992] nix-daemon[869]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5418worker2 # [ 77.858855] nix-daemon[869]: reaped child process 1263, status = succeeded5419lb # [ 77.902570] niks3-server[970]: 2026/09/23 04:42:35 INFO Received uploads request method=POST path=/api/pending_closures5420worker2 # [ 77.877051] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5421worker2 # [ 77.878539] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.010Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5422lb # [ 77.949741] niks3-server[970]: 2026/09/23 04:42:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5423lb # [ 77.958359] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5424lb # [ 77.963697] niks3-server[970]: 2026/09/23 04:42:35 INFO Signed narinfos id=26 count=15425worker2 # [ 77.932128] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.063Z level=INFO msg="Uploading 1 narinfos"5426lb # [ 77.967576] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5427lb # [ 77.972792] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5428lb # [ 77.984901] niks3-server[970]: 2026/09/23 04:42:35 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5429lb # [ 77.988992] niks3-server[970]: 2026/09/23 04:42:35 INFO Completed upload id=265430worker2 # [ 77.954459] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.086Z level=INFO msg="Upload complete. (231ms)"5431lb # [ 77.991413] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5432worker2 # [ 77.961466] nix-grpc-daemon[783]: ts=2026-09-23T04:42:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15433worker2 # [ 77.970622] nix-grpc-daemon[783]: ts=2026-09-23T04:42:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405434worker2 # [ 77.982707] nix-daemon[869]: reaped child process 1242, status = succeeded5435worker1 # [ 78.006286] nix-grpc-daemon[781]: ts=2026-09-23T04:42:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566945436worker1 # [ 78.015921] nix-grpc-daemon[781]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5437worker1 # [ 78.029173] nix-grpc-daemon[781]: ts=2026-09-23T04:42:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=2325438client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.57 seconds)5439client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5440client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5441client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5442client # [ 78.192557] nix-daemon[1112]: reaped child process 1114, status = succeeded5443client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds)5444client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5445client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5446(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.65 seconds)5447subtest: an input only one worker has still reaches the builder5448client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5449client # this derivation will be built:5450client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5451client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5452client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5453client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5454client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5455worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5456worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5457worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5458worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5459client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25460client # [ 78.823794] nix-daemon[1112]: accepted connection from pid 1195, user root (trusted)5461client # this derivation will be built:5462client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5463worker1 # [ 78.946915] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:566945464client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5465worker1 # [ 78.952933] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945466worker1 # [ 79.055182] nix-daemon[846]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)5467worker1 # [ 79.066513] nix-daemon[846]: reaped child process 1082, status = succeeded5468lb # [ 79.090281] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures5469worker1 # [ 79.082241] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5470worker1 # [ 79.084275] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.162Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5471lb # [ 79.115841] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5472worker1 # [ 79.110186] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.188Z level=INFO msg="Uploading 1 narinfos"5473lb # [ 79.126760] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=27 count=15474lb # [ 79.134112] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5475lb # [ 79.140313] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5476lb # [ 79.152686] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5477lb # [ 79.163393] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=275478worker1 # [ 79.150134] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.228Z level=INFO msg="Upload complete. (198ms)"5479lb # [ 79.167130] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5480worker2 # [ 79.151786] nix-grpc-daemon[783]: ts=2026-09-23T04:42:37Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05481worker2 # [ 79.156265] nix-grpc-daemon[783]: ts=2026-09-23T04:42:37Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15482worker1 # [ 79.214933] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945483worker1 # [ 79.228212] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945484client # copying 2 paths to 'lb:50051'...5485client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5486worker1 # [ 79.245331] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566945487client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5488worker1 # [ 79.256628] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5489worker1 # [ 79.362459] nix-daemon[846]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)5490worker1 # [ 79.376978] nix-daemon[846]: reaped child process 1089, status = succeeded5491lb # [ 79.397424] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures5492lb # [ 79.407180] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures5493worker1 # [ 79.399793] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.478Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5494worker1 # [ 79.401303] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.479Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5495worker1 # [ 79.403087] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.481Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5496lb # [ 79.459125] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5497lb # [ 79.461174] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5498lb # [ 79.465967] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5499lb # [ 79.470263] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=28 count=15500lb # [ 79.474394] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5501lb # [ 79.477077] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=29 count=25502lb # [ 79.478927] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5503worker1 # [ 79.462548] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.540Z level=INFO msg="Uploading 3 narinfos"5504lb # [ 79.481487] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5505lb # [ 79.494478] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5506lb # [ 79.497839] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=285507lb # [ 79.500062] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5508lb # [ 79.502659] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5509lb # [ 79.505621] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=295510lb # [ 79.507249] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5511worker1 # [ 79.490947] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.569Z level=INFO msg="Upload complete. (219ms)"5512lb # [ 79.508989] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5513worker1 # [ 79.496393] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=2 nar_bytes_in=9765514worker1 # [ 79.502869] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945515worker1 # [ 79.515107] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5516worker1 # [ 79.525214] nix-daemon[846]: reaped child process 1084, status = succeeded5517client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5518worker1 # [ 79.531560] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5519client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5520client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5521worker1 # [ 80.570281] nix-daemon[846]: reaped child process 1093, status = succeeded5522worker1 # [ 80.647385] nix-daemon[846]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5523worker1 # [ 80.660214] nix-daemon[846]: reaped child process 1112, status = succeeded5524lb # [ 80.685360] niks3-server[970]: 2026/09/23 04:42:38 INFO Received uploads request method=POST path=/api/pending_closures5525worker1 # [ 80.676160] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.754Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5526lb # [ 80.713168] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5527lb # [ 80.721566] niks3-server[970]: 2026/09/23 04:42:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5528lb # [ 80.728865] niks3-server[970]: 2026/09/23 04:42:38 INFO Signed narinfos id=30 count=15529worker1 # [ 80.715751] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.793Z level=INFO msg="Uploading 1 narinfos"5530lb # [ 80.736355] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5531lb # [ 80.750337] niks3-server[970]: 2026/09/23 04:42:38 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5532lb # [ 80.756623] niks3-server[970]: 2026/09/23 04:42:38 INFO Completed upload id=305533worker1 # [ 80.740720] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.818Z level=INFO msg="Upload complete. (201ms)"5534lb # [ 80.761211] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5535client # copying 1 paths...5536worker1 # [ 80.746618] nix-grpc-daemon[781]: ts=2026-09-23T04:42:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15537worker2 # [ 80.736308] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405538worker1 # [ 80.761086] nix-daemon[846]: reaped child process 1091, status = succeeded5539worker2 # [ 80.744701] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5540client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5541worker2 # [ 80.777472] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600405542worker2 # [ 80.783203] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=1205543client # [ 80.923395] nix-daemon[1112]: reaped child process 1198, status = succeeded5544client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.23 seconds)5545client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25546client # [ 81.028095] nix-daemon[1112]: accepted connection from pid 1217, user root (trusted)5547client # this derivation will be built:5548client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5549worker2 # [ 81.111235] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600405550client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5551worker1 # [ 81.133100] nix-grpc-daemon[781]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945552worker1 # [ 81.143260] nix-grpc-daemon[875]: time=2026-09-23T04:42:39.221Z level=INFO msg="All 1 paths already cached"5553worker2 # [ 81.139264] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05554worker2 # [ 81.143636] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15555worker2 # [ 81.193692] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405556worker2 # [ 81.206575] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:600405557client # copying 1 paths to 'lb:50051'...5558client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5559worker2 # [ 81.225664] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:600405560worker2 # [ 81.234977] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5561worker2 # [ 81.254199] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5562worker2 # [ 81.305894] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5)5563worker2 # [ 81.419385] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5)5564worker2 # [ 81.731691] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 4/5)5565worker2 # [ 81.939977] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5566worker2 # [ 81.944361] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5567worker2 # [ 81.949547] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5568worker2 # [ 82.073407] nix-daemon[869]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5569worker2 # [ 82.093518] nix-daemon[869]: reaped child process 1283, status = succeeded5570lb # [ 82.134511] niks3-server[970]: 2026/09/23 04:42:40 INFO Received uploads request method=POST path=/api/pending_closures5571worker2 # [ 82.108551] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.239Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5572worker2 # [ 82.109999] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.239Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5573lb # [ 82.161933] niks3-server[970]: 2026/09/23 04:42:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5574worker2 # [ 82.132806] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.264Z level=INFO msg="Uploading 1 narinfos"5575lb # [ 82.168173] niks3-server[970]: 2026/09/23 04:42:40 INFO Signed narinfos id=31 count=15576lb # [ 82.173869] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5577lb # [ 82.186019] niks3-server[970]: 2026/09/23 04:42:40 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5578lb # [ 82.191182] niks3-server[970]: 2026/09/23 04:42:40 INFO Completed upload id=315579worker2 # [ 82.157507] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.289Z level=INFO msg="Upload complete. (189ms)"5580lb # [ 82.193237] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5581worker2 # [ 82.160461] nix-grpc-daemon[783]: ts=2026-09-23T04:42:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=5605582lb # [ 82.196340] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5583worker2 # [ 82.164194] nix-grpc-daemon[783]: ts=2026-09-23T04:42:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:600405584worker2 # [ 82.170472] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5585worker2 # [ 82.177169] nix-daemon[869]: reaped child process 1273, status = succeeded5586worker2 # [ 82.183384] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5587worker2 # [ 82.190462] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5588client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5589client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5590client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5591worker2 # [ 83.229718] nix-daemon[869]: reaped child process 1288, status = succeeded5592worker2 # [ 83.321826] nix-daemon[869]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5593worker2 # [ 83.337588] nix-daemon[869]: reaped child process 1307, status = succeeded5594lb # [ 83.379242] niks3-server[970]: 2026/09/23 04:42:41 INFO Received uploads request method=POST path=/api/pending_closures5595worker2 # [ 83.359939] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.491Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5596lb # [ 83.418426] niks3-server[970]: 2026/09/23 04:42:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5597worker2 # [ 83.393062] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.524Z level=INFO msg="Uploading 1 narinfos"5598lb # [ 83.428175] niks3-server[970]: 2026/09/23 04:42:41 INFO Signed narinfos id=32 count=15599lb # [ 83.433884] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5600lb # [ 83.440699] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5601lb # [ 83.449216] niks3-server[970]: 2026/09/23 04:42:41 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5602lb # [ 83.471124] niks3-server[970]: 2026/09/23 04:42:41 INFO Completed upload id=325603worker2 # [ 83.436592] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.568Z level=INFO msg="Upload complete. (236ms)"5604lb # [ 83.474253] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5605worker2 # [ 83.444497] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15606client # copying 1 paths...5607worker2 # [ 83.455313] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405608client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5609worker2 # [ 83.465694] nix-daemon[869]: reaped child process 1285, status = succeeded5610worker1 # [ 83.497330] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:566945611worker1 # [ 83.509387] nix-grpc-daemon[781]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5612worker1 # [ 83.523983] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=1205613client # [ 83.644473] nix-daemon[1112]: reaped child process 1220, status = succeeded5614client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.72 seconds)5615client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25616client # [ 83.743628] nix-daemon[1112]: accepted connection from pid 1239, user root (trusted)5617client # this derivation will be built:5618client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5619worker2 # [ 83.821616] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:600405620client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5621worker1 # [ 83.844070] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945622worker1 # [ 83.852308] nix-grpc-daemon[875]: time=2026-09-23T04:42:41.929Z level=INFO msg="All 1 paths already cached"5623worker2 # [ 83.848168] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05624worker2 # [ 83.852967] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15625worker1 # [ 83.874492] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945626worker1 # [ 83.886338] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566945627worker1 # [ 83.904411] nix-grpc-daemon[875]: time=2026-09-23T04:42:41.982Z level=INFO msg="All 1 paths already cached"5628client # copying 1 paths to 'lb:50051'...5629client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5630worker1 # [ 83.912674] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566945631worker1 # [ 83.921695] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5632worker1 # [ 84.022583] nix-daemon[846]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)5633worker1 # [ 84.036425] nix-daemon[846]: reaped child process 1120, status = succeeded5634lb # [ 84.057461] niks3-server[970]: 2026/09/23 04:42:42 INFO Received uploads request method=POST path=/api/pending_closures5635worker1 # [ 84.048816] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.127Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5636worker1 # [ 84.050494] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.128Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5637lb # [ 84.087125] niks3-server[970]: 2026/09/23 04:42:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5638worker1 # [ 84.079491] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.157Z level=INFO msg="Uploading 1 narinfos"5639lb # [ 84.095295] niks3-server[970]: 2026/09/23 04:42:42 INFO Signed narinfos id=33 count=15640lb # [ 84.110202] niks3-server[970]: 2026/09/23 04:42:42 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5641lb # [ 84.113910] niks3-server[970]: 2026/09/23 04:42:42 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5642lb # [ 84.119091] niks3-server[970]: 2026/09/23 04:42:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5643lb # [ 84.124252] niks3-server[970]: 2026/09/23 04:42:42 INFO Completed upload id=335644worker1 # [ 84.108475] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.186Z level=INFO msg="Upload complete. (172ms)"5645worker1 # [ 84.112622] nix-grpc-daemon[781]: ts=2026-09-23T04:42:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=5605646worker1 # [ 84.117439] nix-grpc-daemon[781]: ts=2026-09-23T04:42:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566945647worker1 # [ 84.127065] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5648worker1 # [ 84.132253] nix-daemon[846]: reaped child process 1115, status = succeeded5649client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5650worker1 # [ 84.136777] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5651client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5652client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5653worker1 # [ 85.176281] nix-daemon[846]: reaped child process 1124, status = succeeded5654worker1 # [ 85.252088] nix-daemon[846]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)5655worker1 # [ 85.264246] nix-daemon[846]: reaped child process 1143, status = succeeded5656lb # [ 85.288534] niks3-server[970]: 2026/09/23 04:42:43 INFO Received uploads request method=POST path=/api/pending_closures5657worker1 # [ 85.279267] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.357Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5658lb # [ 85.314890] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5659lb # [ 85.324387] niks3-server[970]: 2026/09/23 04:42:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5660lb # [ 85.329487] niks3-server[970]: 2026/09/23 04:42:43 INFO Signed narinfos id=34 count=15661worker1 # [ 85.317927] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.396Z level=INFO msg="Uploading 1 narinfos"5662lb # [ 85.339723] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5663lb # [ 85.350118] niks3-server[970]: 2026/09/23 04:42:43 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5664lb # [ 85.357925] niks3-server[970]: 2026/09/23 04:42:43 INFO Completed upload id=345665worker1 # [ 85.341644] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.419Z level=INFO msg="Upload complete. (197ms)"5666lb # [ 85.362975] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5667worker1 # [ 85.346795] nix-grpc-daemon[781]: ts=2026-09-23T04:42:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15668worker1 # [ 85.361531] nix-daemon[846]: reaped child process 1122, status = succeeded5669client # copying 1 paths...5670worker2 # [ 85.376499] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:600405671worker2 # [ 85.388737] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5672client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5673worker2 # [ 85.420789] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:600405674worker2 # [ 85.426407] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=1205675client # [ 85.567379] nix-daemon[1112]: reaped child process 1242, status = succeeded5676client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.92 seconds)5677(finished: subtest: an input only one worker has still reaches the builder, in 7.35 seconds)5678subtest: upload whose reference the worker lacks is completed from the cache5679client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5680client # this derivation will be built:5681client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5682client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5683client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.23 seconds)5684client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5685client # this derivation will be built:5686client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5687client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5688client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5689client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5690client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5691worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5692worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5693worker2 # [ 86.274581] systemd[1]: Stopping Nix worker-protocol over gRPC...5694lb # [ 87.010427] envoy[927]: conn peer=[2001:db8:1::1]:45034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=755695lb # [ 87.014129] envoy[927]: conn peer=[2001:db8:1::1]:45026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2372 ms=16685696lb # [ 87.017710] envoy[927]: conn peer=[2001:db8:1::1]:45048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3849 tx=2066 ms=19065697lb # [ 87.021336] envoy[927]: conn peer=[2001:db8:1::1]:45054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=645698lb # [ 87.024954] envoy[927]: conn peer=[2001:db8:1::1]:45056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2123 ms=24475699lb # [ 87.028550] envoy[927]: conn peer=[2001:db8:1::1]:36148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=825700lb # [ 87.031994] envoy[927]: conn peer=[2001:db8:1::1]:36168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=595701lb # [ 87.036201] envoy[927]: conn peer=[2001:db8:1::1]:36162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=2108 ms=16575702worker2 # [ 87.171359] nix-grpc-daemon[783]: ts=2026-09-23T04:42:45Z level=info event=signal_exit5703lb # [ 87.212861] niks3-server[970]: 2026/09/23 04:42:45 INFO lead: released remote=192.168.1.4:357225704worker1 # [ 88.197569] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting addr=lb:500515705worker2 # [ 88.182388] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting5706worker2 # [ 88.186518] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600285707worker2 # [ 88.190226] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_gone addr=192.168.1.3:500515708worker2 # [ 88.193626] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting addr=lb:500515709worker2 # [ 88.196871] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600285710worker2 # [ 88.200660] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_gone addr=192.168.1.4:500515711worker2 # [ 88.222652] nix-daemon[869]: reaped child process 871, status = succeeded5712worker2 # [ 88.327804] nix-grpc-daemon[783]: E0000 00:00:1790138566.459686 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5713worker2 # [ 88.329916] nix-grpc-daemon[783]: E0000 00:00:1790138566.459832 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5714worker1 # [ 88.397941] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:500515715worker1 # [ 88.402346] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5716worker2 # [ 88.387555] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5717worker1 # [ 88.603276] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:500515718worker1 # [ 88.607583] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5719worker1 # [ 88.808842] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:500515720worker1 # [ 88.812903] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5721worker2 # [ 88.961585] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5722worker2 # [ 88.966361] systemd[1]: Stopped Nix worker-protocol over gRPC.5723worker2 # [ 88.969902] systemd[1]: nix-grpc-daemon.service: Consumed 2.554s CPU time over 1min 11.948s wall clock time, 68.4M memory peak, 110.4K incoming IP traffic, 275.9K outgoing IP traffic.5724worker2 # [ 88.982162] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5725worker2 # [ 88.986309] systemd[1]: Closed Nix worker-protocol over gRPC.5726worker2 # [ 88.990710] systemd[1]: nix-grpc-daemon.socket: Consumed 326us CPU time over 1min 21.309s wall clock time, 256K memory peak, 119.1K incoming IP traffic, 101.8K outgoing IP traffic.5727worker1 # [ 89.013725] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:500515728lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5729lb: must succeed: curl -sf localhost:9901/clusters5730worker1 # [ 89.017731] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5731lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5732lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5733client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25734worker1 # [ 89.189337] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_take_over5735lb # [ 89.205716] niks3-server[970]: 2026/09/23 04:42:47 INFO lead: acquired remote=192.168.1.3:528585736worker1 # [ 89.208155] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566945737client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5738worker1 # [ 89.213283] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:566945739worker1 # [ 89.218198] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:500515740worker1 # [ 89.221904] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5741client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5742worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5743worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5744worker2 # [ 89.275218] systemd[1]: Listening on Nix worker-protocol over gRPC.5745worker1 # [ 89.340514] systemd[1]: Stopping Nix worker-protocol over gRPC...5746worker1 # [ 89.422907] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:500515747worker1 # [ 89.434209] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:392685748worker1 # [ 89.438008] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05749worker2 # [ 89.956863] systemd[1]: Starting scheduler-token.service...5750worker2 # [ 90.018092] systemd[1]: scheduler-token.service: Deactivated successfully.5751worker2 # [ 90.020318] systemd[1]: Finished scheduler-token.service.5752worker2 # [ 90.023207] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 61ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5753worker2 # [ 90.031363] systemd[1]: Starting Nix worker-protocol over gRPC...5754worker2 # [ 90.116662] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5755worker2 # [ 90.121162] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5756worker2 # [ 90.124346] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15757worker2 # [ 90.126300] nix-grpc-daemon[1334]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5758worker2 # [ 90.127568] nix-grpc-daemon[1334]: E0000 00:00:1790138568.255814 1334 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5759worker2 # [ 90.136770] nix-grpc-daemon[1334]: E0000 00:00:1790138568.264054 1334 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5760worker2 # [ 90.139087] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5761worker2 # [ 90.143887] systemd[1]: Started Nix worker-protocol over gRPC.5762worker2 # [ 90.146577] nix-grpc-daemon[1334]: E0000 00:00:1790138568.278588 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5763worker2 # [ 90.159907] nix-grpc-daemon[1334]: E0000 00:00:1790138568.291925 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5764worker2 # [ 90.164056] nix-grpc-daemon[1334]: E0000 00:00:1790138568.294198 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5765worker1 # [ 90.189435] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=signal_exit5766worker2 # [ 90.174384] nix-grpc-daemon[1334]: E0000 00:00:1790138568.306399 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5767lb # [ 90.211154] niks3-server[970]: 2026/09/23 04:42:48 INFO lead: released remote=192.168.1.3:528585768worker2 # [ 90.187072] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=scheduler_connected addr=lb:500515769worker1 # [ 90.207955] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:392685770worker1 # [ 90.213126] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05771worker2 # [ 91.177536] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting addr=lb:500515772worker1 # [ 91.194104] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting5773worker1 # [ 91.201368] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392685774worker1 # [ 91.206392] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_gone addr=192.168.1.4:500515775worker1 # [ 91.210653] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting addr=lb:500515776worker1 # [ 91.213871] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:392685777worker1 # [ 91.217290] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_gone addr=192.168.1.3:500515778worker1 # [ 91.237441] nix-daemon[846]: reaped child process 848, status = succeeded5779worker1 # [ 91.314638] nix-grpc-daemon[781]: E0000 00:00:1790138569.392717 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5780worker1 # [ 91.319554] nix-grpc-daemon[781]: E0000 00:00:1790138569.392852 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5781worker2 # [ 91.378196] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:500515782worker2 # [ 91.382299] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5783worker1 # [ 91.413399] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5784worker2 # [ 91.583429] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:500515785worker2 # [ 91.587524] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5786worker2 # [ 91.788722] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:500515787worker2 # [ 91.792721] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5788worker2 # [ 91.993907] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:500515789worker2 # [ 91.997957] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5790worker2 # [ 92.200452] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:500515791worker2 # [ 92.203589] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5792worker2 # [ 92.401553] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:500515793worker2 # [ 92.405440] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5794worker2 # [ 92.606623] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:500515795worker2 # [ 92.610561] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5796worker2 # [ 92.811888] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:500515797worker2 # [ 92.815864] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5798worker1 # [ 92.970947] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5799worker1 # [ 92.975298] systemd[1]: Stopped Nix worker-protocol over gRPC.5800worker1 # [ 92.978861] systemd[1]: nix-grpc-daemon.service: Consumed 1.894s CPU time over 1min 15.980s wall clock time, 60.7M memory peak, 83.5K incoming IP traffic, 153.4K outgoing IP traffic.5801worker1 # [ 92.991146] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5802worker1 # [ 92.995403] systemd[1]: Closed Nix worker-protocol over gRPC.5803worker1 # [ 92.998746] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 1min 25.313s wall clock time, 256K memory peak, 92.7K incoming IP traffic, 81.4K outgoing IP traffic.5804lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5805lb: must succeed: curl -sf localhost:9901/clusters5806worker2 # [ 93.016907] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:500515807lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5808lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5809client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25810worker2 # [ 93.020845] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5811worker2 # [ 93.162665] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367985812worker2 # [ 93.173767] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5813worker2 # [ 93.189806] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)5814worker2 # [ 93.222638] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:500515815worker2 # [ 93.226455] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker2 # [ 93.232384] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 2/5)5817worker2 # [ 93.269800] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 3/5)5818worker2 # [ 93.410590] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 600 ms (attempt 4/5)5819worker2 # [ 93.427391] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 93.429601] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker2 # [ 93.630541] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:500515822worker2 # [ 93.632541] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5823worker2 # [ 93.833422] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:500515824worker2 # [ 93.835524] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5825worker2 # [ 94.012576] nix-grpc-daemon[1334]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5826worker2 # [ 94.014110] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5827client # copying 2 paths...5828worker2 # [ 94.031072] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367985829client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5830client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5831worker2 # [ 94.035843] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 94.037707] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833worker2 # [ 94.041548] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5834worker2 # [ 94.054286] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=niks3_push_started pid=13955835worker2 # [ 94.153531] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.285Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5836worker2 # [ 94.239729] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515837worker2 # [ 94.242659] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker2 # [ 94.247066] nix-daemon[869]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5839worker2 # [ 94.259509] nix-daemon[869]: reaped child process 1410, status = succeeded5840lb # [ 94.302886] niks3-server[970]: 2026/09/23 04:42:52 INFO Received uploads request method=POST path=/api/pending_closures5841lb # [ 94.314564] niks3-server[970]: 2026/09/23 04:42:52 INFO Received uploads request method=POST path=/api/pending_closures5842worker2 # [ 94.288892] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.420Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5843worker2 # [ 94.292053] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.422Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5844worker2 # [ 94.293656] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.422Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5845lb # [ 94.369839] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5846lb # [ 94.376118] niks3-server[970]: 2026/09/23 04:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5847lb # [ 94.388489] niks3-server[970]: 2026/09/23 04:42:52 INFO Signed narinfos id=36 count=25848lb # [ 94.390262] niks3-server[970]: 2026/09/23 04:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5849lb # [ 94.392532] niks3-server[970]: 2026/09/23 04:42:52 INFO Signed narinfos id=35 count=15850worker2 # [ 94.360092] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.491Z level=INFO msg="Uploading 3 narinfos"5851lb # [ 94.396581] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5852lb # [ 94.401867] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5853lb # [ 94.409908] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5854lb # [ 94.419879] niks3-server[970]: 2026/09/23 04:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5855lb # [ 94.424795] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5856lb # [ 94.431084] niks3-server[970]: 2026/09/23 04:42:52 INFO Completed upload id=355857lb # [ 94.432715] niks3-server[970]: 2026/09/23 04:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5858lb # [ 94.436278] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5859lb # [ 94.439028] niks3-server[970]: 2026/09/23 04:42:52 INFO Completed upload id=365860worker2 # [ 94.404572] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.536Z level=INFO msg="Upload complete. (245ms)"5861lb # [ 94.442629] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5862worker2 # [ 94.409956] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=2 nar_bytes_in=3125863worker2 # [ 94.424220] nix-daemon[869]: reaped child process 1393, status = succeeded5864client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.42 seconds)5865worker2 # [ 94.442367] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515866worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5867worker2 # [ 94.446101] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5868worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5869worker1 # [ 94.533666] systemd[1]: Listening on Nix worker-protocol over gRPC.5870worker1 # [ 94.543270] systemd[1]: Starting Nix worker-protocol over gRPC...5871worker1 # [ 94.632394] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5872worker1 # [ 94.636928] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5873worker1 # [ 94.640110] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15874worker1 # [ 94.642088] nix-grpc-daemon[1166]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5875worker1 # [ 94.643390] nix-grpc-daemon[1166]: E0000 00:00:1790138572.715129 1166 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5876worker1 # [ 94.651849] nix-grpc-daemon[1166]: E0000 00:00:1790138572.724144 1166 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5877lb # [ 94.669348] niks3-server[970]: 2026/09/23 04:42:52 INFO lead: acquired remote=192.168.1.3:573705878worker1 # [ 94.656100] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5879worker1 # [ 94.659121] systemd[1]: Started Nix worker-protocol over gRPC.5880worker1 # [ 94.661055] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_take_over5881worker1 # [ 94.665140] nix-grpc-daemon[1166]: E0000 00:00:1790138572.733226 1207 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5882worker2 # [ 94.647261] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515883worker2 # [ 94.651129] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5884worker1 # [ 94.673207] nix-grpc-daemon[1166]: E0000 00:00:1790138572.750321 1207 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5885worker1 # [ 94.676183] nix-grpc-daemon[1166]: E0000 00:00:1790138572.750628 1207 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5886lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5887lb: must succeed: curl -sf localhost:9901/clusters5888worker1 # [ 94.687045] nix-grpc-daemon[1166]: E0000 00:00:1790138572.765266 1207 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5889worker1 # [ 94.698908] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515890worker1 # [ 94.700470] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5892worker2 # [ 94.852393] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515893worker2 # [ 94.856417] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5894worker1 # [ 94.901512] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:500515895worker1 # [ 94.905554] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5896worker2 # [ 95.057642] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:53Z level=info event=scheduler_connected addr=lb:500515897worker1 # [ 95.088171] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=oidc_jwks provider=mock keys=15898worker1 # [ 95.091905] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:397665899worker1 # [ 95.095828] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05900worker1 # [ 95.106111] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=scheduler_connected addr=lb:500515901worker1 # [ 95.110446] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:397665902worker1 # [ 95.113972] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05903lb: must succeed: curl -sf localhost:9901/clusters5904lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5905lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)5906(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.22 seconds)5907subtest: a niks3 restart does not cost the leader its role5908lb: must succeed: curl -sf localhost:9901/clusters5909lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5910lb: must succeed: curl -sf localhost:9901/clusters5911lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5912lb # [ 95.870370] niks3-server[970]: 2026/09/23 04:42:53 INFO Shutdown signal received, draining in-flight requests timeout=10s5913lb # [ 95.874350] systemd[1]: Stopping niks3 server...5914worker2 # [ 95.837243] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:53Z level=info event=scheduler_lock_lost status=2005915lb # [ 95.877431] niks3-server[970]: 2026/09/23 04:42:53 INFO lead: released remote=192.168.1.3:573705916worker1 # [ 95.862876] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=scheduler_lock_lost status=2005917lb # [ 95.888833] systemd[1]: niks3.service: Deactivated successfully.5918lb # [ 95.890445] systemd[1]: Stopped niks3 server.5919lb # [ 95.892080] systemd[1]: niks3.service: Consumed 750ms CPU time over 1min 16.921s wall clock time, 29M memory peak, 100.7K incoming IP traffic, 143.9K outgoing IP traffic.5920lb # [ 95.898440] systemd[1]: Starting niks3 server...5921lb # [ 95.948352] niks3-server[1112]: 2026/09/23 04:42:53 goose: no migrations to run. current version: 202609200000005922lb # [ 95.953495] niks3-server[1112]: 2026/09/23 04:42:53 OK 1_commit_pending_closure.sql (5.98ms)5923lb # [ 95.956993] niks3-server[1112]: 2026/09/23 04:42:53 OK 2_object_stats_trigger.sql (3.13ms)5924lb # [ 95.958073] niks3-server[1112]: 2026/09/23 04:42:53 goose: up to current file version: 25925lb # [ 95.959393] niks3-server[1112]: 2026/09/23 04:42:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5926lb # [ 95.970268] niks3-server[1112]: 2026/09/23 04:42:53 INFO Read proxy enabled — serving cache objects from S35927lb # [ 95.971626] niks3-server[1112]: 2026/09/23 04:42:53 INFO Using socket-activated listener address=0.0.0.0:57515928lb # [ 95.973080] niks3-server[1112]: 2026/09/23 04:42:53 INFO systemd watchdog enabled interval=15s5929lb # [ 95.974686] systemd[1]: Started niks3 server.5930lb # [ 95.975766] niks3-server[1112]: 2026/09/23 04:42:53 INFO Starting HTTP server address=0.0.0.0:57515931lb: waiting for unit niks3.service5932lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5933lb: must succeed: curl -sf localhost:9901/clusters5934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5935worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5936worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5937lb: must succeed: curl -sf localhost:9901/clusters5938lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5939lb: must succeed: curl -sf localhost:9901/clusters5940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5941worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5942worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5943client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25944worker1 # [ 96.284155] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:397745945worker1 # [ 96.296224] nix-daemon[846]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)5946client # don't know how to build these paths:5947client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5948worker2 # [ 96.293180] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367985949worker1 # [ 96.336754] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05950worker1 # [ 96.341150] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15951worker2 # [ 96.371926] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367985952worker2 # [ 96.384736] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367985953client # copying 1 paths to 'lb:50051'...5954client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5955worker2 # [ 96.397232] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367985956worker2 # [ 96.408516] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5957worker2 # [ 96.515471] nix-daemon[869]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5958worker2 # [ 96.529402] nix-daemon[869]: reaped child process 1420, status = succeeded5959lb # [ 96.571310] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received uploads request method=POST path=/api/pending_closures5960worker2 # [ 96.547781] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.679Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5961worker2 # [ 96.550275] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.681Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5962lb # [ 96.610797] niks3-server[1112]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5963lb # [ 96.616617] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5964lb # [ 96.635994] niks3-server[1112]: 2026/09/23 04:42:54 INFO Signed narinfos id=37 count=15965worker2 # [ 96.604811] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.736Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 96.652861] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5967lb # [ 96.657775] niks3-server[1112]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5968lb # [ 96.660974] niks3-server[1112]: 2026/09/23 04:42:54 INFO Completed upload id=375969worker2 # [ 96.626374] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.758Z level=INFO msg="Upload complete. (205ms)"5970worker2 # [ 96.631585] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=5365971worker2 # [ 96.638450] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367985972worker2 # [ 96.650141] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5973worker2 # [ 96.662177] nix-daemon[869]: reaped child process 1415, status = succeeded5974worker2 # [ 96.664932] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5975client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5976client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5977lb # [ 96.883161] niks3-server[1112]: 2026/09/23 04:42:54 INFO lead: acquired remote=192.168.1.3:573865978lb # [ 97.010604] envoy[927]: conn peer=[2001:db8:1::4]:50354 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=51856 tx=3112 ms=711815979lb # [ 97.014441] envoy[927]: conn peer=[2001:db8:1::1]:36178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155980lb # [ 97.017874] envoy[927]: conn peer=[2001:db8:1::3]:59812 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15015 tx=3132 ms=741875981lb # [ 97.021544] envoy[927]: conn peer=[2001:db8:1::1]:45052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=305 ms=12615982client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5983client # farm-a> LOG-farm-a-after-niks3-restart5984worker2 # [ 97.897460] nix-daemon[869]: reaped child process 1424, status = succeeded5985worker2 # [ 97.990820] nix-daemon[869]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5986worker2 # [ 98.003701] nix-daemon[869]: reaped child process 1445, status = succeeded5987lb # [ 98.045853] niks3-server[1112]: 2026/09/23 04:42:55 INFO Received uploads request method=POST path=/api/pending_closures5988worker2 # [ 98.019773] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5989worker2 # [ 98.021470] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.153Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5990lb # [ 98.084427] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5991lb # [ 98.113520] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5992lb # [ 98.117304] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5993lb # [ 98.123530] niks3-server[1112]: 2026/09/23 04:42:56 INFO Signed narinfos id=38 count=15994lb # [ 98.126259] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5995worker2 # [ 98.091807] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.223Z level=INFO msg="Uploading 1 narinfos"5996lb # [ 98.132551] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5997lb # [ 98.137464] niks3-server[1112]: 2026/09/23 04:42:56 INFO Completed upload id=385998worker2 # [ 98.102839] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.234Z level=INFO msg="Upload complete. (228ms)"5999lb # [ 98.142405] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6000worker2 # [ 98.109895] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16001worker1 # [ 98.137973] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:56Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06002worker2 # [ 98.125715] nix-daemon[869]: reaped child process 1422, status = succeeded6003worker1 # [ 98.142155] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:56Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16004worker2 # [ 98.163256] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986005worker2 # [ 98.178176] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986006worker2 # [ 98.191760] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.323Z level=INFO msg="All 1 paths already cached"6007client # copying 1 paths to 'lb:50051'...6008client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6009worker2 # [ 98.200653] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986010worker2 # [ 98.211084] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6011worker2 # [ 98.308799] nix-daemon[869]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)6012worker2 # [ 98.321684] nix-daemon[869]: reaped child process 1452, status = succeeded6013lb # [ 98.363977] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received uploads request method=POST path=/api/pending_closures6014worker2 # [ 98.339416] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.471Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6015worker2 # [ 98.341097] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.473Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6016lb # [ 98.397360] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6017worker2 # [ 98.367082] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.499Z level=INFO msg="Uploading 1 narinfos"6018lb # [ 98.403227] niks3-server[1112]: 2026/09/23 04:42:56 INFO Signed narinfos id=39 count=16019lb # [ 98.413712] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6020lb # [ 98.419260] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6021lb # [ 98.430996] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6022lb # [ 98.438775] niks3-server[1112]: 2026/09/23 04:42:56 INFO Completed upload id=396023worker2 # [ 98.403649] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.535Z level=INFO msg="Upload complete. (180ms)"6024lb # [ 98.442325] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6025worker2 # [ 98.408405] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=6566026worker2 # [ 98.414461] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986027worker2 # [ 98.426109] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6028worker2 # [ 98.433679] nix-daemon[869]: reaped child process 1447, status = succeeded6029worker2 # [ 98.438503] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6030client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6031client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6032client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6033client # farm-b> LOG-farm-b-after-niks3-restart6034worker2 # [ 99.677422] nix-daemon[869]: reaped child process 1457, status = succeeded6035worker2 # [ 99.758919] nix-daemon[869]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6036worker2 # [ 99.771607] nix-daemon[869]: reaped child process 1476, status = succeeded6037lb # [ 99.814795] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received uploads request method=POST path=/api/pending_closures6038worker2 # [ 99.789246] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.921Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6039worker2 # [ 99.791035] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.922Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6040lb # [ 99.857617] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6041lb # [ 99.866331] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6042lb # [ 99.874555] niks3-server[1112]: 2026/09/23 04:42:57 INFO Signed narinfos id=40 count=16043worker2 # [ 99.843573] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.975Z level=INFO msg="Uploading 1 narinfos"6044lb # [ 99.883624] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6045lb # [ 99.892243] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6046lb # [ 99.899124] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6047lb # [ 99.907963] niks3-server[1112]: 2026/09/23 04:42:57 INFO Completed upload id=406048worker2 # [ 99.873505] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.005Z level=INFO msg="Upload complete. (227ms)"6049lb # [ 99.910495] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6050worker2 # [ 99.879793] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16051worker1 # [ 99.905560] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:57Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06052worker1 # [ 99.909634] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:57Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16053worker2 # [ 99.896318] nix-daemon[869]: reaped child process 1455, status = succeeded6054worker2 # [ 99.930764] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986055worker2 # [ 99.942927] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986056worker2 # [ 99.956092] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.087Z level=INFO msg="All 2 paths already cached"6057client # copying 1 paths to 'lb:50051'...6058client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6059worker2 # [ 99.965438] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986060worker2 # [ 99.975346] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6061worker2 # [ 100.073050] nix-daemon[869]: accepted connection from pid 1480, user nix-grpc-daemon (trusted)6062worker2 # [ 100.086289] nix-daemon[869]: reaped child process 1483, status = succeeded6063lb # [ 100.127826] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received uploads request method=POST path=/api/pending_closures6064worker2 # [ 100.110391] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.242Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6065worker2 # [ 100.111880] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.243Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6066lb # [ 100.167935] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6067lb # [ 100.176453] niks3-server[1112]: 2026/09/23 04:42:58 INFO Signed narinfos id=41 count=16068worker2 # [ 100.145495] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.277Z level=INFO msg="Uploading 1 narinfos"6069lb # [ 100.189902] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6070lb # [ 100.196147] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6071lb # [ 100.206317] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6072lb # [ 100.215441] niks3-server[1112]: 2026/09/23 04:42:58 INFO Completed upload id=416073worker2 # [ 100.180372] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.312Z level=INFO msg="Upload complete. (192ms)"6074lb # [ 100.219129] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6075worker2 # [ 100.185575] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=7846076worker2 # [ 100.191296] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986077worker2 # [ 100.203111] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6078worker2 # [ 100.210934] nix-daemon[869]: reaped child process 1478, status = succeeded6079worker2 # [ 100.218834] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6080client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6081client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6082client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6083client # farm-top> LOG-farm-top-after-niks3-restart6084worker2 # [ 101.054898] nix-daemon[869]: reaped child process 1487, status = succeeded6085worker2 # [ 101.125967] nix-daemon[869]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)6086worker2 # [ 101.139814] nix-daemon[869]: reaped child process 1506, status = succeeded6087lb # [ 101.182389] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received uploads request method=POST path=/api/pending_closures6088worker2 # [ 101.161244] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.293Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6089worker2 # [ 101.162934] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.294Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6090lb # [ 101.227503] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6091lb # [ 101.233094] niks3-server[1112]: 2026/09/23 04:42:59 INFO Signed narinfos id=42 count=16092worker2 # [ 101.200661] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.332Z level=INFO msg="Uploading 1 narinfos"6093lb # [ 101.245274] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6094lb # [ 101.283507] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6095lb # [ 101.286795] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6096lb # [ 101.290990] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6097lb # [ 101.298176] niks3-server[1112]: 2026/09/23 04:42:59 INFO Completed upload id=426098worker2 # [ 101.263222] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.395Z level=INFO msg="Upload complete. (237ms)"6099lb # [ 101.301647] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6100worker2 # [ 101.268917] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16101worker2 # [ 101.284399] nix-daemon[869]: reaped child process 1485, status = succeeded6102client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.19 seconds)6103client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6104client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6105(finished: subtest: a niks3 restart does not cost the leader its role, in 5.66 seconds)6106subtest: scheduler down: the other node takes the lock and keeps it6107lb: must succeed: curl -sf localhost:9901/clusters6108lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6109lb: must succeed: curl -sf localhost:9901/clusters6110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6111lb: must succeed: curl -sf localhost:9901/clusters6112lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6113lb: must succeed: curl -sf localhost:9901/clusters6114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6115worker1 # [ 101.598873] systemd[1]: Stopping Nix worker-protocol over gRPC...6116worker1 # [ 101.649259] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:59Z level=info event=signal_exit6117lb # [ 101.885938] niks3-server[1112]: 2026/09/23 04:42:59 INFO lead: released remote=192.168.1.3:573866118worker2 # [ 102.853791] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting addr=lb:500516119worker1 # [ 102.875567] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting6120worker1 # [ 102.878956] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting addr=lb:500516121worker1 # [ 102.883257] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397666122worker1 # [ 102.886631] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_gone addr=192.168.1.3:500516123worker1 # [ 102.890086] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397666124worker1 # [ 102.894230] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_gone addr=192.168.1.4:500516125worker1 # [ 102.904312] nix-daemon[846]: reaped child process 1227, status = succeeded6126worker1 # [ 102.989863] nix-grpc-daemon[1166]: E0000 00:00:1790138581.067863 1212 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6127worker1 # [ 102.995446] nix-grpc-daemon[1166]: E0000 00:00:1790138581.073441 1212 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6128worker2 # [ 103.054143] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:500516129worker2 # [ 103.058415] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6130worker2 # [ 103.259299] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:500516131worker2 # [ 103.263315] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6132worker2 # [ 103.464589] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:500516133worker2 # [ 103.468576] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6134worker2 # [ 103.669867] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:500516135worker2 # [ 103.673855] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6136worker2 # [ 103.875790] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:500516137worker2 # [ 103.879590] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6138worker1 # [ 104.079410] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6139worker1 # [ 104.083421] systemd[1]: Stopped Nix worker-protocol over gRPC.6140worker1 # [ 104.087199] systemd[1]: nix-grpc-daemon.service: Consumed 214ms CPU time over 9.541s wall clock time, 14.1M memory peak, 10.4K incoming IP traffic, 15.1K outgoing IP traffic.6141worker1 # [ 104.097864] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6142worker2 # [ 104.080798] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 104.102357] systemd[1]: Closed Nix worker-protocol over gRPC.6144worker2 # [ 104.084797] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6145worker1 # [ 104.105677] systemd[1]: nix-grpc-daemon.socket: Consumed 521us CPU time over 9.573s wall clock time, 256K memory peak, 21.9K incoming IP traffic, 16.4K outgoing IP traffic.6146worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6147worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)6148worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6149worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6150worker2 # [ 104.285911] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:500516151worker2 # [ 104.289864] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6152worker2 # [ 104.490772] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:500516153worker2 # [ 104.494947] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154worker2 # [ 104.696164] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:500516155worker2 # [ 104.700135] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6156worker2 # [ 104.902483] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:500516157worker2 # [ 104.905835] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6158worker2 # [ 105.106816] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:500516159worker2 # [ 105.110789] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6160worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6161worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6162worker2 # [ 105.311910] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:500516163worker2 # [ 105.315875] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6164worker2 # [ 105.517192] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:500516165worker2 # [ 105.521173] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6166worker2 # [ 105.722203] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:500516167worker2 # [ 105.726281] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6168worker2 # [ 105.928213] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:500516169worker2 # [ 105.931840] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6170worker2 # [ 106.132882] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:500516171worker2 # [ 106.137106] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6172worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6173worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6174worker2 # [ 106.338141] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:500516175worker2 # [ 106.342116] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6176worker2 # [ 106.543385] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:500516177worker2 # [ 106.547322] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker2 # [ 106.748442] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 106.752397] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker2 # [ 106.842351] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_take_over6181lb # [ 106.877612] niks3-server[1112]: 2026/09/23 04:43:04 INFO lead: acquired remote=192.168.1.4:604906182worker2 # [ 106.954310] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:500516183worker2 # [ 106.958196] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6184lb # [ 107.010662] envoy[927]: conn peer=[2001:db8:1::1]:45064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9662 tx=4387 ms=50296185lb # [ 107.014407] envoy[927]: conn peer=[2001:db8:1::3]:48558 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=950 tx=293 ms=83836186worker2 # [ 107.159208] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:500516187worker2 # [ 107.163342] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6188worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6189worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6190worker2 # [ 107.364436] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:500516191worker2 # [ 107.382143] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=oidc_jwks provider=mock keys=16192worker2 # [ 107.386096] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:362886193worker2 # [ 107.389959] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06194worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6195worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6196client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26197client # don't know how to build these paths:6198client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6199worker2 # [ 108.494783] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367986200worker2 # [ 108.501075] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986201worker2 # [ 108.519679] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06202worker2 # [ 108.524794] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16203worker2 # [ 108.564152] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986204worker2 # [ 108.576647] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986205client # copying 1 paths to 'lb:50051'...6206client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6207worker2 # [ 108.589236] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986208worker2 # [ 108.600783] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6209worker2 # [ 108.700216] nix-daemon[869]: accepted connection from pid 1533, user nix-grpc-daemon (trusted)6210worker2 # [ 108.713655] nix-daemon[869]: reaped child process 1536, status = succeeded6211lb # [ 108.755635] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received uploads request method=POST path=/api/pending_closures6212worker2 # [ 108.729179] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6213worker2 # [ 108.731171] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.862Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6214lb # [ 108.787647] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6215worker2 # [ 108.762720] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.894Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 108.797678] niks3-server[1112]: 2026/09/23 04:43:06 INFO Signed narinfos id=43 count=16217lb # [ 108.809923] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6218lb # [ 109.104429] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6219lb # [ 109.111617] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6220lb # [ 109.130742] niks3-server[1112]: 2026/09/23 04:43:07 INFO Completed upload id=436221worker2 # [ 109.096088] nix-grpc-daemon[1395]: time=2026-09-23T04:43:07.227Z level=INFO msg="Upload complete. (483ms)"6222worker2 # [ 109.101134] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=5206223lb # [ 109.139155] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6224worker2 # [ 109.109365] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986225worker2 # [ 109.121642] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6226worker2 # [ 109.129554] nix-daemon[869]: reaped child process 1531, status = succeeded6227client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6228client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6229worker2 # [ 109.142825] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6230client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6231client # farm-a> LOG-farm-a-on-standby6232worker2 # [ 110.765972] nix-daemon[869]: reaped child process 1540, status = succeeded6233worker2 # [ 110.841471] nix-daemon[869]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6234worker2 # [ 110.853852] nix-daemon[869]: reaped child process 1559, status = succeeded6235lb # [ 110.895488] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received uploads request method=POST path=/api/pending_closures6236worker2 # [ 110.910553] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.042Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6237worker2 # [ 110.912374] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.044Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6238lb # [ 110.983248] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6239lb # [ 110.991890] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6240lb # [ 110.997802] niks3-server[1112]: 2026/09/23 04:43:08 INFO Signed narinfos id=44 count=16241worker2 # [ 110.966528] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.098Z level=INFO msg="Uploading 1 narinfos"6242lb # [ 111.003767] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6243lb # [ 111.010225] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6244lb # [ 111.019279] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6245lb # [ 111.029336] niks3-server[1112]: 2026/09/23 04:43:08 INFO Completed upload id=446246worker2 # [ 110.994700] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.126Z level=INFO msg="Upload complete. (238ms)"6247lb # [ 111.035571] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6248worker2 # [ 111.001621] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16249worker2 # [ 111.016559] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06250worker2 # [ 111.021236] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16251worker2 # [ 111.033162] nix-daemon[869]: reaped child process 1538, status = succeeded6252worker2 # [ 111.061263] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986253worker2 # [ 111.074141] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986254worker2 # [ 111.087918] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.219Z level=INFO msg="All 1 paths already cached"6255client # copying 1 paths to 'lb:50051'...6256client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6257worker2 # [ 111.097126] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986258worker2 # [ 111.105227] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6259worker2 # [ 111.204246] nix-daemon[869]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)6260worker2 # [ 111.217775] nix-daemon[869]: reaped child process 1568, status = succeeded6261lb # [ 111.259356] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received uploads request method=POST path=/api/pending_closures6262worker2 # [ 111.232436] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.364Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6263worker2 # [ 111.234308] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.366Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6264lb # [ 111.288364] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6265worker2 # [ 111.264195] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.396Z level=INFO msg="Uploading 1 narinfos"6266lb # [ 111.298744] niks3-server[1112]: 2026/09/23 04:43:09 INFO Signed narinfos id=45 count=16267lb # [ 111.308439] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6268lb # [ 111.313971] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6269lb # [ 111.324901] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6270lb # [ 111.330469] niks3-server[1112]: 2026/09/23 04:43:09 INFO Completed upload id=456271worker2 # [ 111.297398] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.429Z level=INFO msg="Upload complete. (178ms)"6272lb # [ 111.334029] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6273worker2 # [ 111.301477] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=6406274worker2 # [ 111.308352] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986275worker2 # [ 111.317496] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6276worker2 # [ 111.326469] nix-daemon[869]: reaped child process 1563, status = succeeded6277client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6278worker2 # [ 111.332370] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6279client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6280client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6281client # farm-b> LOG-farm-b-on-standby6282worker2 # [ 112.569510] nix-daemon[869]: reaped child process 1572, status = succeeded6283worker2 # [ 112.652946] nix-daemon[869]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)6284worker2 # [ 112.667477] nix-daemon[869]: reaped child process 1591, status = succeeded6285lb # [ 112.707473] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received uploads request method=POST path=/api/pending_closures6286worker2 # [ 112.681557] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.813Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6287worker2 # [ 112.683098] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.815Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6288lb # [ 112.745185] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6289lb # [ 112.759913] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6290worker2 # [ 112.735809] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.867Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 112.770193] niks3-server[1112]: 2026/09/23 04:43:10 INFO Signed narinfos id=46 count=16292lb # [ 112.773229] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6293lb # [ 112.778569] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6294lb # [ 112.788924] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6295lb # [ 112.791702] niks3-server[1112]: 2026/09/23 04:43:10 INFO Completed upload id=466296worker2 # [ 112.757087] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.888Z level=INFO msg="Upload complete. (217ms)"6297lb # [ 112.794095] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6298worker2 # [ 112.765702] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16299worker2 # [ 112.776276] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06300worker2 # [ 112.780196] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16301worker2 # [ 112.789910] nix-daemon[869]: reaped child process 1570, status = succeeded6302worker2 # [ 112.819904] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986303worker2 # [ 112.832277] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986304worker2 # [ 112.846288] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.978Z level=INFO msg="All 2 paths already cached"6305client # copying 1 paths to 'lb:50051'...6306client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6307worker2 # [ 112.855299] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986308worker2 # [ 112.865165] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6309worker2 # [ 112.960590] nix-daemon[869]: accepted connection from pid 1597, user nix-grpc-daemon (trusted)6310worker2 # [ 112.975761] nix-daemon[869]: reaped child process 1600, status = succeeded6311lb # [ 113.016985] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received uploads request method=POST path=/api/pending_closures6312worker2 # [ 113.002338] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.134Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6313worker2 # [ 113.003870] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.135Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6314lb # [ 113.063452] niks3-server[1112]: 2026/09/23 04:43:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6315lb # [ 113.072995] niks3-server[1112]: 2026/09/23 04:43:11 INFO Signed narinfos id=47 count=16316worker2 # [ 113.041865] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.173Z level=INFO msg="Uploading 1 narinfos"6317lb # [ 113.079092] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6318lb # [ 113.082673] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6319lb # [ 113.096245] niks3-server[1112]: 2026/09/23 04:43:11 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6320lb # [ 113.104138] niks3-server[1112]: 2026/09/23 04:43:11 INFO Completed upload id=476321worker2 # [ 113.069366] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.201Z level=INFO msg="Upload complete. (193ms)"6322lb # [ 113.109526] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6323worker2 # [ 113.074170] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=7686324worker2 # [ 113.081111] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986325worker2 # [ 113.093547] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6326worker2 # [ 113.102696] nix-daemon[869]: reaped child process 1595, status = succeeded6327worker2 # [ 113.111072] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6328client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6329client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6330client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6331client # farm-top> LOG-farm-top-on-standby6332worker2 # [ 113.955854] nix-daemon[869]: reaped child process 1604, status = succeeded6333worker2 # [ 114.030914] nix-daemon[869]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6334worker2 # [ 114.044893] nix-daemon[869]: reaped child process 1623, status = succeeded6335lb # [ 114.086521] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received uploads request method=POST path=/api/pending_closures6336worker2 # [ 114.060514] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.192Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6337worker2 # [ 114.062309] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.194Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6338lb # [ 114.136305] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6339lb # [ 114.141182] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6340lb # [ 114.149180] niks3-server[1112]: 2026/09/23 04:43:12 INFO Signed narinfos id=48 count=16341worker2 # [ 114.117576] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.248Z level=INFO msg="Uploading 1 narinfos"6342lb # [ 114.152848] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6343lb # [ 114.158151] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6344lb # [ 114.167714] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6345lb # [ 114.173639] niks3-server[1112]: 2026/09/23 04:43:12 INFO Completed upload id=486346lb # [ 114.174970] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6347worker2 # [ 114.139094] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.270Z level=INFO msg="Upload complete. (220ms)"6348worker2 # [ 114.147777] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16349worker2 # [ 114.161590] nix-daemon[869]: reaped child process 1602, status = succeeded6350client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.86 seconds)6351client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6352client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6353worker1 # [ 114.399604] systemd[1]: Listening on Nix worker-protocol over gRPC.6354worker1 # [ 114.406597] systemd[1]: Starting Nix worker-protocol over gRPC...6355worker1 # [ 114.497106] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6356worker1 # [ 114.499212] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6357worker1 # [ 114.502514] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16358worker1 # [ 114.506798] nix-grpc-daemon[1253]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6359worker1 # [ 114.508141] nix-grpc-daemon[1253]: E0000 00:00:1790138592.584743 1253 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6360worker1 # [ 114.514275] nix-grpc-daemon[1253]: E0000 00:00:1790138592.590140 1253 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6361worker1 # [ 114.516397] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6362worker1 # [ 114.517927] systemd[1]: Started Nix worker-protocol over gRPC.6363worker1 # [ 114.525060] nix-grpc-daemon[1253]: E0000 00:00:1790138592.601583 1295 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6364lb: must succeed: curl -sf localhost:9901/clusters6365worker1 # [ 114.536747] nix-grpc-daemon[1253]: E0000 00:00:1790138592.614993 1295 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6366worker1 # [ 114.540143] nix-grpc-daemon[1253]: E0000 00:00:1790138592.617178 1295 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6367worker1 # [ 114.551045] nix-grpc-daemon[1253]: E0000 00:00:1790138592.629278 1295 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6368worker1 # [ 114.560993] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=scheduler_connected addr=lb:500516369worker2 # [ 114.545077] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:362886370worker2 # [ 114.548807] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6372worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6373worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6374lb: must succeed: curl -sf localhost:9901/clusters6375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6376lb: must succeed: curl -sf localhost:9901/clusters6377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6378worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6379worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6380(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.30 seconds)6381subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6382client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6383client # [ 114.798794] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6384client # Running as unit: rs.service; invocation ID: 4747fc9c4a364f099d01790495a7f9a76385client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6386worker2 # [ 114.905505] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:367986387client # [ 114.943613] nix[1365]: don't know how to build these paths:6388client # [ 114.944650] nix[1365]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6389worker2 # [ 114.918882] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06390worker2 # [ 114.923994] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16391worker2 # [ 114.962796] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986392worker2 # [ 114.975510] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367986393client # [ 115.018227] nix[1365]: copying 1 paths to 'lb:50051'...6394worker2 # [ 114.987723] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367986395client # [ 115.020367] nix[1365]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6396worker2 # [ 115.000069] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6397worker2 # [ 115.097461] nix-daemon[869]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6398worker2 # [ 115.110355] nix-daemon[869]: reaped child process 1639, status = succeeded6399lb # [ 115.152785] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received uploads request method=POST path=/api/pending_closures6400worker2 # [ 115.126335] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6401worker2 # [ 115.128294] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.260Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6402lb # [ 115.185494] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6403worker2 # [ 115.159689] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.291Z level=INFO msg="Uploading 1 narinfos"6404lb # [ 115.194423] niks3-server[1112]: 2026/09/23 04:43:13 INFO Signed narinfos id=49 count=16405lb # [ 115.198395] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6406lb # [ 115.206090] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6407lb # [ 115.223112] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6408lb # [ 115.228180] niks3-server[1112]: 2026/09/23 04:43:13 INFO Completed upload id=496409worker2 # [ 115.193505] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.325Z level=INFO msg="Upload complete. (182ms)"6410lb # [ 115.230076] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6411worker2 # [ 115.199428] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=4166412worker2 # [ 115.206377] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:367986413worker2 # [ 115.215262] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6414worker2 # [ 115.225291] nix-daemon[869]: reaped child process 1634, status = succeeded6415worker2 # [ 115.229897] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6416client # [ 115.261279] nix[1365]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6417lb # [ 117.011386] envoy[927]: conn peer=[2001:db8:1::1]:52648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9590 tx=4414 ms=57046418lb: must succeed: curl -sf localhost:9901/clusters6419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6420lb: must succeed: curl -sf localhost:9901/clusters6421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6422lb: must succeed: curl -sf localhost:9901/clusters6423lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6424lb: must succeed: curl -sf localhost:9901/clusters6425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6426worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6427worker1 # [ 117.459874] systemd-journald[283]: Received client request to rotate journal, rotating.6428worker1 # [ 117.479843] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/055d72bd56914de896f84ccd0e161ef1.6429worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6430worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6431worker2 # [ 117.368280] systemd[1]: Stopping Nix worker-protocol over gRPC...6432worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6433client: must succeed: sleep 5.06434worker2 # [ 118.145392] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:16Z level=info event=draining6435client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6436worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6437worker2 # [ 122.653490] nix-daemon[869]: reaped child process 1643, status = succeeded6438worker2 # [ 122.740869] nix-daemon[869]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6439worker2 # [ 122.754760] nix-daemon[869]: reaped child process 1687, status = succeeded6440lb # [ 122.797742] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received uploads request method=POST path=/api/pending_closures6441worker2 # [ 122.774176] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6442worker2 # [ 122.775720] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.907Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6443lb # [ 122.841680] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6444lb # [ 122.847268] niks3-server[1112]: 2026/09/23 04:43:20 INFO Signed narinfos id=50 count=16445worker2 # [ 122.817040] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.948Z level=INFO msg="Uploading 1 narinfos"6446lb # [ 122.856591] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6447lb # [ 122.862371] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6448lb # [ 122.866792] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6449lb # [ 122.875981] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6450lb # [ 122.879603] niks3-server[1112]: 2026/09/23 04:43:20 INFO Completed upload id=506451worker2 # [ 122.844933] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.976Z level=INFO msg="Upload complete. (201ms)"6452lb # [ 122.881922] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6453worker2 # [ 122.850683] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16454worker2 # [ 122.864880] nix-daemon[869]: reaped child process 1641, status = succeeded6455client # [ 122.905511] systemd[1]: rs.service: Deactivated successfully.6456client # [ 122.907071] systemd[1]: rs.service: Consumed 125ms CPU time over 8.108s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.6457worker2 # [ 123.147610] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:21Z level=info event=signal_exit6458lb # [ 123.882877] niks3-server[1112]: 2026/09/23 04:43:21 INFO lead: released remote=192.168.1.4:604906459lb # [ 124.539718] niks3-server[1112]: 2026/09/23 04:43:22 INFO lead: acquired remote=192.168.1.3:498766460worker1 # [ 124.527805] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:22Z level=info event=scheduler_take_over6461worker1 # [ 124.867379] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting addr=lb:500516462worker2 # [ 124.851713] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting6463worker2 # [ 124.857764] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:362886464worker2 # [ 124.862264] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_gone addr=192.168.1.3:500516465worker2 # [ 124.866228] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting addr=lb:500516466worker2 # [ 124.869793] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:362886467worker2 # [ 124.873791] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_gone addr=192.168.1.4:500516468worker2 # [ 124.894273] nix-daemon[869]: reaped child process 1387, status = succeeded6469worker1 # [ 125.067965] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_connected addr=lb:500516470worker1 # [ 125.072162] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6471worker2 # [ 125.067552] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6472worker2 # [ 125.213489] nix-grpc-daemon[1334]: E0000 00:00:1790138603.345089 1380 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 13, started true, shutdown true6473worker2 # [ 125.220232] nix-grpc-daemon[1334]: E0000 00:00:1790138603.352194 1380 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6474worker1 # [ 125.272842] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_connected addr=lb:500516475worker1 # [ 125.280173] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:533266476worker1 # [ 125.283832] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06477worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.18 seconds)6478client: waiting for success: ! systemctl is-active rs6479client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6480client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6481client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6482client: must fail: journalctl -u rs | grep -q 'reconnecting'6483client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6484lb: must succeed: curl -sf localhost:9901/clusters6485lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6486worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6487worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6488worker2 # [ 126.650360] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6489worker2 # [ 126.653720] systemd[1]: Stopped Nix worker-protocol over gRPC.6490worker2 # [ 126.657544] systemd[1]: nix-grpc-daemon.service: Consumed 1.778s CPU time over 36.618s wall clock time, 45.4M memory peak, 89.7K incoming IP traffic, 243.8K outgoing IP traffic.6491worker2 # [ 126.674180] systemd[1]: Starting scheduler-token.service...6492worker2 # [ 126.722389] systemd[1]: scheduler-token.service: Deactivated successfully.6493worker2 # [ 126.724076] systemd[1]: Finished scheduler-token.service.6494worker2 # [ 126.725311] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 50ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6495worker2 # [ 126.729936] systemd[1]: Starting Nix worker-protocol over gRPC...6496worker2 # [ 126.818228] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6497worker2 # [ 126.823070] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6498worker2 # [ 126.827907] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16499worker2 # [ 126.832133] nix-grpc-daemon[1697]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6500worker2 # [ 126.835220] nix-grpc-daemon[1697]: E0000 00:00:1790138604.958089 1697 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6501worker2 # [ 126.838481] nix-grpc-daemon[1697]: E0000 00:00:1790138604.959817 1697 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6502worker2 # [ 126.841180] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6503worker2 # [ 126.843367] systemd[1]: Started Nix worker-protocol over gRPC.6504worker2 # [ 126.846099] nix-grpc-daemon[1697]: E0000 00:00:1790138604.976671 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6505worker2 # [ 126.856696] nix-grpc-daemon[1697]: E0000 00:00:1790138604.988704 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6506worker2 # [ 126.861054] nix-grpc-daemon[1697]: E0000 00:00:1790138604.991047 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6507worker2 # [ 126.870915] nix-grpc-daemon[1697]: E0000 00:00:1790138605.002922 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6508worker2 # [ 126.880828] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:25Z level=info event=scheduler_connected addr=lb:500516509worker1 # [ 126.907806] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=oidc_jwks provider=mock keys=16510worker1 # [ 126.912149] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:533266511worker1 # [ 126.915979] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06512lb: must succeed: curl -sf localhost:9901/clusters6513lb # [ 127.012791] envoy[927]: conn peer=[2001:db8:1::1]:57328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2999 tx=1540 ms=79586514lb # [ 127.017550] envoy[927]: conn peer=[2001:db8:1::4]:44742 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36575 tx=2345 ms=348706515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6516worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6517worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6518(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.33 seconds)6519subtest: scheduler restart mid-build: build finishes, no double build6520client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6521client # [ 127.129684] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6522client # Running as unit: mid.service; invocation ID: 681202239daa43c4a761e48aa40e08036523client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6524worker1 # [ 127.254255] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533426525worker1 # [ 127.266264] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)6526client # [ 127.291353] nix[1401]: don't know how to build these paths:6527client # [ 127.292302] nix[1401]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6528worker1 # [ 127.287772] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06529worker1 # [ 127.292648] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16530worker1 # [ 127.336718] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533426531worker1 # [ 127.347604] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)6532worker1 # [ 127.363195] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 2/5)6533worker1 # [ 127.372183] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 3/5)6534worker1 # [ 127.494756] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 632 ms (attempt 4/5)6535worker1 # [ 128.137556] nix-grpc-daemon[1253]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6536worker1 # [ 128.141559] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6537worker1 # [ 128.158955] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533426538client # [ 128.182200] nix[1401]: copying 1 paths to 'lb:50051'...6539client # [ 128.186301] nix[1401]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6540worker1 # [ 128.174403] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533426541worker1 # [ 128.182946] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)6542worker1 # [ 128.204402] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=info event=niks3_push_started pid=13826543worker1 # [ 128.317657] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.394Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6544worker1 # [ 128.402899] nix-daemon[846]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)6545worker1 # [ 128.415072] nix-daemon[846]: reaped child process 1402, status = succeeded6546lb # [ 128.438545] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received uploads request method=POST path=/api/pending_closures6547worker1 # [ 128.431658] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6548worker1 # [ 128.433535] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.511Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6549lb # [ 128.475188] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6550lb # [ 128.484629] niks3-server[1112]: 2026/09/23 04:43:26 INFO Signed narinfos id=51 count=16551worker1 # [ 128.473457] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.551Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 128.491097] niks3-server[1112]: 2026/09/23 04:43:26 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6553lb # [ 128.498296] niks3-server[1112]: 2026/09/23 04:43:26 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6554lb # [ 128.508462] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6555worker1 # [ 128.497138] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.575Z level=INFO msg="Upload complete. (177ms)"6556lb # [ 128.513505] niks3-server[1112]: 2026/09/23 04:43:26 INFO Completed upload id=516557worker1 # [ 128.500659] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53342 duration_s=0 paths=1 nar_bytes_in=4246558worker1 # [ 128.505332] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533426559worker1 # [ 128.512878] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)6560worker1 # [ 128.517645] nix-daemon[846]: reaped child process 1380, status = succeeded6561worker1 # [ 128.523412] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)6562client # [ 128.535562] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6563lb: must succeed: curl -sf localhost:9901/clusters6564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6565lb: must succeed: curl -sf localhost:9901/clusters6566lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6567worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6568worker1 # [ 130.748416] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1253 (nix-grpc-daemon) on client request.6569worker1 # [ 130.751316] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6570lb # [ 130.774868] niks3-server[1112]: 2026/09/23 04:43:28 INFO lead: released remote=192.168.1.3:498766571worker1 # [ 130.764113] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6572worker1 # [ 130.766179] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6573client # [ 130.777233] nix[1401]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6574worker1 # [ 130.767397] systemd[1]: nix-grpc-daemon.service: Consumed 382ms CPU time over 16.358s wall clock time, 35.6M memory peak, 15.4K incoming IP traffic, 27.4K outgoing IP traffic.6575worker2 # [ 130.745849] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6576client # [ 130.781961] nix[1401]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6577worker1 # [ 130.782071] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6578worker1 # [ 130.783584] systemd[1]: Starting Nix worker-protocol over gRPC...6579worker1 # [ 130.814349] nix-daemon[846]: reaped child process 1372, status = succeeded6580worker1 # [ 130.815666] nix-daemon[846]: reaped child process 1404, status = succeeded6581worker1 # [ 130.823090] nix-daemon[1406]: unexpected Nix daemon error: error: interrupted by the user6582worker1 # [ 130.829752] nix-daemon[846]: reaped child process 1406, status = failed with exit code 16583worker1 # [ 130.874795] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6584worker1 # [ 130.876755] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6585worker1 # [ 130.879877] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16586worker1 # [ 130.885057] nix-grpc-daemon[1442]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6587worker1 # [ 130.887189] nix-grpc-daemon[1442]: E0000 00:00:1790138608.962340 1442 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6588worker1 # [ 130.893968] nix-grpc-daemon[1442]: E0000 00:00:1790138608.967888 1442 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6589worker1 # [ 130.897845] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6590worker1 # [ 130.901137] systemd[1]: Started Nix worker-protocol over gRPC.6591lb # [ 130.915710] niks3-server[1112]: 2026/09/23 04:43:28 INFO lead: acquired remote=192.168.1.3:386746592worker1 # [ 130.901987] nix-grpc-daemon[1442]: E0000 00:00:1790138608.978214 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6593worker1 # [ 130.908123] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=scheduler_take_over6594worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.19 seconds)6595client: must succeed: sleep 5.06596worker1 # [ 130.915163] nix-grpc-daemon[1442]: E0000 00:00:1790138608.993411 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6597worker1 # [ 130.919139] nix-grpc-daemon[1442]: E0000 00:00:1790138608.995642 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6598worker1 # [ 130.929228] nix-grpc-daemon[1442]: E0000 00:00:1790138609.007529 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6599worker1 # [ 130.938614] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516600worker1 # [ 130.945187] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:533486601worker1 # [ 130.946889] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06602worker2 # [ 130.946918] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516603worker1 # [ 130.970347] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=oidc_jwks provider=mock keys=16604worker1 # [ 130.971673] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:533486605worker1 # [ 130.973212] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06606worker1 # [ 131.278615] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06607worker1 # [ 131.284092] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16608worker1 # [ 131.326938] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533806609worker1 # [ 131.337271] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6610worker1 # [ 131.350950] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6611client # [ 131.376884] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6612worker1 # [ 131.365561] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6613client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6614client: waiting for success: ! systemctl is-active mid6615worker1 # [ 136.184943] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:34Z level=info event=niks3_push_started pid=15276616worker1 # [ 136.217344] nix-daemon[846]: reaped child process 1502, status = succeeded6617worker1 # [ 136.296751] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.374Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6618worker1 # [ 136.381751] nix-daemon[846]: accepted connection from pid 1539, user nix-grpc-daemon (trusted)6619worker1 # [ 136.394464] nix-daemon[846]: reaped child process 1542, status = succeeded6620lb # [ 136.420369] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received uploads request method=POST path=/api/pending_closures6621worker1 # [ 136.412189] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.490Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6622worker1 # [ 136.413706] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.492Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6623lb # [ 136.464265] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6624lb # [ 136.474226] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6625lb # [ 136.480849] niks3-server[1112]: 2026/09/23 04:43:34 INFO Signed narinfos id=52 count=16626worker1 # [ 136.468255] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.546Z level=INFO msg="Uploading 1 narinfos"6627lb # [ 136.491859] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6628lb # [ 136.501735] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6629lb # [ 136.509623] niks3-server[1112]: 2026/09/23 04:43:34 INFO Completed upload id=526630worker1 # [ 136.493525] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.571Z level=INFO msg="Upload complete. (193ms)"6631worker1 # [ 136.500119] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16632lb # [ 136.522439] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6633worker1 # [ 136.514070] nix-daemon[846]: reaped child process 1500, status = succeeded6634client # [ 136.549623] systemd[1]: mid.service: Deactivated successfully.6635client # [ 136.554255] systemd[1]: mid.service: Consumed 142ms CPU time over 9.423s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic.6636lb # [ 137.012137] envoy[927]: conn peer=[2001:db8:1::3]:47004 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1036 tx=393 ms=162076637lb # [ 137.015843] envoy[927]: conn peer=[2001:db8:1::1]:46668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=1922 ms=92656638client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6639client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6640client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6641(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.16 seconds)6642subtest: interrupting the client stops the build on the worker6643client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6644client # [ 137.270133] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6645client # Running as unit: intr.service; invocation ID: 26cfbde22445449b8102d8e3078756736646client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6647worker2 # [ 137.363202] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448906648worker2 # [ 137.374638] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6649client # [ 137.416377] nix[1437]: don't know how to build these paths:6650client # [ 137.417237] nix[1437]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6651worker1 # [ 137.416947] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:35Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06652worker1 # [ 137.421980] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:35Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16653worker2 # [ 137.447623] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906654worker2 # [ 137.459315] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6655worker2 # [ 137.482066] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 2/5)6656worker2 # [ 137.493832] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 3/5)6657worker2 # [ 137.666122] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 4/5)6658worker2 # [ 138.007770] nix-grpc-daemon[1697]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6659worker2 # [ 138.011642] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6660worker2 # [ 138.028670] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448906661client # [ 138.071171] nix[1437]: copying 1 paths to 'lb:50051'...6662worker2 # [ 138.040572] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448906663client # [ 138.073800] nix[1437]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6664worker2 # [ 138.050664] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6665worker2 # [ 138.073491] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=info event=niks3_push_started pid=17906666worker2 # [ 138.175495] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.307Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6667worker2 # [ 138.262140] nix-daemon[869]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6668worker2 # [ 138.273714] nix-daemon[869]: reaped child process 1806, status = succeeded6669lb # [ 138.319378] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received uploads request method=POST path=/api/pending_closures6670worker2 # [ 138.294418] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6671worker2 # [ 138.295958] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.428Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6672lb # [ 138.360648] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6673lb # [ 138.368160] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6674worker2 # [ 138.340707] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.472Z level=INFO msg="Uploading 1 narinfos"6675lb # [ 138.375694] niks3-server[1112]: 2026/09/23 04:43:36 INFO Signed narinfos id=53 count=16676lb # [ 138.387222] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6677lb # [ 138.396606] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6678lb # [ 138.403246] niks3-server[1112]: 2026/09/23 04:43:36 INFO Completed upload id=536679lb # [ 138.405068] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6680worker2 # [ 138.368252] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.500Z level=INFO msg="Upload complete. (189ms)"6681worker2 # [ 138.372902] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=4246682worker2 # [ 138.379313] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906683worker2 # [ 138.392577] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6684worker2 # [ 138.403771] nix-daemon[869]: reaped child process 1788, status = succeeded6685worker2 # [ 138.408173] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6686client # [ 138.439813] nix[1437]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6687client: must succeed: systemctl kill -s INT intr6688client # [ 140.692979] systemd[1]: intr.service: Sent signal SIGINT to main process 1437 (nix) on client request.6689worker2 # [ 140.664345] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:38Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6690client # [ 140.698893] nix[1437]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6691client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6692client # [ 140.705558] nix[1437]: error: interrupted by the user6693client # [ 140.732126] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6694client # [ 140.734891] systemd[1]: intr.service: Failed with result 'exit-code'.6695client # [ 140.738400] systemd[1]: intr.service: Consumed 117ms CPU time over 3.464s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6696worker2 # [ 140.832161] nix-daemon[869]: reaped child process 1811, status = succeeded6697worker2 # [ 140.833381] nix-daemon[1814]: unexpected Nix daemon error: error: interrupted by the user6698worker2 # [ 140.837883] nix-daemon[869]: reaped child process 1814, status = failed with exit code 16699(finished: subtest: interrupting the client stops the build on the worker, in 4.64 seconds)6700subtest: a deploy mid-build drains: the build finishes, the new generation takes over6701client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6702client # [ 141.936537] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6703client # Running as unit: sw.service; invocation ID: f194202b1b6b47beb3caf1134456e7256704client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6705client # [ 142.073164] nix[1458]: don't know how to build these paths:6706client # [ 142.074035] nix[1458]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6707worker2 # [ 142.040382] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448906708worker1 # [ 142.070886] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06709worker1 # [ 142.074948] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16710worker1 # [ 142.114210] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533806711worker1 # [ 142.125093] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6712worker1 # [ 142.213636] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5)6713worker1 # [ 142.357468] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 3/5)6714worker1 # [ 142.471480] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 4/5)6715worker1 # [ 142.663740] nix-grpc-daemon[1442]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6716worker1 # [ 142.667262] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6717worker1 # [ 142.684552] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533806718client # [ 142.707168] nix[1458]: copying 1 paths to 'lb:50051'...6719client # [ 142.710812] nix[1458]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6720worker1 # [ 142.698433] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533806721worker1 # [ 142.709792] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6722worker1 # [ 142.808464] nix-daemon[846]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)6723worker1 # [ 142.820993] nix-daemon[846]: reaped child process 1586, status = succeeded6724lb # [ 142.845447] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received uploads request method=POST path=/api/pending_closures6725worker1 # [ 142.841323] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6726worker1 # [ 142.843667] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.921Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6727lb # [ 142.887699] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6728lb # [ 142.898680] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6729lb # [ 142.908657] niks3-server[1112]: 2026/09/23 04:43:40 INFO Signed narinfos id=54 count=16730worker1 # [ 142.893571] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.970Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 142.919541] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6732lb # [ 142.928395] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6733lb # [ 142.931473] niks3-server[1112]: 2026/09/23 04:43:40 INFO Completed upload id=546734lb # [ 142.934914] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6735worker1 # [ 142.917890] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.993Z level=INFO msg="Upload complete. (194ms)"6736worker1 # [ 142.922572] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=0 paths=1 nar_bytes_in=4166737worker1 # [ 142.928281] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533806738worker1 # [ 142.938058] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6739worker1 # [ 142.948830] nix-daemon[846]: reaped child process 1581, status = succeeded6740worker1 # [ 142.953125] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6741client # [ 142.965978] nix[1458]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6742worker1: must succeed: systemctl reload nix-grpc-daemon.service6743worker1 # [ 144.338318] systemd[1]: Reloading Nix worker-protocol over gRPC...6744worker1 # [ 144.366529] systemd[1]: Reloaded Nix worker-protocol over gRPC.6745worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6746worker1: must succeed: pgrep -f 'read -t [9]0 x'6747worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6748client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26749worker2 # [ 144.534534] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448906750client # don't know how to build these paths:6751client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6752worker1 # [ 144.565302] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533806753worker1 # [ 144.579907] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06754worker1 # [ 144.581769] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16755worker2 # [ 144.605071] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906756worker2 # [ 144.617349] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448906757client # copying 1 paths to 'lb:50051'...6758worker2 # [ 144.629626] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448906759client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6760worker2 # [ 144.650267] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6761worker2 # [ 144.754942] nix-daemon[869]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6762worker2 # [ 144.767742] nix-daemon[869]: reaped child process 1868, status = succeeded6763lb # [ 144.810529] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received uploads request method=POST path=/api/pending_closures6764worker2 # [ 144.783861] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6765worker2 # [ 144.786296] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.917Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6766lb # [ 144.841555] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6767worker2 # [ 144.817356] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.949Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 144.852392] niks3-server[1112]: 2026/09/23 04:43:42 INFO Signed narinfos id=55 count=16769lb # [ 144.857681] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6770lb # [ 144.863119] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6771lb # [ 144.876425] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6772lb # [ 144.883522] niks3-server[1112]: 2026/09/23 04:43:42 INFO Completed upload id=556773lb # [ 144.885925] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6774worker2 # [ 144.850612] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.982Z level=INFO msg="Upload complete. (188ms)"6775worker2 # [ 144.855667] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=5286776worker2 # [ 144.862491] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906777worker2 # [ 144.873109] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6778worker1 # [ 144.896933] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=info event=draining6779worker2 # [ 144.881630] nix-daemon[869]: reaped child process 1863, status = succeeded6780worker2 # [ 144.885709] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6781client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6782client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6783client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6784client # farm-a> LOG-farm-a-during-drain6785worker2 # [ 146.321705] nix-daemon[869]: reaped child process 1872, status = succeeded6786worker2 # [ 146.406091] nix-daemon[869]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6787worker2 # [ 146.418542] nix-daemon[869]: reaped child process 1891, status = succeeded6788lb # [ 146.461527] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received uploads request method=POST path=/api/pending_closures6789worker2 # [ 146.435141] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.566Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6790worker2 # [ 146.436677] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.568Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6791lb # [ 146.507128] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6792lb # [ 146.513907] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6793worker2 # [ 146.488305] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.620Z level=INFO msg="Uploading 1 narinfos"6794lb # [ 146.523122] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6795lb # [ 146.525941] niks3-server[1112]: 2026/09/23 04:43:44 INFO Signed narinfos id=56 count=16796lb # [ 146.532942] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6797lb # [ 146.541452] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6798lb # [ 146.545943] niks3-server[1112]: 2026/09/23 04:43:44 INFO Completed upload id=566799lb # [ 146.547332] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6800worker2 # [ 146.511240] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.643Z level=INFO msg="Upload complete. (215ms)"6801worker2 # [ 146.517722] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16802worker1 # [ 146.540856] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:44Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06803worker1 # [ 146.543207] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:44Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16804worker2 # [ 146.533115] nix-daemon[869]: reaped child process 1870, status = succeeded6805worker2 # [ 146.571751] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906806worker2 # [ 146.583751] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448906807worker2 # [ 146.597267] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.729Z level=INFO msg="All 1 paths already cached"6808client # copying 1 paths to 'lb:50051'...6809worker2 # [ 146.605920] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448906810client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6811worker2 # [ 146.624163] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6812worker2 # [ 146.725524] nix-daemon[869]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6813worker2 # [ 146.739803] nix-daemon[869]: reaped child process 1898, status = succeeded6814lb # [ 146.780491] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received uploads request method=POST path=/api/pending_closures6815worker2 # [ 146.753238] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.885Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6816worker2 # [ 146.754701] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.886Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6817lb # [ 146.814842] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6818lb # [ 146.818951] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6819lb # [ 146.824139] niks3-server[1112]: 2026/09/23 04:43:44 INFO Signed narinfos id=57 count=16820worker2 # [ 146.792616] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.924Z level=INFO msg="Uploading 1 narinfos"6821lb # [ 146.836142] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6822lb # [ 146.843428] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6823lb # [ 146.854717] niks3-server[1112]: 2026/09/23 04:43:44 INFO Completed upload id=576824worker2 # [ 146.819933] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.951Z level=INFO msg="Upload complete. (182ms)"6825lb # [ 146.856420] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6826worker2 # [ 146.824521] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=6406827worker2 # [ 146.831400] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906828worker2 # [ 146.841313] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6829worker2 # [ 146.854525] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6830client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6831worker2 # [ 146.856696] nix-daemon[869]: reaped child process 1893, status = succeeded6832client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6833lb # [ 147.013300] envoy[927]: conn peer=[2001:db8:1::1]:37824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=33276834client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6835client # farm-b> LOG-farm-b-during-drain6836worker2 # [ 147.670059] nix-daemon[869]: reaped child process 1902, status = succeeded6837worker2 # [ 147.745638] nix-daemon[869]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6838worker2 # [ 147.758411] nix-daemon[869]: reaped child process 1921, status = succeeded6839lb # [ 147.801333] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received uploads request method=POST path=/api/pending_closures6840worker2 # [ 147.775792] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.907Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6841worker2 # [ 147.777341] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.909Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6842lb # [ 147.842986] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6843lb # [ 147.848100] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6844worker2 # [ 147.819361] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.951Z level=INFO msg="Uploading 1 narinfos"6845lb # [ 147.854565] niks3-server[1112]: 2026/09/23 04:43:45 INFO Signed narinfos id=58 count=16846lb # [ 147.863443] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6847lb # [ 147.872974] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6848lb # [ 147.879242] niks3-server[1112]: 2026/09/23 04:43:45 INFO Completed upload id=586849worker2 # [ 147.843959] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.975Z level=INFO msg="Upload complete. (182ms)"6850lb # [ 147.884791] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6851lb # [ 147.887902] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6852worker2 # [ 147.849681] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16853worker1 # [ 147.875282] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:45Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06854worker1 # [ 147.877225] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:45Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16855worker2 # [ 147.865339] nix-daemon[869]: reaped child process 1900, status = succeeded6856worker2 # [ 147.900670] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906857worker2 # [ 147.912934] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448906858worker2 # [ 147.925504] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.057Z level=INFO msg="All 2 paths already cached"6859client # copying 1 paths to 'lb:50051'...6860worker2 # [ 147.934436] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448906861client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6862worker2 # [ 147.951802] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6863worker2 # [ 148.052886] nix-daemon[869]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6864worker2 # [ 148.066864] nix-daemon[869]: reaped child process 1928, status = succeeded6865lb # [ 148.107603] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received uploads request method=POST path=/api/pending_closures6866worker2 # [ 148.081026] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.213Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6867worker2 # [ 148.082518] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.214Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6868lb # [ 148.137908] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6869worker2 # [ 148.108474] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.240Z level=INFO msg="Uploading 1 narinfos"6870lb # [ 148.143338] niks3-server[1112]: 2026/09/23 04:43:46 INFO Signed narinfos id=59 count=16871lb # [ 148.153500] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6872lb # [ 148.159092] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6873lb # [ 148.169714] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6874worker2 # [ 148.139937] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.271Z level=INFO msg="Upload complete. (173ms)"6875lb # [ 148.174732] niks3-server[1112]: 2026/09/23 04:43:46 INFO Completed upload id=596876worker2 # [ 148.142510] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=7686877lb # [ 148.181705] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6878worker2 # [ 148.147629] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448906879worker2 # [ 148.156218] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6880worker2 # [ 148.162456] nix-daemon[869]: reaped child process 1923, status = succeeded6881worker2 # [ 148.167612] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6882client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6883client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6884client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6885client # farm-top> LOG-farm-top-during-drain6886worker2 # [ 149.221141] nix-daemon[869]: reaped child process 1932, status = succeeded6887worker2 # [ 149.279157] nix-daemon[869]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)6888worker2 # [ 149.293045] nix-daemon[869]: reaped child process 1951, status = succeeded6889lb # [ 149.334374] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received uploads request method=POST path=/api/pending_closures6890worker2 # [ 149.308166] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.439Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6891worker2 # [ 149.309756] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.441Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6892lb # [ 149.368261] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6893lb # [ 149.380491] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6894lb # [ 149.383463] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6895worker2 # [ 149.353700] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.485Z level=INFO msg="Uploading 1 narinfos"6896lb # [ 149.387986] niks3-server[1112]: 2026/09/23 04:43:47 INFO Signed narinfos id=60 count=16897lb # [ 149.397152] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6898lb # [ 149.400584] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6899lb # [ 149.403854] niks3-server[1112]: 2026/09/23 04:43:47 INFO Completed upload id=606900worker2 # [ 149.369150] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.501Z level=INFO msg="Upload complete. (194ms)"6901lb # [ 149.406418] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6902worker2 # [ 149.374762] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16903worker2 # [ 149.386937] nix-daemon[869]: reaped child process 1930, status = succeeded6904client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.02 seconds)6905client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6906client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6907worker1: must succeed: pkill -f 'read -t [9]0 x'6908worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.03 seconds)6909client: waiting for success: ! systemctl is-active sw6910worker1 # [ 149.818259] nix-daemon[846]: reaped child process 1590, status = succeeded6911worker1 # [ 149.868669] nix-daemon[846]: accepted connection from pid 1638, user nix-grpc-daemon (trusted)6912worker1 # [ 149.882201] nix-daemon[846]: reaped child process 1641, status = succeeded6913lb # [ 149.902651] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received uploads request method=POST path=/api/pending_closures6914worker1 # [ 149.890945] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6915worker1 # [ 149.892407] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.970Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6916lb # [ 149.923272] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6917lb # [ 149.926485] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6918lb # [ 149.927894] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6919lb # [ 149.930417] niks3-server[1112]: 2026/09/23 04:43:47 INFO Signed narinfos id=61 count=16920worker1 # [ 149.915964] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.994Z level=INFO msg="Uploading 1 narinfos"6921lb # [ 149.933907] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6922lb # [ 149.939415] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6923lb # [ 149.942334] niks3-server[1112]: 2026/09/23 04:43:47 INFO Completed upload id=616924worker1 # [ 149.926283] nix-grpc-daemon[1527]: time=2026-09-23T04:43:48.004Z level=INFO msg="Upload complete. (160ms)"6925lb # [ 149.945559] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6926worker1 # [ 149.928847] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16927worker1 # [ 149.936718] nix-daemon[846]: reaped child process 1588, status = succeeded6928client # [ 149.973766] systemd[1]: sw.service: Deactivated successfully.6929client # [ 149.978450] systemd[1]: sw.service: Consumed 130ms CPU time over 8.040s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.6930client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6931client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6932client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6933worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6934worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6935lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6936lb: must succeed: curl -sf localhost:9901/clusters6937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6938worker1 # [ 150.899092] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:48Z level=info event=signal_exit6939lb # [ 150.920985] niks3-server[1112]: 2026/09/23 04:43:48 INFO lead: released remote=192.168.1.3:386746940lb: must succeed: curl -sf localhost:9901/clusters6941lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6942lb # [ 151.875800] niks3-server[1112]: 2026/09/23 04:43:49 INFO lead: acquired remote=192.168.1.4:424966943worker2 # [ 151.842125] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:49Z level=info event=scheduler_take_over6944worker2 # [ 151.887310] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_restarting addr=lb:500516945worker1 # [ 151.907614] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=scheduler_restarting6946worker1 # [ 151.914212] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:533486947worker1 # [ 151.918257] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_gone addr=192.168.1.4:500516948worker1 # [ 151.922312] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=scheduler_restarting addr=lb:500516949worker1 # [ 151.925355] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:533486950worker1 # [ 151.928891] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_gone addr=192.168.1.3:500516951worker1 # [ 151.945770] nix-daemon[846]: reaped child process 1498, status = succeeded6952worker2 # [ 152.088385] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:500516953worker2 # [ 152.092613] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6954worker1 # [ 152.122934] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6955worker1 # [ 152.182295] nix-grpc-daemon[1442]: E0000 00:00:1790138630.260198 1489 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6956worker1 # [ 152.187150] nix-grpc-daemon[1442]: E0000 00:00:1790138630.260424 1489 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6957worker2 # [ 152.293742] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:500516958worker2 # [ 152.297738] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6959worker2 # [ 152.498998] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:500516960worker2 # [ 152.502907] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6961worker2 # [ 152.703991] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:500516962worker2 # [ 152.709291] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6963lb: must succeed: curl -sf localhost:9901/clusters6964lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6965worker1 # [ 152.917451] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6966worker1 # [ 152.922414] systemd[1]: nix-grpc-daemon.service: Consumed 586ms CPU time over 22.156s wall clock time, 36M memory peak, 25K incoming IP traffic, 32.6K outgoing IP traffic.6967worker2 # [ 152.911459] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=scheduler_connected addr=lb:500516968worker2 # [ 152.925139] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=oidc_jwks provider=mock keys=16969worker2 # [ 152.929363] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536566970worker2 # [ 152.933356] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06971lb: must succeed: curl -sf localhost:9901/clusters6972lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6973worker1 # [ 153.913162] systemd[1]: Starting Nix worker-protocol over gRPC...6974worker1 # [ 154.017344] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6975worker1 # [ 154.019613] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6976worker1 # [ 154.022792] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16977worker1 # [ 154.027057] nix-grpc-daemon[1653]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6978worker1 # [ 154.029158] nix-grpc-daemon[1653]: E0000 00:00:1790138632.105159 1653 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6979worker1 # [ 154.034379] nix-grpc-daemon[1653]: E0000 00:00:1790138632.110570 1653 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6980worker1 # [ 154.038105] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6981worker1 # [ 154.039992] systemd[1]: Started Nix worker-protocol over gRPC.6982worker1 # [ 154.046065] nix-grpc-daemon[1653]: E0000 00:00:1790138632.123773 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6983worker1 # [ 154.059440] nix-grpc-daemon[1653]: E0000 00:00:1790138632.137697 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6984worker1 # [ 154.064052] nix-grpc-daemon[1653]: E0000 00:00:1790138632.139960 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6985worker1 # [ 154.073427] nix-grpc-daemon[1653]: E0000 00:00:1790138632.151683 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6986worker1 # [ 154.082309] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516987worker2 # [ 154.065718] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:536566988worker2 # [ 154.070191] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06989lb: must succeed: curl -sf localhost:9901/clusters6990lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6991lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.21 seconds)6992lb: must succeed: curl -sf localhost:9901/clusters6993lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6994worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6995worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6996(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.11 seconds)6997subtest: queue gauges show a burst while every build blocks6998lb: must succeed: curl -sf localhost:9901/clusters6999lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7000lb: must succeed: curl -sf localhost:9901/clusters7001lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7002worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7003worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7004client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'7005client # [ 155.143071] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".7006client # Running as unit: burst.service; invocation ID: 9afc2ac7392b4900bba2e7ee1025d0dd7007client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)7008lb: must succeed: curl -sf localhost:9901/clusters7009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7010lb: must succeed: curl -sf localhost:9901/clusters7011lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7012worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7013worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7014worker1 # [ 155.267567] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355247015worker1 # [ 155.279067] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7016client # [ 155.304359] nix[1514]: don't know how to build these paths:7017client # [ 155.305286] nix[1514]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7018client # [ 155.306308] nix[1514]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7019client # [ 155.307463] nix[1514]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7020client # [ 155.308640] nix[1514]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7021worker2 # [ 155.287428] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=07022worker2 # [ 155.291440] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=17023worker2 # [ 155.295808] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=07024worker2 # [ 155.299687] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=17025worker2 # [ 155.304150] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07026worker2 # [ 155.307927] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07027worker1 # [ 155.349348] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247028worker2 # [ 155.330887] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907029worker1 # [ 155.360238] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)7030worker2 # [ 155.343220] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907031client # [ 155.385171] nix[1514]: copying 1 paths to 'lb:50051'...7032worker2 # [ 155.354394] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448907033client # [ 155.387194] nix[1514]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7034worker2 # [ 155.366740] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7035worker1 # [ 155.467926] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)7036worker2 # [ 155.461937] nix-daemon[869]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)7037worker2 # [ 155.474647] nix-daemon[869]: reaped child process 1969, status = succeeded7038lb # [ 155.517887] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received uploads request method=POST path=/api/pending_closures7039worker2 # [ 155.493190] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.624Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7040worker2 # [ 155.494645] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.624Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7041lb # [ 155.547707] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7042lb # [ 155.558721] niks3-server[1112]: 2026/09/23 04:43:53 INFO Signed narinfos id=62 count=17043worker2 # [ 155.526824] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.658Z level=INFO msg="Uploading 1 narinfos"7044lb # [ 155.576712] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7045lb # [ 155.585683] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7046lb # [ 155.587902] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7047lb # [ 155.592254] niks3-server[1112]: 2026/09/23 04:43:53 INFO Completed upload id=627048worker2 # [ 155.557647] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.689Z level=INFO msg="Upload complete. (180ms)"7049lb # [ 155.594385] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7050worker2 # [ 155.562578] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=4167051worker2 # [ 155.568552] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907052worker2 # [ 155.579512] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7053worker2 # [ 155.589583] nix-daemon[869]: reaped child process 1964, status = succeeded7054worker2 # [ 155.593787] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7055client # [ 155.625271] nix[1514]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7056worker1 # [ 155.663128] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 3/5)7057worker1 # [ 155.770256] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 4/5)7058worker1 # [ 156.024344] nix-grpc-daemon[1653]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7059worker1 # [ 156.027364] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7060worker1 # [ 156.045711] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247061client # [ 156.071190] nix[1514]: copying 1 paths to 'lb:50051'...7062worker1 # [ 156.059290] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247063client # [ 156.073557] nix[1514]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7064worker1 # [ 156.068784] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7065worker1 # [ 156.092092] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=info event=niks3_push_started pid=17147066worker1 # [ 156.194146] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.272Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7067lb: must succeed: curl -sf localhost:9901/clusters7068worker1 # [ 156.281815] nix-daemon[846]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)7069worker1 # [ 156.294307] nix-daemon[846]: reaped child process 1730, status = succeeded7070lb # [ 156.318150] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received uploads request method=POST path=/api/pending_closures7071worker1 # [ 156.312748] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.391Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7072worker1 # [ 156.315326] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.392Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7073lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7074lb: must succeed: curl -sf localhost:9901/clusters7075lb # [ 156.358087] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7076lb # [ 156.360758] niks3-server[1112]: 2026/09/23 04:43:54 INFO Signed narinfos id=63 count=17077worker1 # [ 156.346916] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.424Z level=INFO msg="Uploading 1 narinfos"7078lb # [ 156.366274] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7079lb # [ 156.374100] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7080lb # [ 156.378483] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7081lb # [ 156.384187] niks3-server[1112]: 2026/09/23 04:43:54 INFO Completed upload id=637082worker1 # [ 156.368381] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.446Z level=INFO msg="Upload complete. (170ms)"7083worker1 # [ 156.370527] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=4167084lb # [ 156.387471] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7085worker1 # [ 156.373832] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7087worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7088worker1 # [ 156.380351] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7089worker1 # [ 156.382904] nix-daemon[846]: reaped child process 1712, status = succeeded7090worker1 # [ 156.387524] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7091client # [ 156.400861] nix[1514]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7092worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7093lb # [ 157.013502] envoy[927]: conn peer=[2001:db8:1::1]:46424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4384 ms=48577094lb # [ 157.017423] envoy[927]: conn peer=[2001:db8:1::1]:37832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1557 ms=78837095lb # [ 157.020954] envoy[927]: conn peer=[2001:db8:1::3]:55412 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1018 tx=442 ms=211797096worker1 # [ 157.601305] nix-daemon[846]: reaped child process 1734, status = succeeded7097worker2 # [ 157.610951] nix-daemon[869]: reaped child process 1973, status = succeeded7098worker1 # [ 157.676354] nix-daemon[846]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7099worker1 # [ 157.688621] nix-daemon[846]: reaped child process 1762, status = succeeded7100lb # [ 157.713389] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures7101worker2 # [ 157.685304] nix-daemon[869]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)7102worker1 # [ 157.706445] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.784Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7103worker1 # [ 157.707873] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.786Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7104worker2 # [ 157.697265] nix-daemon[869]: reaped child process 2004, status = succeeded7105lb # [ 157.738769] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures7106worker2 # [ 157.714336] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.846Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7107worker2 # [ 157.715971] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.847Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7108lb # [ 157.772329] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7109lb # [ 157.784070] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7110lb # [ 157.786661] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7111lb # [ 157.789707] niks3-server[1112]: 2026/09/23 04:43:55 INFO Signed narinfos id=64 count=17112worker1 # [ 157.775480] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.853Z level=INFO msg="Uploading 1 narinfos"7113lb # [ 157.794728] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7114lb # [ 157.797226] niks3-server[1112]: 2026/09/23 04:43:55 INFO Signed narinfos id=65 count=17115worker2 # [ 157.761931] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.893Z level=INFO msg="Uploading 1 narinfos"7116lb # [ 157.803177] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7117lb # [ 157.806253] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7118lb # [ 157.808828] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7119lb # [ 157.814039] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7120lb # [ 157.817682] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7121lb # [ 157.821856] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7122lb # [ 157.825836] niks3-server[1112]: 2026/09/23 04:43:55 INFO Completed upload id=657123worker2 # [ 157.791161] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.923Z level=INFO msg="Upload complete. (188ms)"7124lb # [ 157.827847] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7125lb # [ 157.831817] niks3-server[1112]: 2026/09/23 04:43:55 INFO Completed upload id=647126lb # [ 157.833776] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7127worker1 # [ 157.815959] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.894Z level=INFO msg="Upload complete. (222ms)"7128worker2 # [ 157.797346] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17129worker2 # [ 157.804912] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=17130worker1 # [ 157.823312] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17131worker2 # [ 157.809182] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=17132worker2 # [ 157.817212] nix-daemon[869]: reaped child process 1971, status = succeeded7133worker1 # [ 157.836795] nix-daemon[846]: reaped child process 1732, status = succeeded7134worker2 # [ 157.847681] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907135worker1 # [ 157.866703] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247136worker2 # [ 157.860233] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907137worker1 # [ 157.878948] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247138client # [ 157.902807] nix[1514]: copying 1 paths to 'lb:50051'...7139worker1 # [ 157.890106] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247140client # [ 157.904725] nix[1514]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7141client # [ 157.909207] nix[1514]: copying 1 paths to 'lb:50051'...7142worker2 # [ 157.879128] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448907143client # [ 157.912275] nix[1514]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7144worker1 # [ 157.905212] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7145worker2 # [ 157.888844] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7146worker2 # [ 157.984068] nix-daemon[869]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)7147worker1 # [ 158.004753] nix-daemon[846]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)7148worker2 # [ 157.996398] nix-daemon[869]: reaped child process 2011, status = succeeded7149worker1 # [ 158.017457] nix-daemon[846]: reaped child process 1769, status = succeeded7150lb # [ 158.039343] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures7151lb # [ 158.043939] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures7152worker1 # [ 158.037672] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.115Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7153worker2 # [ 158.019801] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7154worker1 # [ 158.039268] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.117Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7155worker2 # [ 158.021328] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.153Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7156lb # [ 158.085622] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7157lb # [ 158.087722] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7158worker1 # [ 158.076395] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.154Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 158.092241] niks3-server[1112]: 2026/09/23 04:43:56 INFO Signed narinfos id=66 count=17160lb # [ 158.102856] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7161lb # [ 158.108083] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7162lb # [ 158.111903] niks3-server[1112]: 2026/09/23 04:43:56 INFO Completed upload id=667163worker1 # [ 158.096062] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.174Z level=INFO msg="Upload complete. (178ms)"7164lb # [ 158.114573] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7165lb # [ 158.117176] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7166worker1 # [ 158.100647] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=4167167lb # [ 158.120707] niks3-server[1112]: 2026/09/23 04:43:56 INFO Signed narinfos id=67 count=17168worker2 # [ 158.086099] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.218Z level=INFO msg="Uploading 1 narinfos"7169lb # [ 158.123749] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7170worker1 # [ 158.106143] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247171lb # [ 158.127820] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7172lb # [ 158.131740] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7173worker1 # [ 158.115580] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7174lb # [ 158.134284] niks3-server[1112]: 2026/09/23 04:43:56 INFO Completed upload id=677175worker1 # [ 158.121213] nix-daemon[846]: reaped child process 1764, status = succeeded7176lb # [ 158.137282] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo7177worker2 # [ 158.100947] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.232Z level=INFO msg="Upload complete. (199ms)"7178worker1 # [ 158.125327] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7179client # [ 158.137983] nix[1514]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7180worker2 # [ 158.105826] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=4167181worker2 # [ 158.112273] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907182worker2 # [ 158.125109] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7183worker2 # [ 158.132858] nix-daemon[869]: reaped child process 2006, status = succeeded7184client # [ 158.172107] nix[1514]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7185worker2 # [ 158.141820] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7186worker1 # [ 159.961385] nix-daemon[846]: reaped child process 1773, status = succeeded7187worker2 # [ 159.984115] nix-daemon[869]: reaped child process 2015, status = succeeded7188worker1 # [ 160.044605] nix-daemon[846]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)7189worker1 # [ 160.057233] nix-daemon[846]: reaped child process 1801, status = succeeded7190lb # [ 160.079965] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures7191worker1 # [ 160.073116] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7192worker1 # [ 160.074714] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.153Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7193worker2 # [ 160.058974] nix-daemon[869]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)7194worker2 # [ 160.073092] nix-daemon[869]: reaped child process 2043, status = succeeded7195lb # [ 160.113579] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures7196worker2 # [ 160.086275] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7197worker2 # [ 160.087764] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.219Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7198lb # [ 160.126540] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7199lb # [ 160.131665] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7200lb # [ 160.148885] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7201worker2 # [ 160.117611] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.249Z level=INFO msg="Uploading 1 narinfos"7202lb # [ 160.152355] niks3-server[1112]: 2026/09/23 04:43:58 INFO Signed narinfos id=69 count=17203lb # [ 160.155273] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7204lb # [ 160.157206] niks3-server[1112]: 2026/09/23 04:43:58 INFO Signed narinfos id=68 count=17205lb # [ 160.159707] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7206worker1 # [ 160.142926] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.220Z level=INFO msg="Uploading 1 narinfos"7207lb # [ 160.161567] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7208lb # [ 160.167248] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7209lb # [ 160.173066] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7210lb # [ 160.174760] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7211lb # [ 160.178748] niks3-server[1112]: 2026/09/23 04:43:58 INFO Completed upload id=697212worker2 # [ 160.143552] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.275Z level=INFO msg="Upload complete. (191ms)"7213lb # [ 160.183077] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7214lb # [ 160.185890] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7215worker2 # [ 160.148489] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17216lb # [ 160.189686] niks3-server[1112]: 2026/09/23 04:43:58 INFO Completed upload id=687217worker1 # [ 160.173556] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.251Z level=INFO msg="Upload complete. (240ms)"7218lb # [ 160.192604] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7219worker2 # [ 160.158995] nix-daemon[869]: reaped child process 2013, status = succeeded7220worker1 # [ 160.179562] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17221worker1 # [ 160.194635] nix-daemon[846]: reaped child process 1771, status = succeeded7222client # [ 160.224770] systemd[1]: burst.service: Deactivated successfully.7223client # [ 160.228831] systemd[1]: burst.service: Consumed 153ms CPU time over 5.084s wall clock time, 55.7M memory peak, 14K incoming IP traffic, 18.2K outgoing IP traffic.7224client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7225client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)7226lb: must succeed: curl -sf localhost:9901/clusters7227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7228lb: must succeed: curl -sf localhost:9901/clusters7229lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7230worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7231worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7232(finished: subtest: queue gauges show a burst while every build blocks, in 6.20 seconds)7233subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7234client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17235worker2 # [ 161.315291] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448907236worker1 # [ 161.344584] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247237worker2 # [ 161.373158] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07238worker2 # [ 161.377200] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17239worker2 # [ 161.381691] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07240worker2 # [ 161.386490] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07241worker1 # [ 161.450675] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247242worker1 # [ 161.464573] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247243worker1 # [ 161.476406] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247244worker1 # [ 161.484961] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7245worker1 # [ 161.590615] nix-daemon[846]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7246worker1 # [ 161.603183] nix-daemon[846]: reaped child process 1812, status = succeeded7247lb # [ 161.627450] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received uploads request method=POST path=/api/pending_closures7248worker1 # [ 161.620743] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.698Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7249worker1 # [ 161.622784] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.700Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7250lb # [ 161.661392] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7251lb # [ 161.665605] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7252lb # [ 161.671756] niks3-server[1112]: 2026/09/23 04:43:59 INFO Signed narinfos id=70 count=17253worker1 # [ 161.659026] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.737Z level=INFO msg="Uploading 1 narinfos"7254lb # [ 161.681404] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7255lb # [ 161.692777] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7256lb # [ 161.699453] niks3-server[1112]: 2026/09/23 04:43:59 INFO Completed upload id=707257lb # [ 161.702259] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7258worker1 # [ 161.685147] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.763Z level=INFO msg="Upload complete. (186ms)"7259worker1 # [ 161.689918] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=5047260worker1 # [ 161.695710] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247261worker1 # [ 161.706570] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7262worker1 # [ 161.715400] nix-daemon[846]: reaped child process 1807, status = succeeded7263worker1 # [ 161.720421] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7264worker1 # [ 162.534422] nix-daemon[846]: reaped child process 1816, status = succeeded7265worker1 # [ 162.609200] nix-daemon[846]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)7266worker1 # [ 162.621861] nix-daemon[846]: reaped child process 1835, status = succeeded7267lb # [ 162.645868] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures7268worker1 # [ 162.641271] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.719Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7269worker1 # [ 162.642804] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.721Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7270lb # [ 162.688704] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7271lb # [ 162.696890] niks3-server[1112]: 2026/09/23 04:44:00 INFO Signed narinfos id=71 count=17272worker1 # [ 162.683893] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.762Z level=INFO msg="Uploading 1 narinfos"7273lb # [ 162.707506] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7274lb # [ 162.712065] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7275lb # [ 162.717750] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7276lb # [ 162.727662] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7277lb # [ 162.730885] niks3-server[1112]: 2026/09/23 04:44:00 INFO Completed upload id=717278worker1 # [ 162.714898] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.793Z level=INFO msg="Upload complete. (188ms)"7279lb # [ 162.733636] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7280worker1 # [ 162.720899] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17281worker2 # [ 162.703702] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:00Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17282worker1 # [ 162.729953] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247283worker2 # [ 162.713720] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:00Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07284worker1 # [ 162.740139] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247285worker1 # [ 162.748078] nix-daemon[846]: reaped child process 1814, status = succeeded7286worker1 # [ 162.749497] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247287worker1 # [ 162.753987] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7288worker1 # [ 162.836593] nix-daemon[846]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)7289worker1 # [ 162.850420] nix-daemon[846]: reaped child process 1842, status = succeeded7290lb # [ 162.871875] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures7291worker1 # [ 162.865361] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7292worker1 # [ 162.867372] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.945Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7293lb # [ 162.903313] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7294worker1 # [ 162.891914] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.970Z level=INFO msg="Uploading 1 narinfos"7295lb # [ 162.909177] niks3-server[1112]: 2026/09/23 04:44:00 INFO Signed narinfos id=72 count=17296lb # [ 162.919568] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7297lb # [ 162.929677] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7298lb # [ 162.932753] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7299lb # [ 162.942088] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7300lb # [ 162.945827] niks3-server[1112]: 2026/09/23 04:44:00 INFO Completed upload id=727301worker1 # [ 162.929976] nix-grpc-daemon[1714]: time=2026-09-23T04:44:01.008Z level=INFO msg="Upload complete. (171ms)"7302worker1 # [ 162.934320] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=5047303worker1 # [ 162.940326] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247304worker1 # [ 162.949231] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7305worker1 # [ 162.963213] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7306worker1 # [ 162.965415] nix-daemon[846]: reaped child process 1837, status = succeeded7307worker1 # [ 164.201585] nix-daemon[846]: reaped child process 1846, status = succeeded7308worker1 # [ 164.266522] nix-daemon[846]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)7309worker1 # [ 164.278655] nix-daemon[846]: reaped child process 1865, status = succeeded7310lb # [ 164.303382] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received uploads request method=POST path=/api/pending_closures7311worker1 # [ 164.295782] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.373Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7312worker1 # [ 164.297236] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.375Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7313lb # [ 164.348516] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7314lb # [ 164.352672] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7315lb # [ 164.358543] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7316worker1 # [ 164.346955] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.425Z level=INFO msg="Uploading 1 narinfos"7317lb # [ 164.362908] niks3-server[1112]: 2026/09/23 04:44:02 INFO Signed narinfos id=73 count=17318lb # [ 164.373065] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7319lb # [ 164.383492] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7320lb # [ 164.390209] niks3-server[1112]: 2026/09/23 04:44:02 INFO Completed upload id=737321worker1 # [ 164.374205] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.452Z level=INFO msg="Upload complete. (202ms)"7322lb # [ 164.392214] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7323worker2 # [ 164.362696] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:02Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17324worker1 # [ 164.380794] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17325worker2 # [ 164.372446] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:02Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07326worker1 # [ 164.395484] nix-daemon[846]: reaped child process 1844, status = succeeded7327worker1 # [ 164.432162] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247328worker1 # [ 164.443932] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247329worker1 # [ 164.455133] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247330worker1 # [ 164.464334] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7331worker1 # [ 164.562376] nix-daemon[846]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)7332worker1 # [ 164.577205] nix-daemon[846]: reaped child process 1872, status = succeeded7333lb # [ 164.598660] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received uploads request method=POST path=/api/pending_closures7334worker1 # [ 164.590319] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7335worker1 # [ 164.591780] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.670Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7336lb # [ 164.631418] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7337lb # [ 164.638909] niks3-server[1112]: 2026/09/23 04:44:02 INFO Signed narinfos id=74 count=17338worker1 # [ 164.626154] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.704Z level=INFO msg="Uploading 1 narinfos"7339lb # [ 164.643910] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7340lb # [ 164.648590] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7341lb # [ 164.663817] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7342worker1 # [ 164.653417] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.731Z level=INFO msg="Upload complete. (177ms)"7343lb # [ 164.669409] niks3-server[1112]: 2026/09/23 04:44:02 INFO Completed upload id=747344worker1 # [ 164.656093] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=5047345lb # [ 164.675360] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7346worker1 # [ 164.661079] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247347worker1 # [ 164.671246] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7348worker1 # [ 164.680133] nix-daemon[846]: reaped child process 1867, status = succeeded7349worker1 # [ 164.683128] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7350worker1 # [ 166.119921] nix-daemon[846]: reaped child process 1876, status = succeeded7351worker1 # [ 166.203089] nix-daemon[846]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)7352worker1 # [ 166.215625] nix-daemon[846]: reaped child process 1895, status = succeeded7353lb # [ 166.237630] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received uploads request method=POST path=/api/pending_closures7354worker1 # [ 166.231805] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7355worker1 # [ 166.234244] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.311Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7356lb # [ 166.283413] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7357worker1 # [ 166.274610] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.352Z level=INFO msg="Uploading 1 narinfos"7358lb # [ 166.290581] niks3-server[1112]: 2026/09/23 04:44:04 INFO Signed narinfos id=75 count=17359lb # [ 166.297969] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7360lb # [ 166.302422] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7361lb # [ 166.311063] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7362lb # [ 166.319467] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7363lb # [ 166.324118] niks3-server[1112]: 2026/09/23 04:44:04 INFO Completed upload id=757364worker1 # [ 166.307833] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.386Z level=INFO msg="Upload complete. (216ms)"7365lb # [ 166.328553] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7366worker1 # [ 166.313580] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17367worker2 # [ 166.297809] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:04Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17368worker1 # [ 166.323478] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247369worker2 # [ 166.307407] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:04Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07370worker1 # [ 166.335540] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247371worker1 # [ 166.341141] nix-daemon[846]: reaped child process 1874, status = succeeded7372worker1 # [ 166.346903] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.425Z level=INFO msg="All 1 paths already cached"7373worker1 # [ 166.352880] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247374worker1 # [ 166.359431] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7375worker1 # [ 166.446252] nix-daemon[846]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)7376worker1 # [ 166.459710] nix-daemon[846]: reaped child process 1902, status = succeeded7377lb # [ 166.481656] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received uploads request method=POST path=/api/pending_closures7378worker1 # [ 166.474061] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.552Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7379worker1 # [ 166.475946] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.553Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7380lb # [ 166.514426] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7381lb # [ 166.522686] niks3-server[1112]: 2026/09/23 04:44:04 INFO Signed narinfos id=76 count=17382worker1 # [ 166.510478] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.588Z level=INFO msg="Uploading 1 narinfos"7383lb # [ 166.526846] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7384lb # [ 166.534942] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7385lb # [ 166.544258] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7386lb # [ 166.551157] niks3-server[1112]: 2026/09/23 04:44:04 INFO Completed upload id=767387worker1 # [ 166.535350] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.613Z level=INFO msg="Upload complete. (168ms)"7388lb # [ 166.554336] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7389worker1 # [ 166.540079] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=6247390worker1 # [ 166.545433] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247391worker1 # [ 166.554463] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7392worker1 # [ 166.564591] nix-daemon[846]: reaped child process 1897, status = succeeded7393worker1 # [ 166.568210] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7394lb # [ 167.013668] envoy[927]: conn peer=[2001:db8:1::1]:52430 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=5163 ms=49267395worker1 # [ 167.783200] nix-daemon[846]: reaped child process 1906, status = succeeded7396worker1 # [ 167.858087] nix-daemon[846]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)7397worker1 # [ 167.869372] nix-daemon[846]: reaped child process 1925, status = succeeded7398lb # [ 167.893748] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received uploads request method=POST path=/api/pending_closures7399worker1 # [ 167.887850] nix-grpc-daemon[1714]: time=2026-09-23T04:44:05.965Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7400worker1 # [ 167.889365] nix-grpc-daemon[1714]: time=2026-09-23T04:44:05.967Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7401lb # [ 167.934326] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7402lb # [ 167.946982] niks3-server[1112]: 2026/09/23 04:44:05 INFO Signed narinfos id=77 count=17403worker1 # [ 167.934461] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.012Z level=INFO msg="Uploading 1 narinfos"7404lb # [ 167.953701] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7405lb # [ 167.960222] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7406lb # [ 167.965478] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7407lb # [ 167.975280] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7408lb # [ 167.981050] niks3-server[1112]: 2026/09/23 04:44:05 INFO Completed upload id=777409lb # [ 167.983689] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7410worker1 # [ 167.966628] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.044Z level=INFO msg="Upload complete. (192ms)"7411worker1 # [ 167.972580] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17412worker2 # [ 167.955616] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:06Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17413worker2 # [ 167.965470] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:06Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07414worker1 # [ 167.988313] nix-daemon[846]: reaped child process 1904, status = succeeded7415worker1 # [ 168.017402] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247416worker1 # [ 168.030098] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247417worker1 # [ 168.043131] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.121Z level=INFO msg="All 1 paths already cached"7418worker1 # [ 168.050870] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247419worker1 # [ 168.058785] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7420worker1 # [ 168.156894] nix-daemon[846]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)7421worker1 # [ 168.169376] nix-daemon[846]: reaped child process 1932, status = succeeded7422lb # [ 168.193686] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received uploads request method=POST path=/api/pending_closures7423worker1 # [ 168.186212] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7424worker1 # [ 168.187814] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.266Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7425lb # [ 168.227636] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7426lb # [ 168.234907] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7427lb # [ 168.239981] niks3-server[1112]: 2026/09/23 04:44:06 INFO Signed narinfos id=78 count=17428worker1 # [ 168.227871] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.306Z level=INFO msg="Uploading 1 narinfos"7429lb # [ 168.247518] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7430lb # [ 168.261248] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7431lb # [ 168.268402] niks3-server[1112]: 2026/09/23 04:44:06 INFO Completed upload id=787432lb # [ 168.270376] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7433worker1 # [ 168.252357] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.330Z level=INFO msg="Upload complete. (181ms)"7434worker1 # [ 168.256722] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=6247435worker1 # [ 168.262758] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247436worker1 # [ 168.273304] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7437worker1 # [ 168.283498] nix-daemon[846]: reaped child process 1927, status = succeeded7438worker1 # [ 168.287476] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7439worker1 # [ 169.113719] nix-daemon[846]: reaped child process 1936, status = succeeded7440worker1 # [ 169.185087] nix-daemon[846]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)7441worker1 # [ 169.196516] nix-daemon[846]: reaped child process 1955, status = succeeded7442lb # [ 169.220682] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures7443worker1 # [ 169.213636] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7444worker1 # [ 169.215125] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.293Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7445lb # [ 169.267719] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7446lb # [ 169.273115] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7447lb # [ 169.278132] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7448worker1 # [ 169.268034] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.346Z level=INFO msg="Uploading 1 narinfos"7449lb # [ 169.283969] niks3-server[1112]: 2026/09/23 04:44:07 INFO Signed narinfos id=79 count=17450lb # [ 169.289821] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7451lb # [ 169.298879] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7452lb # [ 169.303289] niks3-server[1112]: 2026/09/23 04:44:07 INFO Completed upload id=797453worker1 # [ 169.287587] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.365Z level=INFO msg="Upload complete. (194ms)"7454lb # [ 169.304804] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7455worker1 # [ 169.292636] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17456worker2 # [ 169.275745] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:07Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17457worker1 # [ 169.299360] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247458worker2 # [ 169.285709] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:07Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07459worker1 # [ 169.308555] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247460worker1 # [ 169.316520] nix-daemon[846]: reaped child process 1934, status = succeeded7461worker1 # [ 169.318562] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.396Z level=INFO msg="All 1 paths already cached"7462worker1 # [ 169.322497] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247463worker1 # [ 169.326148] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7464worker1 # [ 169.412249] nix-daemon[846]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)7465worker1 # [ 169.427361] nix-daemon[846]: reaped child process 1962, status = succeeded7466lb # [ 169.447750] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures7467worker1 # [ 169.440544] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.518Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7468worker1 # [ 169.442502] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.520Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7469lb # [ 169.478385] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7470lb # [ 169.485524] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7471worker1 # [ 169.475629] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.553Z level=INFO msg="Uploading 1 narinfos"7472lb # [ 169.491472] niks3-server[1112]: 2026/09/23 04:44:07 INFO Signed narinfos id=80 count=17473lb # [ 169.502847] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7474lb # [ 169.512367] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7475lb # [ 169.517892] niks3-server[1112]: 2026/09/23 04:44:07 INFO Completed upload id=807476worker1 # [ 169.501518] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.579Z level=INFO msg="Upload complete. (169ms)"7477worker1 # [ 169.503676] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=6247478lb # [ 169.521346] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7479worker1 # [ 169.508271] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247480worker1 # [ 169.514143] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7481worker1 # [ 169.516513] nix-daemon[846]: reaped child process 1957, status = succeeded7482worker1 # [ 169.521680] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7483worker1 # [ 170.534985] nix-daemon[846]: reaped child process 1966, status = succeeded7484worker1 # [ 170.608892] nix-daemon[846]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)7485worker1 # [ 170.621206] nix-daemon[846]: reaped child process 1985, status = succeeded7486lb # [ 170.646842] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures7487worker1 # [ 170.637352] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7488worker1 # [ 170.639731] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.717Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7489lb # [ 170.686548] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7490lb # [ 170.694801] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7491lb # [ 170.701167] niks3-server[1112]: 2026/09/23 04:44:08 INFO Signed narinfos id=81 count=17492worker1 # [ 170.688095] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.766Z level=INFO msg="Uploading 1 narinfos"7493lb # [ 170.706435] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7494lb # [ 170.711706] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7495lb # [ 170.723142] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7496lb # [ 170.726377] niks3-server[1112]: 2026/09/23 04:44:08 INFO Completed upload id=817497worker1 # [ 170.710418] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.788Z level=INFO msg="Upload complete. (182ms)"7498lb # [ 170.728406] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7499worker1 # [ 170.715894] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17500worker2 # [ 170.698383] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:08Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17501worker1 # [ 170.724296] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247502worker2 # [ 170.708440] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:08Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07503worker1 # [ 170.734363] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247504worker1 # [ 170.743694] nix-daemon[846]: reaped child process 1964, status = succeeded7505worker1 # [ 170.745849] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.823Z level=INFO msg="All 2 paths already cached"7506worker1 # [ 170.747951] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247507worker1 # [ 170.751724] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7508worker1 # [ 170.833913] nix-daemon[846]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)7509worker1 # [ 170.846866] nix-daemon[846]: reaped child process 1992, status = succeeded7510lb # [ 170.869455] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures7511worker1 # [ 170.861298] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.939Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7512worker1 # [ 170.863373] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.941Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7513lb # [ 170.898565] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7514lb # [ 170.905317] niks3-server[1112]: 2026/09/23 04:44:08 INFO Signed narinfos id=82 count=17515worker1 # [ 170.892761] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.970Z level=INFO msg="Uploading 1 narinfos"7516lb # [ 170.914431] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7517lb # [ 170.928143] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7518lb # [ 170.930962] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7519lb # [ 170.937971] niks3-server[1112]: 2026/09/23 04:44:08 INFO Completed upload id=827520worker1 # [ 170.921981] nix-grpc-daemon[1714]: time=2026-09-23T04:44:09.000Z level=INFO msg="Upload complete. (164ms)"7521lb # [ 170.941229] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7522worker1 # [ 170.925425] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=7527523worker1 # [ 170.930416] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247524worker1 # [ 170.939108] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7525worker1 # [ 170.946280] nix-daemon[846]: reaped child process 1987, status = succeeded7526worker1 # [ 170.950053] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7527worker1 # [ 172.390574] nix-daemon[846]: reaped child process 1996, status = succeeded7528worker1 # [ 172.470496] nix-daemon[846]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7529worker1 # [ 172.483784] nix-daemon[846]: reaped child process 2015, status = succeeded7530lb # [ 172.506637] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received uploads request method=POST path=/api/pending_closures7531worker1 # [ 172.499182] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.577Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7532worker1 # [ 172.500732] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.579Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7533lb # [ 172.546701] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7534lb # [ 172.553250] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7535lb # [ 172.562518] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7536worker1 # [ 172.550678] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.628Z level=INFO msg="Uploading 1 narinfos"7537lb # [ 172.566841] niks3-server[1112]: 2026/09/23 04:44:10 INFO Signed narinfos id=83 count=17538lb # [ 172.578477] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7539lb # [ 172.586907] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7540lb # [ 172.592136] niks3-server[1112]: 2026/09/23 04:44:10 INFO Completed upload id=837541worker1 # [ 172.575825] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.654Z level=INFO msg="Upload complete. (217ms)"7542lb # [ 172.595372] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7543worker1 # [ 172.581752] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17544worker2 # [ 172.564527] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:10Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17545worker1 # [ 172.596972] nix-daemon[846]: reaped child process 1994, status = succeeded7546worker1 # [ 172.633734] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247547worker1 # [ 172.646133] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247548worker1 # [ 172.659250] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.737Z level=INFO msg="All 2 paths already cached"7549worker1 # [ 172.667184] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247550worker1 # [ 172.675496] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7551worker1 # [ 172.769465] nix-daemon[846]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)7552worker1 # [ 172.781458] nix-daemon[846]: reaped child process 2022, status = succeeded7553lb # [ 172.805568] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received uploads request method=POST path=/api/pending_closures7554worker1 # [ 172.797206] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.875Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7555worker1 # [ 172.798734] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.877Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7556lb # [ 172.838781] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7557worker1 # [ 172.827696] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.906Z level=INFO msg="Uploading 1 narinfos"7558lb # [ 172.845179] niks3-server[1112]: 2026/09/23 04:44:10 INFO Signed narinfos id=84 count=17559lb # [ 172.850629] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7560lb # [ 172.855242] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7561lb # [ 172.867754] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7562lb # [ 172.875696] niks3-server[1112]: 2026/09/23 04:44:10 INFO Completed upload id=847563worker1 # [ 172.859538] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.937Z level=INFO msg="Upload complete. (171ms)"7564lb # [ 172.878273] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7565worker1 # [ 172.863488] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=7527566worker1 # [ 172.868238] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247567worker1 # [ 172.877449] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7568worker1 # [ 172.885386] nix-daemon[846]: reaped child process 2017, status = succeeded7569worker1 # [ 172.890190] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7570worker1 # [ 174.325402] nix-daemon[846]: reaped child process 2026, status = succeeded7571worker1 # [ 174.401885] nix-daemon[846]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7572worker1 # [ 174.414439] nix-daemon[846]: reaped child process 2045, status = succeeded7573lb # [ 174.436442] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received uploads request method=POST path=/api/pending_closures7574worker1 # [ 174.428959] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.507Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7575worker1 # [ 174.430446] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.508Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7576lb # [ 174.477283] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7577lb # [ 174.483124] niks3-server[1112]: 2026/09/23 04:44:12 INFO Signed narinfos id=85 count=17578worker1 # [ 174.470773] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.548Z level=INFO msg="Uploading 1 narinfos"7579lb # [ 174.494391] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7580lb # [ 174.498248] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7581lb # [ 174.502594] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7582lb # [ 174.514265] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7583lb # [ 174.517631] niks3-server[1112]: 2026/09/23 04:44:12 INFO Completed upload id=857584worker1 # [ 174.501897] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.580Z level=INFO msg="Upload complete. (203ms)"7585lb # [ 174.520516] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7586worker1 # [ 174.507947] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17587worker2 # [ 174.490811] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:12Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17588worker1 # [ 174.523199] nix-daemon[846]: reaped child process 2024, status = succeeded7589worker1 # [ 174.552977] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247590worker1 # [ 174.565478] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247591worker1 # [ 174.578498] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.656Z level=INFO msg="All 2 paths already cached"7592worker1 # [ 174.586295] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355247593worker1 # [ 174.596155] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7594worker1 # [ 174.688866] nix-daemon[846]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)7595worker1 # [ 174.701196] nix-daemon[846]: reaped child process 2052, status = succeeded7596lb # [ 174.724988] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received uploads request method=POST path=/api/pending_closures7597worker1 # [ 174.717112] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.795Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7598worker1 # [ 174.718672] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.796Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7599lb # [ 174.755225] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7600lb # [ 174.760856] niks3-server[1112]: 2026/09/23 04:44:12 INFO Signed narinfos id=86 count=17601worker1 # [ 174.748043] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.826Z level=INFO msg="Uploading 1 narinfos"7602lb # [ 174.769128] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7603lb # [ 174.782154] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7604worker1 # [ 174.773778] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.852Z level=INFO msg="Upload complete. (166ms)"7605lb # [ 174.790127] niks3-server[1112]: 2026/09/23 04:44:12 INFO Completed upload id=867606worker1 # [ 174.776080] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=7527607lb # [ 174.794502] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7608lb # [ 174.799849] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7609worker1 # [ 174.784060] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355247610worker1 # [ 174.792914] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7611worker1 # [ 174.795247] nix-daemon[846]: reaped child process 2047, status = succeeded7612worker1 # [ 174.800549] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted)7613worker1 # [ 176.230249] nix-daemon[846]: reaped child process 2056, status = succeeded7614worker1 # [ 176.300226] nix-daemon[846]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7615worker1 # [ 176.312025] nix-daemon[846]: reaped child process 2075, status = succeeded7616lb # [ 176.334886] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received uploads request method=POST path=/api/pending_closures7617worker1 # [ 176.328236] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.406Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7618worker1 # [ 176.331036] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.408Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7619lb # [ 176.382328] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7620lb # [ 176.385440] niks3-server[1112]: 2026/09/23 04:44:14 INFO Signed narinfos id=87 count=17621worker1 # [ 176.371721] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.449Z level=INFO msg="Uploading 1 narinfos"7622lb # [ 176.391745] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7623lb # [ 176.397451] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7624lb # [ 176.403928] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7625lb # [ 176.412981] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7626lb # [ 176.416294] niks3-server[1112]: 2026/09/23 04:44:14 INFO Completed upload id=877627worker1 # [ 176.401611] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.479Z level=INFO msg="Upload complete. (192ms)"7628lb # [ 176.420094] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7629worker1 # [ 176.406604] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17630worker1 # [ 176.418928] nix-daemon[846]: reaped child process 2054, status = succeeded7631client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.26 seconds)7632worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7633worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7634client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7635client # [ 176.549849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7636client # Running as unit: gpu.service; invocation ID: ebf89567922a4b9eb0a10950b2f27ab47637client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7638lb: must succeed: curl -sf localhost:9901/clusters7639lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7640lb: must succeed: curl -sf localhost:9901/clusters7641lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7642worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7643client # [ 176.682834] nix[1584]: don't know how to build these paths:7644client # [ 176.683685] nix[1584]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7645worker1 # [ 176.668742] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355247646worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7647worker1 # [ 176.673979] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355247648client # [ 176.707251] nix[1584]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7649worker2 # [ 176.675730] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:14Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07650lb # [ 177.014173] envoy[927]: conn peer=[2001:db8:1::1]:52446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27189 tx=12286 ms=150927651lb: must succeed: curl -sf localhost:9901/clusters7652lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7653lb: must succeed: curl -sf localhost:9901/clusters7654lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7655worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7656worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7657client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7658client # [ 206.708637] nix[1584]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7659client # [ 206.718388] nix[1584]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7660client # [ 206.751119] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7661client # [ 206.754924] systemd[1]: gpu.service: Failed with result 'exit-code'.7662client # [ 206.759243] systemd[1]: gpu.service: Consumed 153ms CPU time over 30.205s wall clock time, 37.6M memory peak, 5.5K incoming IP traffic, 5.2K outgoing IP traffic.7663lb # [ 207.015374] envoy[927]: conn peer=[2001:db8:1::1]:35228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300467664client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.44 seconds)7665client: must succeed: journalctl -u gpu -o cat7666client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7667lb: must succeed: curl -sf localhost:9901/clusters7668lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7669lb: must succeed: curl -sf localhost:9901/clusters7670lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7671worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7672worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7673(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.19 seconds)7674subtest: low disk drains a worker and builds go to the other7675worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7676worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds)7677worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7678worker1 # [ 209.055431] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:47Z level=info event=unhealthy reason=min_free7679worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)7680client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27681client # don't know how to build these paths:7682client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7683worker1 # [ 209.566994] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355247684worker2 # [ 209.554842] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907685worker2 # [ 209.575163] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07686worker2 # [ 209.580296] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17687worker2 # [ 209.620078] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907688worker2 # [ 209.632795] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907689client # copying 1 paths to 'lb:50051'...7690client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7691worker2 # [ 209.645386] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448907692worker2 # [ 209.656303] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7693worker2 # [ 209.758937] nix-daemon[869]: accepted connection from pid 2083, user nix-grpc-daemon (trusted)7694worker2 # [ 209.773224] nix-daemon[869]: reaped child process 2086, status = succeeded7695lb # [ 209.814827] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received uploads request method=POST path=/api/pending_closures7696worker2 # [ 209.787949] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7697worker2 # [ 209.790066] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.921Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7698lb # [ 209.857238] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7699lb # [ 209.865089] niks3-server[1112]: 2026/09/23 04:44:47 INFO Signed narinfos id=88 count=17700worker2 # [ 209.833718] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.965Z level=INFO msg="Uploading 1 narinfos"7701lb # [ 209.874525] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7702lb # [ 209.886067] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7703lb # [ 209.891904] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7704lb # [ 209.894869] niks3-server[1112]: 2026/09/23 04:44:47 INFO Completed upload id=887705lb # [ 209.897175] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7706worker2 # [ 209.861612] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.993Z level=INFO msg="Upload complete. (193ms)"7707worker2 # [ 209.866609] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=5127708worker2 # [ 209.872742] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907709worker2 # [ 209.883503] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7710worker2 # [ 209.893908] nix-daemon[869]: reaped child process 2081, status = succeeded7711client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7712worker2 # [ 209.901332] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7713client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7714client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7715client # farm-a> LOG-farm-a-drain7716worker2 # [ 210.949395] nix-daemon[869]: reaped child process 2090, status = succeeded7717worker2 # [ 211.028895] nix-daemon[869]: accepted connection from pid 2106, user nix-grpc-daemon (trusted)7718worker2 # [ 211.042512] nix-daemon[869]: reaped child process 2109, status = succeeded7719lb # [ 211.084852] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received uploads request method=POST path=/api/pending_closures7720worker2 # [ 211.059213] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.191Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7721worker2 # [ 211.060802] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.192Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7722lb # [ 211.126989] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7723lb # [ 211.134992] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7724lb # [ 211.137878] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7725worker2 # [ 211.109660] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.241Z level=INFO msg="Uploading 1 narinfos"7726lb # [ 211.144344] niks3-server[1112]: 2026/09/23 04:44:49 INFO Signed narinfos id=89 count=17727lb # [ 211.149413] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7728lb # [ 211.155066] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7729lb # [ 211.158924] niks3-server[1112]: 2026/09/23 04:44:49 INFO Completed upload id=897730worker2 # [ 211.124308] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.256Z level=INFO msg="Upload complete. (215ms)"7731lb # [ 211.161931] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7732worker2 # [ 211.132134] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17733worker2 # [ 211.144358] nix-daemon[869]: reaped child process 2088, status = succeeded7734worker2 # [ 211.178243] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07735worker2 # [ 211.182355] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17736worker2 # [ 211.232224] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907737worker2 # [ 211.245190] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907738worker2 # [ 211.258499] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.390Z level=INFO msg="All 1 paths already cached"7739client # copying 1 paths to 'lb:50051'...7740client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7741worker2 # [ 211.267475] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448907742worker2 # [ 211.276451] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7743worker2 # [ 211.379332] nix-daemon[869]: accepted connection from pid 2115, user nix-grpc-daemon (trusted)7744worker2 # [ 211.392840] nix-daemon[869]: reaped child process 2118, status = succeeded7745lb # [ 211.434488] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received uploads request method=POST path=/api/pending_closures7746worker2 # [ 211.408880] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.540Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7747worker2 # [ 211.410485] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.542Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7748lb # [ 211.468998] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7749lb # [ 211.474760] niks3-server[1112]: 2026/09/23 04:44:49 INFO Signed narinfos id=90 count=17750worker2 # [ 211.446671] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.578Z level=INFO msg="Uploading 1 narinfos"7751lb # [ 211.483960] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7752lb # [ 211.489299] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7753lb # [ 211.498935] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7754lb # [ 211.502611] niks3-server[1112]: 2026/09/23 04:44:49 INFO Completed upload id=907755worker2 # [ 211.467810] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.599Z level=INFO msg="Upload complete. (176ms)"7756lb # [ 211.504990] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7757worker2 # [ 211.472160] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=6247758worker2 # [ 211.477136] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907759worker2 # [ 211.486811] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7760worker2 # [ 211.492669] nix-daemon[869]: reaped child process 2113, status = succeeded7761worker2 # [ 211.497480] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7762client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7763client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7764client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7765client # farm-b> LOG-farm-b-drain7766worker2 # [ 212.134477] nix-daemon[869]: reaped child process 2122, status = succeeded7767worker2 # [ 212.215431] nix-daemon[869]: accepted connection from pid 2138, user nix-grpc-daemon (trusted)7768worker2 # [ 212.228413] nix-daemon[869]: reaped child process 2141, status = succeeded7769lb # [ 212.271400] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures7770worker2 # [ 212.246610] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7771worker2 # [ 212.248265] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.380Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7772lb # [ 212.321120] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7773lb # [ 212.326432] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7774lb # [ 212.332484] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7775worker2 # [ 212.301094] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.432Z level=INFO msg="Uploading 1 narinfos"7776lb # [ 212.335897] niks3-server[1112]: 2026/09/23 04:44:50 INFO Signed narinfos id=91 count=17777lb # [ 212.341345] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7778lb # [ 212.350497] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7779lb # [ 212.353496] niks3-server[1112]: 2026/09/23 04:44:50 INFO Completed upload id=917780lb # [ 212.355714] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7781worker2 # [ 212.320281] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.452Z level=INFO msg="Upload complete. (215ms)"7782worker2 # [ 212.325891] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17783worker2 # [ 212.340124] nix-daemon[869]: reaped child process 2120, status = succeeded7784worker2 # [ 212.373672] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07785worker2 # [ 212.380093] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17786worker2 # [ 212.422816] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907787worker2 # [ 212.435684] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448907788worker2 # [ 212.449545] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.581Z level=INFO msg="All 2 paths already cached"7789client # copying 1 paths to 'lb:50051'...7790client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7791worker2 # [ 212.458670] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448907792worker2 # [ 212.469179] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7793worker2 # [ 212.570592] nix-daemon[869]: accepted connection from pid 2147, user nix-grpc-daemon (trusted)7794worker2 # [ 212.585601] nix-daemon[869]: reaped child process 2150, status = succeeded7795lb # [ 212.625859] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures7796worker2 # [ 212.599912] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.731Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7797worker2 # [ 212.601500] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.733Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7798lb # [ 212.661473] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7799lb # [ 212.669662] niks3-server[1112]: 2026/09/23 04:44:50 INFO Signed narinfos id=92 count=17800worker2 # [ 212.638835] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.770Z level=INFO msg="Uploading 1 narinfos"7801lb # [ 212.677590] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7802lb # [ 212.682014] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7803lb # [ 212.696199] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7804lb # [ 212.703041] niks3-server[1112]: 2026/09/23 04:44:50 INFO Completed upload id=927805worker2 # [ 212.668330] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.800Z level=INFO msg="Upload complete. (186ms)"7806lb # [ 212.705417] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7807worker2 # [ 212.672477] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=7527808worker2 # [ 212.677384] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448907809worker2 # [ 212.687208] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7810worker2 # [ 212.696239] nix-daemon[869]: reaped child process 2145, status = succeeded7811worker2 # [ 212.700925] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7812client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7813client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7814client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7815client # farm-top> LOG-farm-top-drain7816worker2 # [ 213.537825] nix-daemon[869]: reaped child process 2154, status = succeeded7817worker2 # [ 213.622525] nix-daemon[869]: accepted connection from pid 2170, user nix-grpc-daemon (trusted)7818worker2 # [ 213.635376] nix-daemon[869]: reaped child process 2173, status = succeeded7819lb # [ 213.678799] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received uploads request method=POST path=/api/pending_closures7820worker2 # [ 213.652889] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7821worker2 # [ 213.654472] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.786Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7822lb # [ 213.721203] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7823worker2 # [ 213.694792] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.826Z level=INFO msg="Uploading 1 narinfos"7824lb # [ 213.730350] niks3-server[1112]: 2026/09/23 04:44:51 INFO Signed narinfos id=93 count=17825lb # [ 213.735595] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7826lb # [ 213.740203] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7827lb # [ 213.744917] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7828lb # [ 213.755696] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7829lb # [ 213.760169] niks3-server[1112]: 2026/09/23 04:44:51 INFO Completed upload id=937830lb # [ 213.763319] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7831worker2 # [ 213.726906] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.858Z level=INFO msg="Upload complete. (218ms)"7832worker2 # [ 213.735119] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=17833worker2 # [ 213.748802] nix-daemon[869]: reaped child process 2152, status = succeeded7834client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.36 seconds)7835client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7836client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7837worker1: must succeed: rm /nix/.rw-store/fill7838worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7839worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7840worker1 # [ 215.057383] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:53Z level=info event=healthy reason=min_free7841worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.06 seconds)7842(finished: subtest: low disk drains a worker and builds go to the other, in 7.78 seconds)7843(finished: run the VM test script, in 216.05 seconds)7844test script finished in 216.22s7845cleanup7846kill QemuMachine (pid 50)7847client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7848client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down7849client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down7850client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down7851kill QemuMachine (pid 49)7852lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7853lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7854lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7855lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7856kill QemuMachine (pid 51)7857worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7858worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7859worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7860worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7861kill QemuMachine (pid 52)7862worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7863worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7864worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7865worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down7866(finished: cleanup, in 1.10 seconds)7867additionally exposed symbols:7868 client, lb, worker1, worker2,7869 vlan1,7870 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh