vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #560
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14client # Formatting '/build/vm-state-client/tmp.aMRXmqMcSt', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: 9015bf7c-119a-4136-b544-1ca8f265942419client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26client # 27client # Virtualisation disk image created.28client # Starting virtiofs daemons...29client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)30client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether31client # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...32client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)33client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether34client # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...35client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)36client # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether37client # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...38client # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests39client # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests40client # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests41worker1 # Disk image does not exist, creating the virtualisation disk image...42worker1: QEMU running (pid 50)43worker1 # Formatting '/build/vm-state-worker1/tmp.coeHc2uJCp', fmt=raw size=107374182444worker1 # mke2fs 1.47.4 (6-Mar-2025)45worker1 # Discarding device blocks: 0/262144 done46worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes47worker1 # Filesystem UUID: 64106a2a-e6ea-4ade-9149-0aa75911ce2348worker1 # Superblock backups stored on blocks:49worker1 # 32768, 98304, 163840, 22937650worker1 # 51worker1 # Allocating group tables: 0/8 done52worker1 # Writing inode tables: 0/8 done53worker1 # Creating journal (8192 blocks): done54worker1 # Writing superblocks and filesystem accounting information: 0/8 done55worker1 # 56worker1 # Virtualisation disk image created.57worker1 # Starting virtiofs daemons...58worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)59worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether60lb # Disk image does not exist, creating the virtualisation disk image...61lb: QEMU running (pid 51)62lb # Formatting '/build/vm-state-lb/tmp.iA0XHVnSzH', fmt=raw size=107374182463worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...64lb # mke2fs 1.47.4 (6-Mar-2025)65lb # Discarding device blocks: 0/262144 done66worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67lb # Creating filesystem with 262144 4k blocks and 65536 inodes68worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69lb # Filesystem UUID: 07335991-04c2-49b5-b1b4-54c37da3080a70worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...71lb # Superblock backups stored on blocks:72worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73lb # 32768, 98304, 163840, 22937674worker1 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75lb # 76worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...77lb # Allocating group tables: 0/8 done78worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests79lb # Writing inode tables: 0/8 done80worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests81lb # Creating journal (8192 blocks): done82worker1 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests83lb # Writing superblocks and filesystem accounting information: 0/8 done84lb # 85lb # Virtualisation disk image created.86lb # Starting virtiofs daemons...87lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # Disk image does not exist, creating the virtualisation disk image...90worker2: QEMU running (pid 52)91worker2 # Formatting '/build/vm-state-worker2/tmp.x9OtHfwvN0', fmt=raw size=107374182492lb # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # mke2fs 1.47.4 (6-Mar-2025)94lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker2 # Discarding device blocks: 0/262144 done96lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes98lb # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...99worker2 # Filesystem UUID: f16a84ab-7980-4d0e-a794-679d69a5229f100lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101worker2 # Superblock backups stored on blocks:102lb # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether103worker2 # 32768, 98304, 163840, 229376104lb # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker2 # 106lb # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests107worker2 # Allocating group tables: 0/8 done108lb # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests109worker2 # Writing inode tables: 0/8 done110lb # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests111worker2 # Creating journal (8192 blocks): done112(finished: start all VMs, in 0.29 seconds)113worker2 # Writing superblocks and filesystem accounting information: 0/8 done114client: waiting for unit network-addresses-eth1.service115worker2 # 116client: waiting for the VM to finish booting117worker2 # Virtualisation disk image created.118worker2 # Starting virtiofs daemons...119worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125worker2 # [2026-09-25T16:49:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...127worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-25T16:49:39Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137lb # 138lb # 139worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141client # 142client # 143client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00144worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...145worker1 # 146worker1 # 147worker1 # 148worker1 # 149worker2 # 150worker2 # 151lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...152lb # 153lb # 154lb # 155lb # 156worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00157worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00158client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...159client # 160client # 161client # 162client # 163lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00164client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...166worker2 # 167worker2 # 168worker2 # 169worker2 # 170worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...171worker1 # 172worker1 # 173worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00174lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175lb # 176lb # 177client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178client # 179client # 180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183worker1 # Booting from ROM...184lb # Booting from ROM...185worker2 # Booting from ROM...186client # Booting from ROM...187worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xi6vvckx9rwwnrsyw13p0hn9l4ymyaqm-nixos-system-worker1-test/init regInfo=/nix/store/0crbgjyg4d9ilxgszl4bmvhb2biinlkn-closure-info/registration console=ttyS0,115200n8 console=tty0189worker1 # [ 0.000000] BIOS-provided physical RAM map:190worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved198worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved199worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/i0c2a3l8xkam6y95ah3pciy5d6i6rs74-nixos-system-worker2-test/init regInfo=/nix/store/5gpdilpnzviwpqhacyfc0v04nf2rz70p-closure-info/registration console=ttyS0,115200n8 console=tty0202worker2 # [ 0.000000] BIOS-provided physical RAM map:203worker1 # [ 0.000000] NX (Execute Disable) protection: active204worker1 # [ 0.000000] APIC: Static calls initialized205worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable206worker1 # [ 0.000000] SMBIOS 2.8 present.207worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved208worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014209worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved210worker1 # [ 0.000000] DMI: Memory slots populated: 1/1211lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026212worker1 # [ 0.000000] Hypervisor detected: KVM213worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable214client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026215worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000216worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved217worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00218worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved219worker1 # [ 0.000000] kvm-clock: using sched offset of 475296145 cycles220worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved221worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns222lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0223worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved224worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor225lb # [ 0.000000] BIOS-provided physical RAM map:226client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w787jbyr7zgkhjvgnixpm9jzpqdir8w7-nixos-system-client-test/init regInfo=/nix/store/q8bf3j9221ah2d8xbgfry1x0xgb7nh6i-closure-info/registration console=ttyS0,115200n8 console=tty0227worker1 # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000228worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved229client # [ 0.000000] BIOS-provided physical RAM map:230lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable231worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved232client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable233worker1 # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs234lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved235worker2 # [ 0.000000] NX (Execute Disable) protection: active236worker1 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT237client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved238worker2 # [ 0.000000] APIC: Static calls initialized239lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved240worker2 # [ 0.000000] SMBIOS 2.8 present.241worker1 # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]242client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved243lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable244worker1 # [ 0.002738] Using GB pages for direct mapping245client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable246worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014247worker1 # [ 0.002861] RAMDISK: [mem 0x5e370000-0x5ffcffff]248lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved249worker2 # [ 0.000000] DMI: Memory slots populated: 1/1250client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved251worker1 # [ 0.002869] ACPI: Early table checksum verification disabled252worker2 # [ 0.000000] Hypervisor detected: KVM253lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved254worker1 # [ 0.002872] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )255client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved256worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000257lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved258worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved260worker1 # [ 0.002875] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)261lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved262worker2 # [ 0.000000] kvm-clock: using sched offset of 487228444 cycles263client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved264worker1 # [ 0.002879] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)265lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved266client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved267worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns268lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved269worker1 # [ 0.002886] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)270worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor271client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved272lb # [ 0.000000] NX (Execute Disable) protection: active273worker1 # [ 0.002888] ACPI: FACS 0x000000005FFE0000 000040274client # [ 0.000000] NX (Execute Disable) protection: active275lb # [ 0.000000] APIC: Static calls initialized276worker2 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000277lb # [ 0.000000] SMBIOS 2.8 present.278client # [ 0.000000] APIC: Static calls initialized279worker1 # [ 0.002890] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)280client # [ 0.000000] SMBIOS 2.8 present.281worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs282worker1 # [ 0.002891] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)283lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014284worker2 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT285client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014286lb # [ 0.000000] DMI: Memory slots populated: 1/1287worker2 # [ 0.002717] found SMP MP-table at [mem 0x000f5450-0x000f545f]288lb # [ 0.000000] Hypervisor detected: KVM289worker1 # [ 0.002893] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)290client # [ 0.000000] DMI: Memory slots populated: 1/1291worker2 # [ 0.002729] Using GB pages for direct mapping292client # [ 0.000000] Hypervisor detected: KVM293lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000294worker2 # [ 0.002874] RAMDISK: [mem 0x5e36f000-0x5ffcffff]295client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000296worker1 # [ 0.002894] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)297lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00298worker2 # [ 0.002882] ACPI: Early table checksum verification disabled299client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00300worker1 # [ 0.002896] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]301lb # [ 0.000000] kvm-clock: using sched offset of 462093309 cycles302worker2 # [ 0.002885] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )303client # [ 0.000000] kvm-clock: using sched offset of 475598181 cycles304worker1 # [ 0.002897] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]305lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns306worker2 # [ 0.002888] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)307worker1 # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]308client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns309lb # [ 0.000005] tsc: Detected 2400.012 MHz processor310client # [ 0.000005] tsc: Detected 2400.012 MHz processor311worker2 # [ 0.002892] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)312worker1 # [ 0.002898] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]313lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000314client # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000315worker1 # [ 0.002898] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]316worker2 # [ 0.002899] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)317lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs318worker2 # [ 0.002901] ACPI: FACS 0x000000005FFE0000 000040319client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs320worker1 # [ 0.002899] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]321lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT322client # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT323worker2 # [ 0.002902] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)324worker1 # [ 0.002899] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]325lb # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]326worker1 # [ 0.003123] No NUMA configuration found327client # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]328lb # [ 0.002742] Using GB pages for direct mapping329client # [ 0.002748] Using GB pages for direct mapping330worker2 # [ 0.002904] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)331worker1 # [ 0.003124] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]332lb # [ 0.002903] RAMDISK: [mem 0x5e36e000-0x5ffcffff]333client # [ 0.002909] RAMDISK: [mem 0x5e370000-0x5ffcffff]334lb # [ 0.002912] ACPI: Early table checksum verification disabled335worker2 # [ 0.002906] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)336worker1 # [ 0.003126] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]337client # [ 0.002918] ACPI: Early table checksum verification disabled338worker1 # [ 0.005796] Zone ranges:339lb # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )340client # [ 0.002921] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )341worker2 # [ 0.002907] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)342worker1 # [ 0.005798] DMA [mem 0x0000000000001000-0x0000000000ffffff]343lb # [ 0.002919] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)344worker1 # [ 0.005799] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]345worker2 # [ 0.002908] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]346client # [ 0.002925] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)347worker1 # [ 0.005801] Normal empty348worker1 # [ 0.005801] Device empty349lb # [ 0.002923] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)350worker2 # [ 0.002909] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]351client # [ 0.002928] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)352worker1 # [ 0.005802] Movable zone start for each node353worker1 # [ 0.005803] Early memory node ranges354worker2 # [ 0.002910] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]355lb # [ 0.002930] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)356client # [ 0.002935] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)357worker1 # [ 0.005803] node 0: [mem 0x0000000000001000-0x000000000009efff]358lb # [ 0.002932] ACPI: FACS 0x000000005FFE0000 000040359worker2 # [ 0.002910] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]360client # [ 0.002937] ACPI: FACS 0x000000005FFE0000 000040361worker1 # [ 0.005804] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]362worker2 # [ 0.002911] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]363lb # [ 0.002934] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)364client # [ 0.002939] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)365worker1 # [ 0.005805] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]366worker2 # [ 0.002911] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]367lb # [ 0.002935] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)368worker1 # [ 0.005825] On node 0, zone DMA: 1 pages in unavailable ranges369client # [ 0.002941] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)370worker2 # [ 0.002912] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]371worker1 # [ 0.006088] On node 0, zone DMA: 97 pages in unavailable ranges372lb # [ 0.002937] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)373worker2 # [ 0.003130] No NUMA configuration found374client # [ 0.002942] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)375worker1 # [ 0.032724] On node 0, zone DMA32: 40 pages in unavailable ranges376worker2 # [ 0.003131] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]377lb # [ 0.002938] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)378worker1 # [ 0.033174] ACPI: PM-Timer IO Port: 0x608379client # [ 0.002944] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker2 # [ 0.003134] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]381worker1 # [ 0.033185] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])382lb # [ 0.002940] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]383worker2 # [ 0.005455] Zone ranges:384client # [ 0.002945] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]385worker1 # [ 0.033212] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23386worker2 # [ 0.005455] DMA [mem 0x0000000000001000-0x0000000000ffffff]387lb # [ 0.002941] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]388client # [ 0.002946] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]389worker1 # [ 0.033214] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)390worker2 # [ 0.005457] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]391lb # [ 0.002941] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]392client # [ 0.002946] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]393worker2 # [ 0.005458] Normal empty394worker1 # [ 0.033216] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)395worker2 # [ 0.005459] Device empty396lb # [ 0.002942] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]397client # [ 0.002947] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]398worker2 # [ 0.005459] Movable zone start for each node399worker1 # [ 0.033217] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)400worker2 # [ 0.005460] Early memory node ranges401lb # [ 0.002942] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]402client # [ 0.002947] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]403worker1 # [ 0.033218] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)404worker2 # [ 0.005460] node 0: [mem 0x0000000000001000-0x000000000009efff]405lb # [ 0.002943] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]406client # [ 0.002948] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]407worker1 # [ 0.033219] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)408worker2 # [ 0.005461] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]409lb # [ 0.002943] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]410client # [ 0.002948] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]411worker1 # [ 0.033221] ACPI: Using ACPI (MADT) for SMP configuration information412worker2 # [ 0.005462] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]413lb # [ 0.003182] No NUMA configuration found414client # [ 0.003181] No NUMA configuration found415worker1 # [ 0.033222] ACPI: HPET id: 0x8086a201 base: 0xfed00000416worker2 # [ 0.005481] On node 0, zone DMA: 1 pages in unavailable ranges417lb # [ 0.003183] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]418client # [ 0.003182] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]419worker1 # [ 0.033226] TSC deadline timer available420worker2 # [ 0.005749] On node 0, zone DMA: 97 pages in unavailable ranges421client # [ 0.003184] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]422lb # [ 0.003186] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]423worker1 # [ 0.033230] CPU topo: Max. logical packages: 1424client # [ 0.005600] Zone ranges:425lb # [ 0.005960] Zone ranges:426worker1 # [ 0.033230] CPU topo: Max. logical dies: 1427worker2 # [ 0.032436] On node 0, zone DMA32: 40 pages in unavailable ranges428worker2 # [ 0.032897] ACPI: PM-Timer IO Port: 0x608429client # [ 0.005600] DMA [mem 0x0000000000001000-0x0000000000ffffff]430lb # [ 0.005961] DMA [mem 0x0000000000001000-0x0000000000ffffff]431worker1 # [ 0.033231] CPU topo: Max. dies per package: 1432worker1 # [ 0.033234] CPU topo: Max. threads per core: 1433worker2 # [ 0.032908] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])434lb # [ 0.005963] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]435client # [ 0.005602] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]436client # [ 0.005603] Normal empty437worker1 # [ 0.033235] CPU topo: Num. cores per package: 1438lb # [ 0.005964] Normal empty439client # [ 0.005604] Device empty440lb # [ 0.005965] Device empty441worker2 # [ 0.032934] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23442worker1 # [ 0.033235] CPU topo: Num. threads per package: 1443client # [ 0.005605] Movable zone start for each node444lb # [ 0.005965] Movable zone start for each node445worker2 # [ 0.032936] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)446worker1 # [ 0.033235] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs447client # [ 0.005606] Early memory node ranges448lb # [ 0.005966] Early memory node ranges449worker2 # [ 0.032938] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)450client # [ 0.005606] node 0: [mem 0x0000000000001000-0x000000000009efff]451lb # [ 0.005967] node 0: [mem 0x0000000000001000-0x000000000009efff]452worker1 # [ 0.033253] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()453worker2 # [ 0.032939] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)454lb # [ 0.005968] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]455client # [ 0.005607] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]456worker1 # [ 0.033283] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]457worker2 # [ 0.032940] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)458client # [ 0.005608] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]459lb # [ 0.005969] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]460worker1 # [ 0.033284] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]461client # [ 0.005632] On node 0, zone DMA: 1 pages in unavailable ranges462worker2 # [ 0.032941] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)463lb # [ 0.005989] On node 0, zone DMA: 1 pages in unavailable ranges464worker1 # [ 0.033286] [mem 0x60000000-0xafffffff] available for PCI devices465client # [ 0.005902] On node 0, zone DMA: 97 pages in unavailable ranges466lb # [ 0.006276] On node 0, zone DMA: 97 pages in unavailable ranges467worker2 # [ 0.032944] ACPI: Using ACPI (MADT) for SMP configuration information468worker1 # [ 0.033287] Booting paravirtualized kernel on KVM469client # [ 0.034053] On node 0, zone DMA32: 40 pages in unavailable ranges470worker2 # [ 0.032945] ACPI: HPET id: 0x8086a201 base: 0xfed00000471lb # [ 0.034488] On node 0, zone DMA32: 40 pages in unavailable ranges472worker2 # [ 0.032948] TSC deadline timer available473lb # [ 0.034953] ACPI: PM-Timer IO Port: 0x608474client # [ 0.034520] ACPI: PM-Timer IO Port: 0x608475worker1 # [ 0.033290] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns476worker2 # [ 0.032952] CPU topo: Max. logical packages: 1477client # [ 0.034533] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])478lb # [ 0.034964] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])479worker2 # [ 0.032953] CPU topo: Max. logical dies: 1480worker1 # [ 0.037756] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1481client # [ 0.034570] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23482lb # [ 0.034994] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23483worker2 # [ 0.032953] CPU topo: Max. dies per package: 1484worker1 # [ 0.039851] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152485client # [ 0.034572] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)486worker2 # [ 0.032956] CPU topo: Max. threads per core: 1487lb # [ 0.034996] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)488worker1 # [ 0.039893] kvm-guest: PV spinlocks disabled, single CPU489worker2 # [ 0.032957] CPU topo: Num. cores per package: 1490lb # [ 0.034998] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)491client # [ 0.034574] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)492worker2 # [ 0.032957] CPU topo: Num. threads per package: 1493client # [ 0.034575] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)494lb # [ 0.035000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)495worker2 # [ 0.032958] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs496client # [ 0.034576] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)497lb # [ 0.035001] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)498worker2 # [ 0.032976] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()499client # [ 0.034577] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)500lb # [ 0.035001] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)501worker2 # [ 0.033005] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]502client # [ 0.034579] ACPI: Using ACPI (MADT) for SMP configuration information503worker1 # [ 0.039895] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xi6vvckx9rwwnrsyw13p0hn9l4ymyaqm-nixos-system-worker1-test/init regInfo=/nix/store/0crbgjyg4d9ilxgszl4bmvhb2biinlkn-closure-info/registration console=ttyS0,115200n8 console=tty0504lb # [ 0.035004] ACPI: Using ACPI (MADT) for SMP configuration information505client # [ 0.034580] ACPI: HPET id: 0x8086a201 base: 0xfed00000506worker2 # [ 0.033007] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]507lb # [ 0.035005] ACPI: HPET id: 0x8086a201 base: 0xfed00000508client # [ 0.034584] TSC deadline timer available509lb # [ 0.035008] TSC deadline timer available510worker2 # [ 0.033008] [mem 0x60000000-0xafffffff] available for PCI devices511client # [ 0.034588] CPU topo: Max. logical packages: 1512lb # [ 0.035012] CPU topo: Max. logical packages: 1513worker1 # [ 0.039986] Unknown kernel command line parameters "regInfo=/nix/store/0crbgjyg4d9ilxgszl4bmvhb2biinlkn-closure-info/registration", will be passed to user space.514worker2 # [ 0.033009] Booting paravirtualized kernel on KVM515client # [ 0.034589] CPU topo: Max. logical dies: 1516worker1 # [ 0.039999] random: crng init done517lb # [ 0.035013] CPU topo: Max. logical dies: 1518client # [ 0.034589] CPU topo: Max. dies per package: 1519lb # [ 0.035013] CPU topo: Max. dies per package: 1520worker1 # [ 0.040000] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes521worker2 # [ 0.033012] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns522client # [ 0.034592] CPU topo: Max. threads per core: 1523lb # [ 0.035016] CPU topo: Max. threads per core: 1524client # [ 0.034593] CPU topo: Num. cores per package: 1525worker2 # [ 0.037484] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1526worker1 # [ 0.042064] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)527lb # [ 0.035017] CPU topo: Num. cores per package: 1528client # [ 0.034593] CPU topo: Num. threads per package: 1529lb # [ 0.035017] CPU topo: Num. threads per package: 1530worker2 # [ 0.039594] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152531worker1 # [ 0.043096] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)532client # [ 0.034593] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs533lb # [ 0.035017] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs534worker2 # [ 0.039635] kvm-guest: PV spinlocks disabled, single CPU535worker1 # [ 0.043126] Fallback order for Node 0: 0536client # [ 0.034613] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()537lb # [ 0.035039] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()538worker1 # [ 0.043129] Built 1 zonelists, mobility grouping on. Total pages: 393078539worker1 # [ 0.043130] Policy zone: DMA32540client # [ 0.034645] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]541lb # [ 0.035068] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]542worker1 # [ 0.045710] mem auto-init: stack:all(zero), heap alloc:on, heap free:off543client # [ 0.034647] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]544lb # [ 0.035070] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]545worker1 # [ 0.048741] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1546client # [ 0.034648] [mem 0x60000000-0xafffffff] available for PCI devices547worker1 # [ 0.051961] allocated 3145728 bytes of page_ext548lb # [ 0.035071] [mem 0x60000000-0xafffffff] available for PCI devices549client # [ 0.034650] Booting paravirtualized kernel on KVM550worker2 # [ 0.039636] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/i0c2a3l8xkam6y95ah3pciy5d6i6rs74-nixos-system-worker2-test/init regInfo=/nix/store/5gpdilpnzviwpqhacyfc0v04nf2rz70p-closure-info/registration console=ttyS0,115200n8 console=tty0551worker1 # [ 0.061722] ftrace: allocating 48787 entries in 192 pages552lb # [ 0.035073] Booting paravirtualized kernel on KVM553worker1 # [ 0.061724] ftrace: allocated 192 pages with 2 groups554client # [ 0.034653] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns555worker1 # [ 0.062532] Dynamic Preempt: lazy556lb # [ 0.035076] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns557worker2 # [ 0.039728] Unknown kernel command line parameters "regInfo=/nix/store/5gpdilpnzviwpqhacyfc0v04nf2rz70p-closure-info/registration", will be passed to user space.558worker1 # [ 0.062659] rcu: Preemptible hierarchical RCU implementation.559client # [ 0.039137] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1560worker2 # [ 0.039741] random: crng init done561lb # [ 0.039560] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1562worker1 # [ 0.062660] rcu: RCU event tracing is enabled.563client # [ 0.041706] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152564worker2 # [ 0.039741] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes565lb # [ 0.041831] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152566worker1 # [ 0.062661] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.567client # [ 0.041752] kvm-guest: PV spinlocks disabled, single CPU568lb # [ 0.041877] kvm-guest: PV spinlocks disabled, single CPU569worker2 # [ 0.041824] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)570worker1 # [ 0.062662] Trampoline variant of Tasks RCU enabled.571worker1 # [ 0.062662] Rude variant of Tasks RCU enabled.572worker1 # [ 0.062663] Tracing variant of Tasks RCU enabled.573worker2 # [ 0.043079] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)574worker2 # [ 0.043108] Fallback order for Node 0: 0575worker1 # [ 0.062663] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.576worker2 # [ 0.043111] Built 1 zonelists, mobility grouping on. Total pages: 393078577worker2 # [ 0.043112] Policy zone: DMA32578worker1 # [ 0.062664] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1579client # [ 0.041754] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w787jbyr7zgkhjvgnixpm9jzpqdir8w7-nixos-system-client-test/init regInfo=/nix/store/q8bf3j9221ah2d8xbgfry1x0xgb7nh6i-closure-info/registration console=ttyS0,115200n8 console=tty0580worker2 # [ 0.045573] mem auto-init: stack:all(zero), heap alloc:on, heap free:off581lb # [ 0.041879] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0582worker1 # [ 0.062674] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.583worker2 # [ 0.048602] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1584worker2 # [ 0.051816] allocated 3145728 bytes of page_ext585worker1 # [ 0.062676] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586client # [ 0.041851] Unknown kernel command line parameters "regInfo=/nix/store/q8bf3j9221ah2d8xbgfry1x0xgb7nh6i-closure-info/registration", will be passed to user space.587lb # [ 0.041975] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.588client # [ 0.041863] random: crng init done589worker2 # [ 0.061414] ftrace: allocating 48787 entries in 192 pages590lb # [ 0.041987] random: crng init done591worker1 # [ 0.062677] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.592worker2 # [ 0.061416] ftrace: allocated 192 pages with 2 groups593client # [ 0.041864] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes594worker2 # [ 0.062241] Dynamic Preempt: lazy595lb # [ 0.041987] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes596worker1 # [ 0.067069] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16597worker2 # [ 0.062400] rcu: Preemptible hierarchical RCU implementation.598client # [ 0.044103] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)599worker1 # [ 0.067347] rcu: srcu_init: Setting srcu_struct sizes based on contention.600lb # [ 0.044226] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)601worker2 # [ 0.062400] rcu: RCU event tracing is enabled.602client # [ 0.045226] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)603lb # [ 0.045351] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)604worker2 # [ 0.062401] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.605worker1 # [ 0.067353] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns606client # [ 0.045257] Fallback order for Node 0: 0607lb # [ 0.045383] Fallback order for Node 0: 0608worker2 # [ 0.062402] Trampoline variant of Tasks RCU enabled.609client # [ 0.045260] Built 1 zonelists, mobility grouping on. Total pages: 393078610worker2 # [ 0.062403] Rude variant of Tasks RCU enabled.611client # [ 0.045261] Policy zone: DMA32612lb # [ 0.045386] Built 1 zonelists, mobility grouping on. Total pages: 393078613worker1 # [ 0.067452] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)614worker2 # [ 0.062403] Tracing variant of Tasks RCU enabled.615lb # [ 0.045387] Policy zone: DMA32616worker1 # [ 0.070995] Console: colour VGA+ 80x25617client # [ 0.047898] mem auto-init: stack:all(zero), heap alloc:on, heap free:off618worker1 # [ 0.070998] printk: legacy console [tty0] enabled619lb # [ 0.048286] mem auto-init: stack:all(zero), heap alloc:on, heap free:off620worker2 # [ 0.062404] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.621client # [ 0.051293] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1622worker1 # [ 0.100374] printk: legacy console [ttyS0] enabled623client # [ 0.055084] allocated 3145728 bytes of page_ext624worker2 # [ 0.062404] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1625lb # [ 0.051647] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1626worker1 # [ 0.195858] ACPI: Core revision 20250807627lb # [ 0.055147] allocated 3145728 bytes of page_ext628client # [ 0.065026] ftrace: allocating 48787 entries in 192 pages629worker2 # [ 0.062426] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.630lb # [ 0.065141] ftrace: allocating 48787 entries in 192 pages631client # [ 0.065028] ftrace: allocated 192 pages with 2 groups632client # [ 0.065880] Dynamic Preempt: lazy633lb # [ 0.065143] ftrace: allocated 192 pages with 2 groups634worker1 # [ 0.196705] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns635lb # [ 0.066015] Dynamic Preempt: lazy636worker2 # [ 0.062427] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.637client # [ 0.066011] rcu: Preemptible hierarchical RCU implementation.638worker1 # [ 0.198177] APIC: Switch to symmetric I/O mode setup639lb # [ 0.066151] rcu: Preemptible hierarchical RCU implementation.640client # [ 0.066012] rcu: RCU event tracing is enabled.641worker2 # [ 0.062428] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.642lb # [ 0.066151] rcu: RCU event tracing is enabled.643worker1 # [ 0.199102] x2apic enabled644client # [ 0.066012] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.645worker2 # [ 0.067026] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16646lb # [ 0.066152] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.647client # [ 0.066014] Trampoline variant of Tasks RCU enabled.648worker1 # [ 0.199831] APIC: Switched APIC routing to: physical x2apic649client # [ 0.066014] Rude variant of Tasks RCU enabled.650worker2 # [ 0.067308] rcu: srcu_init: Setting srcu_struct sizes based on contention.651lb # [ 0.066153] Trampoline variant of Tasks RCU enabled.652lb # [ 0.066154] Rude variant of Tasks RCU enabled.653client # [ 0.066015] Tracing variant of Tasks RCU enabled.654lb # [ 0.066154] Tracing variant of Tasks RCU enabled.655worker2 # [ 0.067314] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns656client # [ 0.066015] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.657worker1 # [ 0.201607] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1658lb # [ 0.066155] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.659client # [ 0.066016] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1660worker2 # [ 0.067413] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)661worker2 # [ 0.070926] Console: colour VGA+ 80x25662lb # [ 0.066155] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1663client # [ 0.066077] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.664worker1 # [ 0.202546] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns665worker2 # [ 0.070929] printk: legacy console [tty0] enabled666worker2 # [ 0.100326] printk: legacy console [ttyS0] enabled667lb # [ 0.066216] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668client # [ 0.066079] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.669worker2 # [ 0.196032] ACPI: Core revision 20250807670worker1 # [ 0.204104] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)671lb # [ 0.066218] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.672client # [ 0.066079] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.673worker1 # [ 0.206187] x86/cpu: User Mode Instruction Prevention (UMIP) activated674lb # [ 0.066219] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.675client # [ 0.070513] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16676worker2 # [ 0.196923] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns677worker1 # [ 0.207234] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127678lb # [ 0.070653] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16679client # [ 0.070792] rcu: srcu_init: Setting srcu_struct sizes based on contention.680worker2 # [ 0.198400] APIC: Switch to symmetric I/O mode setup681lb # [ 0.070934] rcu: srcu_init: Setting srcu_struct sizes based on contention.682worker1 # [ 0.208101] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0683worker2 # [ 0.199325] x2apic enabled684client # [ 0.070800] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns685lb # [ 0.070942] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns686worker2 # [ 0.200050] APIC: Switched APIC routing to: physical x2apic687client # [ 0.070906] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)688worker1 # [ 0.209104] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto689client # [ 0.074436] Console: colour VGA+ 80x25690lb # [ 0.071052] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)691client # [ 0.074438] printk: legacy console [tty0] enabled692lb # [ 0.074586] Console: colour VGA+ 80x25693worker1 # [ 0.210101] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl694client # [ 0.103870] printk: legacy console [ttyS0] enabled695worker2 # [ 0.201830] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1696lb # [ 0.074589] printk: legacy console [tty0] enabled697lb # [ 0.104071] printk: legacy console [ttyS0] enabled698client # [ 0.198604] ACPI: Core revision 20250807699worker1 # [ 0.212101] Transient Scheduler Attacks: Vulnerable: No microcode700lb # [ 0.200139] ACPI: Core revision 20250807701worker2 # [ 0.202795] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns702worker1 # [ 0.213101] Spectre V2 : Mitigation: Enhanced / Automatic IBRS703client # [ 0.199491] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns704worker1 # [ 0.214101] Speculative Return Stack Overflow: Mitigation: Safe RET705worker2 # [ 0.204423] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)706lb # [ 0.201039] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns707client # [ 0.200992] APIC: Switch to symmetric I/O mode setup708client # [ 0.201923] x2apic enabled709lb # [ 0.202545] APIC: Switch to symmetric I/O mode setup710worker2 # [ 0.206506] x86/cpu: User Mode Instruction Prevention (UMIP) activated711lb # [ 0.203508] x2apic enabled712client # [ 0.202656] APIC: Switched APIC routing to: physical x2apic713worker2 # [ 0.207531] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127714lb # [ 0.204289] APIC: Switched APIC routing to: physical x2apic715worker2 # [ 0.208414] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0716worker1 # [ 0.215101] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization717client # [ 0.204423] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1718worker1 # [ 0.217107] Spectre V2 : Enabling IBPB for BPF719lb # [ 0.206130] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1720worker2 # [ 0.209418] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto721client # [ 0.205371] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns722worker1 # [ 0.218623] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier723worker2 # [ 0.210414] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl724worker1 # [ 0.219101] active return thunk: srso_alias_return_thunk725lb # [ 0.207119] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns726client # [ 0.206942] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)727worker2 # [ 0.211414] Transient Scheduler Attacks: Vulnerable: No microcode728worker1 # [ 0.220726] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'729lb # [ 0.208779] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)730client # [ 0.209028] x86/cpu: User Mode Instruction Prevention (UMIP) activated731worker2 # [ 0.212414] Spectre V2 : Mitigation: Enhanced / Automatic IBRS732worker1 # [ 0.221964] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'733client # [ 0.210115] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127734lb # [ 0.209864] x86/cpu: User Mode Instruction Prevention (UMIP) activated735worker2 # [ 0.213414] Speculative Return Stack Overflow: Mitigation: Safe RET736worker1 # [ 0.222827] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'737client # [ 0.210939] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0738lb # [ 0.210954] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127739worker1 # [ 0.223822] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'740lb # [ 0.212474] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0741worker1 # [ 0.224101] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'742client # [ 0.211943] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto743worker1 # [ 0.225101] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'744lb # [ 0.213533] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto745client # [ 0.212939] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl746worker2 # [ 0.214414] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization747worker1 # [ 0.226100] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'748client # [ 0.213939] Transient Scheduler Attacks: Vulnerable: No microcode749worker2 # [ 0.216420] Spectre V2 : Enabling IBPB for BPF750lb # [ 0.214776] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl751worker1 # [ 0.227984] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256752client # [ 0.214939] Spectre V2 : Mitigation: Enhanced / Automatic IBRS753lb # [ 0.215776] Transient Scheduler Attacks: Vulnerable: No microcode754worker2 # [ 0.217415] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier755worker1 # [ 0.228805] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64756client # [ 0.215939] Speculative Return Stack Overflow: Mitigation: Safe RET757lb # [ 0.217476] Spectre V2 : Mitigation: Enhanced / Automatic IBRS758worker2 # [ 0.218414] active return thunk: srso_alias_return_thunk759worker1 # [ 0.229801] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512760lb # [ 0.218454] Speculative Return Stack Overflow: Mitigation: Safe RET761worker1 # [ 0.230100] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024762worker2 # [ 0.219435] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'763worker1 # [ 0.231100] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8764worker2 # [ 0.220414] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'765client # [ 0.216939] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization766worker2 # [ 0.222201] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'767worker1 # [ 0.232100] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.768client # [ 0.218947] Spectre V2 : Enabling IBPB for BPF769worker2 # [ 0.223154] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'770lb # [ 0.218776] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization771worker2 # [ 0.224170] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'772client # [ 0.220449] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier773lb # [ 0.220784] Spectre V2 : Enabling IBPB for BPF774client # [ 0.221939] active return thunk: srso_alias_return_thunk775worker2 # [ 0.225152] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776lb # [ 0.221777] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier777client # [ 0.222783] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'778worker2 # [ 0.226213] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'779lb # [ 0.222776] active return thunk: srso_alias_return_thunk780client # [ 0.223939] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'781worker2 # [ 0.227414] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256782lb # [ 0.224429] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'783client # [ 0.224939] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'784worker2 # [ 0.228414] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64785lb # [ 0.225682] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'786client # [ 0.225939] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'787worker2 # [ 0.229352] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512788lb # [ 0.226536] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'789client # [ 0.226939] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'790worker2 # [ 0.230414] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024791lb # [ 0.227776] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'792client # [ 0.227939] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'793worker2 # [ 0.231414] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8794lb # [ 0.228776] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'795client # [ 0.228939] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'796worker2 # [ 0.232355] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.797lb # [ 0.229776] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'798client # [ 0.229940] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256799lb # [ 0.230776] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'800client # [ 0.230939] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64801client # [ 0.231939] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512802lb # [ 0.231776] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256803client # [ 0.232939] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024804lb # [ 0.232776] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64805client # [ 0.233939] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8806lb # [ 0.233776] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512807lb # [ 0.234775] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024808client # [ 0.234939] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.809lb # [ 0.235776] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810lb # [ 0.236776] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811worker1 # [ 0.266016] Freeing SMP alternatives memory: 44K812worker1 # [ 0.266662] pid_max: default: 32768 minimum: 301813worker1 # [ 0.267187] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker1 # [ 0.268204] landlock: Up and running.815worker1 # [ 0.269101] Yama: becoming mindful.816worker1 # [ 0.270196] LSM support for eBPF active817worker1 # [ 0.270952] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.272145] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker1 # [ 0.274719] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker2 # [ 0.266490] Freeing SMP alternatives memory: 44K821worker2 # [ 0.267299] pid_max: default: 32768 minimum: 301822worker1 # [ 0.275670] Performance Events: Fam17h+ core perfctr, AMD PMU driver.823worker1 # [ 0.276106] ... version: 2824worker2 # [ 0.268068] LSM: initializing lsm=capability,landlock,yama,bpf,ima825worker1 # [ 0.276810] ... bit width: 48826worker2 # [ 0.268505] landlock: Up and running.827worker1 # [ 0.277103] ... generic counters: 6828worker2 # [ 0.269874] Yama: becoming mindful.829worker1 # [ 0.277788] ... generic bitmap: 000000000000003f830worker2 # [ 0.270629] LSM support for eBPF active831worker1 # [ 0.278108] ... fixed-purpose counters: 0832worker1 # [ 0.278811] ... fixed-purpose bitmap: 0000000000000000833worker2 # [ 0.271513] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)834worker1 # [ 0.279103] ... value mask: 0000ffffffffffff835client # [ 0.269836] Freeing SMP alternatives memory: 44K836worker2 # [ 0.272460] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)837worker1 # [ 0.279974] ... max period: 00007fffffffffff838client # [ 0.270497] pid_max: default: 32768 minimum: 301839worker1 # [ 0.280743] ... global_ctrl mask: 000000000000003f840client # [ 0.271059] LSM: initializing lsm=capability,landlock,yama,bpf,ima841worker1 # [ 0.281199] signal: max sigframe size: 3376842client # [ 0.272059] landlock: Up and running.843worker2 # [ 0.275204] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)844lb # [ 0.272761] Freeing SMP alternatives memory: 44K845worker1 # [ 0.281961] rcu: Hierarchical SRCU implementation.846client # [ 0.272939] Yama: becoming mindful.847lb # [ 0.273329] pid_max: default: 32768 minimum: 301848worker1 # [ 0.282665] rcu: Max phase no-delay instances is 400.849client # [ 0.273979] LSM support for eBPF active850worker2 # [ 0.275944] Performance Events: Fam17h+ core perfctr, AMD PMU driver.851lb # [ 0.273881] LSM: initializing lsm=capability,landlock,yama,bpf,ima852worker2 # [ 0.276419] ... version: 2853client # [ 0.274720] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)854lb # [ 0.274883] landlock: Up and running.855worker2 # [ 0.277104] ... bit width: 48856lb # [ 0.276219] Yama: becoming mindful.857worker2 # [ 0.277416] ... generic counters: 6858client # [ 0.275818] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)859lb # [ 0.276990] LSM support for eBPF active860worker2 # [ 0.278138] ... generic bitmap: 000000000000003f861worker2 # [ 0.278462] ... fixed-purpose counters: 0862lb # [ 0.277926] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)863client # [ 0.278223] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)864worker2 # [ 0.279193] ... fixed-purpose bitmap: 0000000000000000865worker1 # [ 0.287650] smp: Bringing up secondary CPUs ...866lb # [ 0.278823] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)867worker1 # [ 0.288117] smp: Brought up 1 node, 1 CPU868worker2 # [ 0.279417] ... value mask: 0000ffffffffffff869client # [ 0.279530] Performance Events: Fam17h+ core perfctr, AMD PMU driver.870worker2 # [ 0.280328] ... max period: 00007fffffffffff871worker1 # [ 0.288780] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)872client # [ 0.279947] ... version: 2873worker2 # [ 0.281136] ... global_ctrl mask: 000000000000003f874client # [ 0.280617] ... bit width: 48875worker2 # [ 0.281548] signal: max sigframe size: 3376876lb # [ 0.281876] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)877client # [ 0.280942] ... generic counters: 6878worker1 # [ 0.289259] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)879client # [ 0.281614] ... generic bitmap: 000000000000003f880worker2 # [ 0.282333] rcu: Hierarchical SRCU implementation.881worker1 # [ 0.290324] devtmpfs: initialized882client # [ 0.281941] ... fixed-purpose counters: 0883worker2 # [ 0.283015] rcu: Max phase no-delay instances is 400.884lb # [ 0.283375] Performance Events: Fam17h+ core perfctr, AMD PMU driver.885worker1 # [ 0.291129] x86/mm: Memory block size: 128MB886client # [ 0.282608] ... fixed-purpose bitmap: 0000000000000000887lb # [ 0.283785] ... version: 2888lb # [ 0.284529] ... bit width: 48889client # [ 0.282941] ... value mask: 0000ffffffffffff890lb # [ 0.284779] ... generic counters: 6891client # [ 0.283806] ... max period: 00007fffffffffff892worker1 # [ 0.292877] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)893lb # [ 0.285511] ... generic bitmap: 000000000000003f894client # [ 0.284586] ... global_ctrl mask: 000000000000003f895lb # [ 0.285779] ... fixed-purpose counters: 0896worker1 # [ 0.293973] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).897client # [ 0.285053] signal: max sigframe size: 3376898lb # [ 0.286502] ... fixed-purpose bitmap: 0000000000000000899client # [ 0.285866] rcu: Hierarchical SRCU implementation.900worker1 # [ 0.295190] pinctrl core: initialized pinctrl subsystem901worker2 # [ 0.287974] smp: Bringing up secondary CPUs ...902lb # [ 0.286778] ... value mask: 0000ffffffffffff903client # [ 0.286488] rcu: Max phase no-delay instances is 400.904worker2 # [ 0.288429] smp: Brought up 1 node, 1 CPU905worker1 # [ 0.296333] PM: RTC time: 16:49:39, date: 2026-09-25906lb # [ 0.287632] ... max period: 00007fffffffffff907worker2 # [ 0.289075] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)908lb # [ 0.288457] ... global_ctrl mask: 000000000000003f909lb # [ 0.288884] signal: max sigframe size: 3376910lb # [ 0.289702] rcu: Hierarchical SRCU implementation.911worker2 # [ 0.289570] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)912worker1 # [ 0.299756] NET: Registered PF_NETLINK/PF_ROUTE protocol family913lb # [ 0.290380] rcu: Max phase no-delay instances is 400.914worker2 # [ 0.290659] devtmpfs: initialized915worker2 # [ 0.291519] x86/mm: Memory block size: 128MB916client # [ 0.291562] smp: Bringing up secondary CPUs ...917worker1 # [ 0.300598] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations918client # [ 0.291954] smp: Brought up 1 node, 1 CPU919worker1 # [ 0.301124] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations920client # [ 0.292604] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)921worker2 # [ 0.293303] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)922worker1 # [ 0.302375] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations923worker1 # [ 0.303113] audit: initializing netlink subsys (disabled)924worker2 # [ 0.294372] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).925client # [ 0.293107] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)926lb # [ 0.295444] smp: Bringing up secondary CPUs ...927worker2 # [ 0.295497] pinctrl core: initialized pinctrl subsystem928client # [ 0.294210] devtmpfs: initialized929worker1 # [ 0.304179] thermal_sys: Registered thermal governor 'fair_share'930lb # [ 0.295801] smp: Brought up 1 node, 1 CPU931client # [ 0.295026] x86/mm: Memory block size: 128MB932worker1 # [ 0.304181] thermal_sys: Registered thermal governor 'bang_bang'933worker2 # [ 0.296635] PM: RTC time: 16:49:39, date: 2026-09-25934lb # [ 0.296498] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)935worker1 # [ 0.305094] thermal_sys: Registered thermal governor 'step_wise'936worker1 # [ 0.305802] thermal_sys: Registered thermal governor 'user_space'937client # [ 0.296783] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)938lb # [ 0.296933] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)939worker1 # [ 0.306107] audit: type=2000 audit(1790354980.020:1): state=initialized audit_enabled=0 res=1940client # [ 0.297789] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).941lb # [ 0.298052] devtmpfs: initialized942worker2 # [ 0.300059] NET: Registered PF_NETLINK/PF_ROUTE protocol family943worker1 # [ 0.308097] thermal_sys: Registered thermal governor 'power_allocator'944client # [ 0.298992] pinctrl core: initialized pinctrl subsystem945lb # [ 0.298848] x86/mm: Memory block size: 128MB946worker1 # [ 0.308116] cpuidle: using governor menu947worker2 # [ 0.300913] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations948client # [ 0.300071] PM: RTC time: 16:49:39, date: 2026-09-25949worker1 # [ 0.310113] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5950worker2 # [ 0.301438] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations951lb # [ 0.300582] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)952worker1 # [ 0.311326] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]953worker2 # [ 0.302691] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations954lb # [ 0.301695] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).955worker2 # [ 0.303426] audit: initializing netlink subsys (disabled)956worker1 # [ 0.312106] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry957lb # [ 0.302859] pinctrl core: initialized pinctrl subsystem958client # [ 0.303926] NET: Registered PF_NETLINK/PF_ROUTE protocol family959worker1 # [ 0.313115] PCI: Using configuration type 1 for base access960lb # [ 0.304010] PM: RTC time: 16:49:39, date: 2026-09-25961worker2 # [ 0.304627] thermal_sys: Registered thermal governor 'fair_share'962worker2 # [ 0.304629] thermal_sys: Registered thermal governor 'bang_bang'963client # [ 0.305240] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations964worker1 # [ 0.314248] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.965client # [ 0.305980] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations966worker2 # [ 0.305435] audit: type=2000 audit(1790354980.030:1): state=initialized audit_enabled=0 res=1967worker2 # [ 0.307418] thermal_sys: Registered thermal governor 'step_wise'968lb # [ 0.307697] NET: Registered PF_NETLINK/PF_ROUTE protocol family969client # [ 0.307236] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations970worker2 # [ 0.307420] thermal_sys: Registered thermal governor 'user_space'971client # [ 0.307953] audit: initializing netlink subsys (disabled)972worker2 # [ 0.308393] thermal_sys: Registered thermal governor 'power_allocator'973lb # [ 0.309021] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations974client # [ 0.309077] thermal_sys: Registered thermal governor 'fair_share'975worker2 # [ 0.309174] cpuidle: using governor menu976worker1 # [ 0.319342] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages977lb # [ 0.309801] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations978client # [ 0.309079] thermal_sys: Registered thermal governor 'bang_bang'979worker1 # [ 0.320104] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page980worker2 # [ 0.310629] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5981client # [ 0.309942] thermal_sys: Registered thermal governor 'step_wise'982lb # [ 0.311068] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations983client # [ 0.310872] thermal_sys: Registered thermal governor 'user_space'984lb # [ 0.311789] audit: initializing netlink subsys (disabled)985worker2 # [ 0.311645] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]986client # [ 0.311669] audit: type=2000 audit(1790354980.038:1): state=initialized audit_enabled=0 res=1987worker2 # [ 0.312419] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry988lb # [ 0.312908] thermal_sys: Registered thermal governor 'fair_share'989client # [ 0.312944] thermal_sys: Registered thermal governor 'power_allocator'990worker2 # [ 0.313428] PCI: Using configuration type 1 for base access991lb # [ 0.312911] thermal_sys: Registered thermal governor 'bang_bang'992client # [ 0.312959] cpuidle: using governor menu993lb # [ 0.313779] thermal_sys: Registered thermal governor 'step_wise'994worker1 # [ 0.325104] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages995worker2 # [ 0.314496] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.996lb # [ 0.314692] thermal_sys: Registered thermal governor 'user_space'997worker1 # [ 0.326103] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page998client # [ 0.315222] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5999lb # [ 0.315518] audit: type=2000 audit(1790354980.040:1): state=initialized audit_enabled=0 res=11000client # [ 0.316173] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1001lb # [ 0.316781] thermal_sys: Registered thermal governor 'power_allocator'1002client # [ 0.316944] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1003lb # [ 0.316794] cpuidle: using governor menu1004client # [ 0.317961] PCI: Using configuration type 1 for base access1005lb # [ 0.319043] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51006client # [ 0.319167] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1007lb # [ 0.320032] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1008worker2 # [ 0.321432] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1009lb # [ 0.320782] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1010worker2 # [ 0.322417] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1011lb # [ 0.321792] PCI: Using configuration type 1 for base access1012lb # [ 0.322922] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1013worker1 # [ 0.336530] ACPI: Added _OSI(Module Device)1014worker1 # [ 0.337106] ACPI: Added _OSI(Processor Device)1015client # [ 0.324177] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1016worker1 # [ 0.337822] ACPI: Added _OSI(Processor Aggregator Device)1017client # [ 0.324943] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1018worker2 # [ 0.327418] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1019worker2 # [ 0.328417] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1020lb # [ 0.328057] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1021lb # [ 0.328779] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1022client # [ 0.329942] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1023client # [ 0.330941] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1024worker1 # [ 0.346652] ACPI: 1 ACPI AML tables successfully acquired and loaded1025lb # [ 0.333780] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1026lb # [ 0.334778] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1027worker2 # [ 0.338741] ACPI: Added _OSI(Module Device)1028worker2 # [ 0.339418] ACPI: Added _OSI(Processor Device)1029worker1 # [ 0.350356] ACPI: Interpreter enabled1030worker2 # [ 0.340198] ACPI: Added _OSI(Processor Aggregator Device)1031worker1 # [ 0.351033] ACPI: PM: (supports S0 S3 S4 S5)1032worker1 # [ 0.351644] ACPI: Using IOAPIC for interrupt routing1033worker1 # [ 0.352162] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1034client # [ 0.341328] ACPI: Added _OSI(Module Device)1035client # [ 0.341943] ACPI: Added _OSI(Processor Device)1036worker1 # [ 0.353104] PCI: Using E820 reservations for host bridge windows1037client # [ 0.342666] ACPI: Added _OSI(Processor Aggregator Device)1038worker1 # [ 0.356191] ACPI: Enabled 2 GPEs in block 00 to 3F1039lb # [ 0.345172] ACPI: Added _OSI(Module Device)1040lb # [ 0.345780] ACPI: Added _OSI(Processor Device)1041worker2 # [ 0.348046] ACPI: 1 ACPI AML tables successfully acquired and loaded1042lb # [ 0.346563] ACPI: Added _OSI(Processor Aggregator Device)1043client # [ 0.348799] ACPI: 1 ACPI AML tables successfully acquired and loaded1044worker2 # [ 0.351674] ACPI: Interpreter enabled1045worker2 # [ 0.352341] ACPI: PM: (supports S0 S3 S4 S5)1046worker2 # [ 0.352992] ACPI: Using IOAPIC for interrupt routing1047worker1 # [ 0.364776] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1048worker2 # [ 0.353474] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049worker2 # [ 0.354417] PCI: Using E820 reservations for host bridge windows1050worker1 # [ 0.365109] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1051client # [ 0.355344] ACPI: Interpreter enabled1052client # [ 0.355956] ACPI: PM: (supports S0 S3 S4 S5)1053lb # [ 0.355340] ACPI: 1 ACPI AML tables successfully acquired and loaded1054worker1 # [ 0.366423] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1055client # [ 0.356634] ACPI: Using IOAPIC for interrupt routing1056worker1 # [ 0.367228] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1057worker2 # [ 0.357563] ACPI: Enabled 2 GPEs in block 00 to 3F1058lb # [ 0.356717] ACPI: Interpreter enabled1059worker1 # [ 0.368431] PCI host bridge to bus 0000:001060lb # [ 0.357229] ACPI: PM: (supports S0 S3 S4 S5)1061worker1 # [ 0.369107] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1062client # [ 0.359003] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1063worker1 # [ 0.370103] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1064lb # [ 0.359779] ACPI: Using IOAPIC for interrupt routing1065client # [ 0.359942] PCI: Using E820 reservations for host bridge windows1066worker1 # [ 0.371103] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1067worker1 # [ 0.372103] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1068lb # [ 0.360692] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1069worker1 # [ 0.373103] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1070client # [ 0.363021] ACPI: Enabled 2 GPEs in block 00 to 3F1071worker1 # [ 0.374103] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1072lb # [ 0.363778] PCI: Using E820 reservations for host bridge windows1073worker2 # [ 0.365708] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1074worker1 # [ 0.375104] pci_bus 0000:00: root bus resource [bus 00-ff]1075lb # [ 0.364844] ACPI: Enabled 2 GPEs in block 00 to 3F1076worker2 # [ 0.366423] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1077worker1 # [ 0.376071] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1078worker2 # [ 0.367493] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1079worker1 # [ 0.377567] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1080worker2 # [ 0.368538] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker2 # [ 0.369754] PCI host bridge to bus 0000:001082client # [ 0.370392] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1083worker2 # [ 0.370421] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1084worker2 # [ 0.371417] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1085client # [ 0.370948] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1086worker2 # [ 0.372417] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1087client # [ 0.372020] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1088lb # [ 0.373542] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1089worker2 # [ 0.373416] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1090client # [ 0.373065] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1091client # [ 0.374303] PCI host bridge to bus 0000:001092worker2 # [ 0.374417] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093worker1 # [ 0.380165] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1094lb # [ 0.374516] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1095client # [ 0.374946] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1096worker2 # [ 0.375417] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1097worker1 # [ 0.381116] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1098lb # [ 0.375857] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1099client # [ 0.375942] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1100worker2 # [ 0.376417] pci_bus 0000:00: root bus resource [bus 00-ff]1101worker1 # [ 0.382036] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1102lb # [ 0.376913] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1103client # [ 0.376941] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104worker2 # [ 0.377372] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1105lb # [ 0.378122] PCI host bridge to bus 0000:001106worker1 # [ 0.382980] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1107client # [ 0.377942] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1108worker2 # [ 0.378872] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1109lb # [ 0.378782] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1110client # [ 0.378941] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1111lb # [ 0.379779] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1112worker1 # [ 0.384791] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1113client # [ 0.379941] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1114lb # [ 0.380778] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1115client # [ 0.380942] pci_bus 0000:00: root bus resource [bus 00-ff]1116lb # [ 0.381779] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1117client # [ 0.381898] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1118worker1 # [ 0.388121] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1119lb # [ 0.382778] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1120worker1 # [ 0.388941] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1121client # [ 0.383394] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1122lb # [ 0.383779] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1123worker1 # [ 0.389820] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1124lb # [ 0.384778] pci_bus 0000:00: root bus resource [bus 00-ff]1125worker1 # [ 0.390109] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1126lb # [ 0.385824] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1127worker2 # [ 0.381505] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1128worker1 # [ 0.391600] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1129worker2 # [ 0.382430] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1130lb # [ 0.387263] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1131worker2 # [ 0.383398] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1132worker1 # [ 0.393120] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1133worker2 # [ 0.384662] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1134worker1 # [ 0.393962] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1135worker1 # [ 0.394838] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1136client # [ 0.386009] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1137worker2 # [ 0.386312] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138client # [ 0.387022] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1139worker1 # [ 0.395655] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1140client # [ 0.387965] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1141worker2 # [ 0.388458] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1142client # [ 0.389302] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1143worker2 # [ 0.389351] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1144lb # [ 0.389890] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1145worker2 # [ 0.390200] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1146worker1 # [ 0.398122] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1147lb # [ 0.390790] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1148client # [ 0.390650] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1149worker1 # [ 0.399712] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1150worker2 # [ 0.391258] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1151lb # [ 0.391697] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1152worker1 # [ 0.400125] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1153worker2 # [ 0.392999] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1154lb # [ 0.393102] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1155client # [ 0.393988] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1156worker1 # [ 0.401670] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1157client # [ 0.394796] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1158worker2 # [ 0.394457] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1159client # [ 0.395759] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1160lb # [ 0.394524] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1161worker1 # [ 0.403121] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1162worker2 # [ 0.395323] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1163worker1 # [ 0.403931] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1164client # [ 0.396740] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1165worker2 # [ 0.396170] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1166worker1 # [ 0.404814] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1167worker1 # [ 0.405109] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1168lb # [ 0.397820] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1169client # [ 0.398503] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1170worker2 # [ 0.396952] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1171lb # [ 0.398670] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1172worker1 # [ 0.407207] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1173lb # [ 0.399608] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1174client # [ 0.400982] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1175lb # [ 0.400814] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1176client # [ 0.401851] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1177worker1 # [ 0.409125] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1178worker2 # [ 0.399615] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1179client # [ 0.402704] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1180worker2 # [ 0.400423] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1181worker1 # [ 0.410068] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1182lb # [ 0.402366] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183worker2 # [ 0.401355] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1184client # [ 0.403585] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1185worker1 # [ 0.411453] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186lb # [ 0.404758] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1187worker2 # [ 0.402881] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1188lb # [ 0.405400] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1189worker1 # [ 0.413126] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1190lb # [ 0.405800] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1191worker1 # [ 0.414072] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1192worker2 # [ 0.404424] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1193client # [ 0.406981] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1194worker2 # [ 0.405277] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195lb # [ 0.407349] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196client # [ 0.407817] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1197worker2 # [ 0.406226] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1198worker1 # [ 0.415478] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1199client # [ 0.408641] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1200worker2 # [ 0.407244] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1201worker1 # [ 0.418116] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1202client # [ 0.409534] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1203worker1 # [ 0.419054] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1204worker2 # [ 0.408999] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1205lb # [ 0.409822] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1206lb # [ 0.410747] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1207worker1 # [ 0.420475] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1208worker2 # [ 0.410247] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1209client # [ 0.411892] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1210lb # [ 0.411552] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1211worker2 # [ 0.411185] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1212client # [ 0.412573] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1213worker1 # [ 0.422145] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1214lb # [ 0.413182] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1215worker2 # [ 0.412005] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker1 # [ 0.423109] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1217client # [ 0.412963] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1218client # [ 0.413948] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1219worker2 # [ 0.413462] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1220lb # [ 0.415817] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1221worker1 # [ 0.424642] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1222worker2 # [ 0.414438] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1223lb # [ 0.416672] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1224client # [ 0.415491] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225worker1 # [ 0.427149] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1226lb # [ 0.417540] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1227worker2 # [ 0.416490] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1228worker1 # [ 0.427965] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1229client # [ 0.416993] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1230lb # [ 0.418571] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1231worker1 # [ 0.428797] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1232client # [ 0.417946] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1233worker2 # [ 0.418244] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1234lb # [ 0.420076] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1235worker2 # [ 0.419234] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1236worker1 # [ 0.429674] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1237client # [ 0.419540] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1238worker2 # [ 0.420774] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239lb # [ 0.421826] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1240worker1 # [ 0.431144] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1241lb # [ 0.422800] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1242client # [ 0.421942] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1243worker2 # [ 0.422430] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1244client # [ 0.422920] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1245worker1 # [ 0.431984] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1246lb # [ 0.424336] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1247worker2 # [ 0.423439] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1248worker1 # [ 0.432834] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249client # [ 0.424302] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1250worker2 # [ 0.425454] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1251worker1 # [ 0.433685] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.426518] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1253client # [ 0.425954] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1254lb # [ 0.426828] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1255worker1 # [ 0.434813] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1256worker2 # [ 0.427468] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1257client # [ 0.426909] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1258worker2 # [ 0.428326] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1259worker1 # [ 0.435345] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1260lb # [ 0.428358] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1261worker2 # [ 0.429151] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1262client # [ 0.428352] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1263worker1 # [ 0.436764] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1264lb # [ 0.429791] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1265worker2 # [ 0.429954] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1266worker1 # [ 0.437330] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1267lb # [ 0.430779] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1268client # [ 0.429954] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1269worker1 # [ 0.439416] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1270client # [ 0.431737] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1271lb # [ 0.432337] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1272worker2 # [ 0.431424] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1273worker2 # [ 0.432451] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1274client # [ 0.433308] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1275worker2 # [ 0.433438] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1276lb # [ 0.434456] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1277lb # [ 0.434801] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1278worker2 # [ 0.435002] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1279client # [ 0.435766] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1280client # [ 0.436563] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1281worker2 # [ 0.435937] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1282lb # [ 0.436414] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1283client # [ 0.436963] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1284worker2 # [ 0.436610] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1285lb # [ 0.438565] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1286worker2 # [ 0.437951] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1287client # [ 0.438517] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1288lb # [ 0.439438] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1289worker2 # [ 0.438603] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1290lb # [ 0.439801] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1291client # [ 0.440566] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1292worker2 # [ 0.439983] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1293worker1 # [ 0.440322] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1294client # [ 0.440950] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1295lb # [ 0.441339] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1296worker1 # [ 0.442708] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1297client # [ 0.441942] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1298worker1 # [ 0.443406] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1299lb # [ 0.443436] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1300client # [ 0.443499] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1301lb # [ 0.443786] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1302worker1 # [ 0.444392] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1303client # [ 0.444815] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1304lb # [ 0.444744] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1305worker1 # [ 0.445359] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1306lb # [ 0.446201] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1307client # [ 0.445769] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1308worker1 # [ 0.447091] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1309lb # [ 0.447557] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1310client # [ 0.447804] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1311worker1 # [ 0.447796] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1312client # [ 0.448781] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1313worker1 # [ 0.448448] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1314lb # [ 0.448000] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1315worker2 # [ 0.440651] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1316client # [ 0.450683] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1317worker1 # [ 0.449798] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1318lb # [ 0.449578] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1319worker2 # [ 0.442960] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320lb # [ 0.450594] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1321worker2 # [ 0.443699] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1322lb # [ 0.452470] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1323worker2 # [ 0.444704] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1324worker1 # [ 0.453138] ACPI: PCI: Interrupt link LNKA configured for IRQ 101325worker1 # [ 0.454210] ACPI: PCI: Interrupt link LNKB configured for IRQ 101326worker2 # [ 0.445656] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1327worker1 # [ 0.455204] ACPI: PCI: Interrupt link LNKC configured for IRQ 111328worker1 # [ 0.456204] ACPI: PCI: Interrupt link LNKD configured for IRQ 111329worker2 # [ 0.447439] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1330worker1 # [ 0.457224] ACPI: PCI: Interrupt link LNKE configured for IRQ 101331worker2 # [ 0.448356] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1332worker1 # [ 0.458199] ACPI: PCI: Interrupt link LNKF configured for IRQ 101333worker2 # [ 0.449488] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1334worker1 # [ 0.459210] ACPI: PCI: Interrupt link LNKG configured for IRQ 111335worker1 # [ 0.460209] ACPI: PCI: Interrupt link LNKH configured for IRQ 111336worker2 # [ 0.451012] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1337client # [ 0.451231] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1338worker1 # [ 0.461143] ACPI: PCI: Interrupt link GSIA configured for IRQ 161339worker1 # [ 0.462119] ACPI: PCI: Interrupt link GSIB configured for IRQ 171340client # [ 0.453746] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1341worker1 # [ 0.463118] ACPI: PCI: Interrupt link GSIC configured for IRQ 181342lb # [ 0.453000] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1343worker1 # [ 0.464120] ACPI: PCI: Interrupt link GSID configured for IRQ 191344client # [ 0.454983] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1345worker2 # [ 0.454512] ACPI: PCI: Interrupt link LNKA configured for IRQ 101346worker1 # [ 0.465116] ACPI: PCI: Interrupt link GSIE configured for IRQ 201347worker2 # [ 0.455527] ACPI: PCI: Interrupt link LNKB configured for IRQ 101348lb # [ 0.455549] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1349client # [ 0.456226] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1350worker1 # [ 0.466116] ACPI: PCI: Interrupt link GSIF configured for IRQ 211351worker2 # [ 0.456514] ACPI: PCI: Interrupt link LNKC configured for IRQ 111352worker1 # [ 0.467097] ACPI: PCI: Interrupt link GSIG configured for IRQ 221353lb # [ 0.456764] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1354client # [ 0.457242] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1355worker2 # [ 0.457521] ACPI: PCI: Interrupt link LNKD configured for IRQ 111356worker1 # [ 0.468041] ACPI: PCI: Interrupt link GSIH configured for IRQ 231357worker2 # [ 0.458524] ACPI: PCI: Interrupt link LNKE configured for IRQ 101358lb # [ 0.458066] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1359client # [ 0.458969] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1360worker2 # [ 0.459516] ACPI: PCI: Interrupt link LNKF configured for IRQ 101361worker1 # [ 0.469964] iommu: Default domain type: Translated1362lb # [ 0.459083] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1363client # [ 0.459861] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1364worker2 # [ 0.460518] ACPI: PCI: Interrupt link LNKG configured for IRQ 111365worker1 # [ 0.470710] iommu: DMA domain TLB invalidation policy: lazy mode1366worker1 # [ 0.471377] ACPI: bus type USB registered1367worker2 # [ 0.461513] ACPI: PCI: Interrupt link LNKH configured for IRQ 111368client # [ 0.460998] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1369lb # [ 0.460809] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1370worker1 # [ 0.472128] usbcore: registered new interface driver usbfs1371worker2 # [ 0.462454] ACPI: PCI: Interrupt link GSIA configured for IRQ 161372lb # [ 0.461665] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1373worker1 # [ 0.473066] usbcore: registered new interface driver hub1374worker2 # [ 0.463412] ACPI: PCI: Interrupt link GSIB configured for IRQ 171375client # [ 0.462679] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1376worker1 # [ 0.473779] usbcore: registered new device driver usb1377worker2 # [ 0.464410] ACPI: PCI: Interrupt link GSIC configured for IRQ 181378lb # [ 0.462829] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1379worker1 # [ 0.474980] NetLabel: Initializing1380worker2 # [ 0.465145] ACPI: PCI: Interrupt link GSID configured for IRQ 191381worker1 # [ 0.475541] NetLabel: domain hash size = 1281382worker2 # [ 0.465436] ACPI: PCI: Interrupt link GSIE configured for IRQ 201383lb # [ 0.464469] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1384worker1 # [ 0.476103] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1385worker2 # [ 0.466391] ACPI: PCI: Interrupt link GSIF configured for IRQ 211386client # [ 0.466030] ACPI: PCI: Interrupt link LNKA configured for IRQ 101387worker1 # [ 0.477075] NetLabel: unlabeled traffic allowed by default1388worker2 # [ 0.467165] ACPI: PCI: Interrupt link GSIG configured for IRQ 221389worker1 # [ 0.477813] PCI: Using ACPI for IRQ routing1390client # [ 0.467061] ACPI: PCI: Interrupt link LNKB configured for IRQ 101391worker2 # [ 0.467432] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392client # [ 0.468048] ACPI: PCI: Interrupt link LNKC configured for IRQ 111393lb # [ 0.467895] ACPI: PCI: Interrupt link LNKA configured for IRQ 101394client # [ 0.469054] ACPI: PCI: Interrupt link LNKD configured for IRQ 111395worker2 # [ 0.469400] iommu: Default domain type: Translated1396lb # [ 0.468881] ACPI: PCI: Interrupt link LNKB configured for IRQ 101397client # [ 0.470038] ACPI: PCI: Interrupt link LNKE configured for IRQ 101398worker2 # [ 0.470087] iommu: DMA domain TLB invalidation policy: lazy mode1399lb # [ 0.469881] ACPI: PCI: Interrupt link LNKC configured for IRQ 111400worker2 # [ 0.470713] ACPI: bus type USB registered1401client # [ 0.471037] ACPI: PCI: Interrupt link LNKF configured for IRQ 101402lb # [ 0.470877] ACPI: PCI: Interrupt link LNKD configured for IRQ 111403worker2 # [ 0.471446] usbcore: registered new interface driver usbfs1404client # [ 0.472042] ACPI: PCI: Interrupt link LNKG configured for IRQ 111405lb # [ 0.471874] ACPI: PCI: Interrupt link LNKE configured for IRQ 101406worker2 # [ 0.472391] usbcore: registered new interface driver hub1407lb # [ 0.472878] ACPI: PCI: Interrupt link LNKF configured for IRQ 101408client # [ 0.473047] ACPI: PCI: Interrupt link LNKH configured for IRQ 111409worker2 # [ 0.473104] usbcore: registered new device driver usb1410client # [ 0.473982] ACPI: PCI: Interrupt link GSIA configured for IRQ 161411lb # [ 0.473877] ACPI: PCI: Interrupt link LNKG configured for IRQ 111412worker2 # [ 0.474208] NetLabel: Initializing1413client # [ 0.474956] ACPI: PCI: Interrupt link GSIB configured for IRQ 171414worker2 # [ 0.474422] NetLabel: domain hash size = 1281415lb # [ 0.474902] ACPI: PCI: Interrupt link LNKH configured for IRQ 111416client # [ 0.475954] ACPI: PCI: Interrupt link GSIC configured for IRQ 181417worker2 # [ 0.475160] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1418lb # [ 0.475816] ACPI: PCI: Interrupt link GSIA configured for IRQ 161419client # [ 0.476923] ACPI: PCI: Interrupt link GSID configured for IRQ 191420worker2 # [ 0.475476] NetLabel: unlabeled traffic allowed by default1421lb # [ 0.476791] ACPI: PCI: Interrupt link GSIB configured for IRQ 171422worker2 # [ 0.476390] PCI: Using ACPI for IRQ routing1423client # [ 0.477699] ACPI: PCI: Interrupt link GSIE configured for IRQ 201424lb # [ 0.477754] ACPI: PCI: Interrupt link GSIC configured for IRQ 181425client # [ 0.477954] ACPI: PCI: Interrupt link GSIF configured for IRQ 211426lb # [ 0.478513] ACPI: PCI: Interrupt link GSID configured for IRQ 191427client # [ 0.478953] ACPI: PCI: Interrupt link GSIG configured for IRQ 221428lb # [ 0.478793] ACPI: PCI: Interrupt link GSIE configured for IRQ 201429client # [ 0.479954] ACPI: PCI: Interrupt link GSIH configured for IRQ 231430lb # [ 0.479781] ACPI: PCI: Interrupt link GSIF configured for IRQ 211431lb # [ 0.480759] ACPI: PCI: Interrupt link GSIG configured for IRQ 221432client # [ 0.482003] iommu: Default domain type: Translated1433lb # [ 0.481565] ACPI: PCI: Interrupt link GSIH configured for IRQ 231434client # [ 0.482850] iommu: DMA domain TLB invalidation policy: lazy mode1435client # [ 0.484054] ACPI: bus type USB registered1436lb # [ 0.483645] iommu: Default domain type: Translated1437client # [ 0.484887] usbcore: registered new interface driver usbfs1438lb # [ 0.484384] iommu: DMA domain TLB invalidation policy: lazy mode1439client # [ 0.485672] usbcore: registered new interface driver hub1440lb # [ 0.485064] ACPI: bus type USB registered1441client # [ 0.485950] usbcore: registered new device driver usb1442lb # [ 0.485821] usbcore: registered new interface driver usbfs1443lb # [ 0.486726] usbcore: registered new interface driver hub1444client # [ 0.487737] NetLabel: Initializing1445lb # [ 0.487467] usbcore: registered new device driver usb1446client # [ 0.487947] NetLabel: domain hash size = 1281447client # [ 0.488694] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1448lb # [ 0.488617] NetLabel: Initializing1449client # [ 0.488986] NetLabel: unlabeled traffic allowed by default1450lb # [ 0.489232] NetLabel: domain hash size = 1281451client # [ 0.489890] PCI: Using ACPI for IRQ routing1452lb # [ 0.489778] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453lb # [ 0.490766] NetLabel: unlabeled traffic allowed by default1454lb # [ 0.491513] PCI: Using ACPI for IRQ routing1455worker1 # [ 0.521679] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker1 # [ 0.522099] pci 0000:00:01.0: vgaarb: bridge control possible1457worker1 # [ 0.522099] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker1 # [ 0.522108] vgaarb: loaded1459worker1 # [ 0.522846] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker1 # [ 0.523105] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker1 # [ 0.529186] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.520244] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463worker2 # [ 0.520413] pci 0000:00:01.0: vgaarb: bridge control possible1464worker1 # [ 0.532648] VFS: Disk quotas dquot_6.6.01465worker2 # [ 0.520413] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1466worker2 # [ 0.520421] vgaarb: loaded1467worker1 # [ 0.533365] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468worker2 # [ 0.521175] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01469worker1 # [ 0.534772] pnp: PnP ACPI init1470worker2 # [ 0.521417] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1471worker1 # [ 0.535539] ACPI: IRQ 4 override to edge(!), high(!)1472worker1 # [ 0.536508] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1473worker1 # [ 0.538093] pnp: PnP ACPI: found 5 devices1474worker2 # [ 0.525503] clocksource: Switched to clocksource kvm-clock1475worker2 # [ 0.529045] VFS: Disk quotas dquot_6.6.01476worker2 # [ 0.529776] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1477client # [ 0.533337] pci 0000:00:01.0: vgaarb: setting as boot VGA device1478worker2 # [ 0.531180] pnp: PnP ACPI init1479worker1 # [ 0.545648] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1480client # [ 0.533938] pci 0000:00:01.0: vgaarb: bridge control possible1481worker1 # [ 0.547023] clocksource: Switched to clocksource acpi_pm1482worker2 # [ 0.532222] ACPI: IRQ 4 override to edge(!), high(!)1483worker1 # [ 0.547951] NET: Registered PF_INET protocol family1484lb # [ 0.535148] pci 0000:00:01.0: vgaarb: setting as boot VGA device1485client # [ 0.533938] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1486worker2 # [ 0.533178] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1487client # [ 0.533946] vgaarb: loaded1488lb # [ 0.535775] pci 0000:00:01.0: vgaarb: bridge control possible1489worker1 # [ 0.549076] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1490worker2 # [ 0.534693] pnp: PnP ACPI: found 5 devices1491client # [ 0.534661] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01492lb # [ 0.535775] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1493lb # [ 0.535784] vgaarb: loaded1494client # [ 0.534944] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495lb # [ 0.536528] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01496lb # [ 0.536781] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1497client # [ 0.538037] clocksource: Switched to clocksource kvm-clock1498worker2 # [ 0.542251] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1499client # [ 0.541589] VFS: Disk quotas dquot_6.6.01500worker2 # [ 0.543741] clocksource: Switched to clocksource acpi_pm1501lb # [ 0.541873] clocksource: Switched to clocksource kvm-clock1502client # [ 0.542331] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1503worker2 # [ 0.544690] NET: Registered PF_INET protocol family1504client # [ 0.543880] pnp: PnP ACPI init1505worker2 # [ 0.545848] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1506client # [ 0.544736] ACPI: IRQ 4 override to edge(!), high(!)1507lb # [ 0.545743] VFS: Disk quotas dquot_6.6.01508client # [ 0.545698] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1509lb # [ 0.546458] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1510client # [ 0.547285] pnp: PnP ACPI: found 5 devices1511lb # [ 0.547895] pnp: PnP ACPI init1512lb # [ 0.548694] ACPI: IRQ 4 override to edge(!), high(!)1513worker1 # [ 0.565564] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1514lb # [ 0.549662] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1515worker1 # [ 0.567154] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1516lb # [ 0.551210] pnp: PnP ACPI: found 5 devices1517worker1 # [ 0.568502] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1518worker1 # [ 0.569921] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1519worker1 # [ 0.571238] TCP: Hash tables configured (established 16384 bind 16384)1520client # [ 0.554878] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1521worker1 # [ 0.572366] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1522client # [ 0.556306] clocksource: Switched to clocksource acpi_pm1523worker1 # [ 0.573618] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1524client # [ 0.557278] NET: Registered PF_INET protocol family1525worker1 # [ 0.574705] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1526client # [ 0.558485] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1527worker1 # [ 0.575935] NET: Registered PF_UNIX/PF_LOCAL protocol family1528lb # [ 0.558855] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1529worker1 # [ 0.576879] NET: Registered PF_XDP protocol family1530lb # [ 0.560348] clocksource: Switched to clocksource acpi_pm1531worker2 # [ 0.562001] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1532worker1 # [ 0.577687] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1533lb # [ 0.561353] NET: Registered PF_INET protocol family1534worker1 # [ 0.578671] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535worker2 # [ 0.563711] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1536lb # [ 0.562525] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1537worker1 # [ 0.579666] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1538worker2 # [ 0.565027] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1539worker1 # [ 0.580738] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1540worker2 # [ 0.566406] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1541worker1 # [ 0.581856] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1542worker2 # [ 0.567683] TCP: Hash tables configured (established 16384 bind 16384)1543worker1 # [ 0.582978] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1544worker2 # [ 0.568863] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1545worker1 # [ 0.584683] ACPI: \_SB_.GSIA: Enabled at IRQ 161546worker2 # [ 0.570276] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1547worker2 # [ 0.571517] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker1 # [ 0.586993] ACPI: \_SB_.GSIB: Enabled at IRQ 171549worker2 # [ 0.572810] NET: Registered PF_UNIX/PF_LOCAL protocol family1550worker2 # [ 0.573829] NET: Registered PF_XDP protocol family1551worker1 # [ 0.589464] ACPI: \_SB_.GSIC: Enabled at IRQ 181552worker2 # [ 0.574728] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1553worker2 # [ 0.575737] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554worker1 # [ 0.591499] ACPI: \_SB_.GSID: Enabled at IRQ 191555worker2 # [ 0.576777] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1556client # [ 0.575004] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1557worker2 # [ 0.577846] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1558worker1 # [ 0.593315] PCI: CLS 0 bytes, default 641559client # [ 0.576747] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1560worker2 # [ 0.578973] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1561client # [ 0.577991] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1562worker1 # [ 0.594287] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1563worker2 # [ 0.580056] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1564worker1 # [ 0.595969] Trying to unpack rootfs image as initramfs...1565client # [ 0.579302] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1566worker2 # [ 0.581720] ACPI: \_SB_.GSIA: Enabled at IRQ 161567lb # [ 0.579404] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1568client # [ 0.580518] TCP: Hash tables configured (established 16384 bind 16384)1569lb # [ 0.580971] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1570client # [ 0.581738] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1571worker2 # [ 0.583936] ACPI: \_SB_.GSIB: Enabled at IRQ 171572lb # [ 0.582312] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1573client # [ 0.583060] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1574lb # [ 0.583699] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1575client # [ 0.584247] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1576worker2 # [ 0.586296] ACPI: \_SB_.GSIC: Enabled at IRQ 181577lb # [ 0.584971] TCP: Hash tables configured (established 16384 bind 16384)1578client # [ 0.585415] NET: Registered PF_UNIX/PF_LOCAL protocol family1579client # [ 0.586383] NET: Registered PF_XDP protocol family1580worker2 # [ 0.588222] ACPI: \_SB_.GSID: Enabled at IRQ 191581lb # [ 0.586137] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1582client # [ 0.587237] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1583lb # [ 0.587415] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1584worker2 # [ 0.589885] PCI: CLS 0 bytes, default 641585client # [ 0.588255] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1586lb # [ 0.588482] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1587client # [ 0.589271] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1588worker2 # [ 0.590767] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1589lb # [ 0.589738] NET: Registered PF_UNIX/PF_LOCAL protocol family1590client # [ 0.590339] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1591worker2 # [ 0.592489] Trying to unpack rootfs image as initramfs...1592lb # [ 0.590697] NET: Registered PF_XDP protocol family1593client # [ 0.591413] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1594lb # [ 0.591515] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1595client # [ 0.592506] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1596lb # [ 0.592592] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1597client # [ 0.594248] ACPI: \_SB_.GSIA: Enabled at IRQ 161598lb # [ 0.593599] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1599lb # [ 0.594737] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1600lb # [ 0.595960] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1601client # [ 0.597251] ACPI: \_SB_.GSIB: Enabled at IRQ 171602lb # [ 0.597115] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1603client # [ 0.599205] ACPI: \_SB_.GSIC: Enabled at IRQ 181604lb # [ 0.598937] ACPI: \_SB_.GSIA: Enabled at IRQ 161605client # [ 0.601133] ACPI: \_SB_.GSID: Enabled at IRQ 191606lb # [ 0.601813] ACPI: \_SB_.GSIB: Enabled at IRQ 171607client # [ 0.602874] PCI: CLS 0 bytes, default 641608lb # [ 0.603804] ACPI: \_SB_.GSIC: Enabled at IRQ 181609client # [ 0.603778] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1610lb # [ 0.605889] ACPI: \_SB_.GSID: Enabled at IRQ 191611client # [ 0.605504] Trying to unpack rootfs image as initramfs...1612lb # [ 0.607619] PCI: CLS 0 bytes, default 641613lb # [ 0.608659] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614lb # [ 0.610459] Trying to unpack rootfs image as initramfs...1615worker1 # [ 0.640158] Initialise system trusted keyrings1616worker1 # [ 0.643826] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker2 # [ 0.636149] Initialise system trusted keyrings1618worker2 # [ 0.639182] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker1 # [ 0.663057] Key type asymmetric registered1620worker1 # [ 0.665780] Asymmetric key parser 'x509' registered1621client # [ 0.649585] Initialise system trusted keyrings1622worker1 # [ 0.666615] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1623worker1 # [ 0.669921] io scheduler mq-deadline registered1624client # [ 0.653213] workingset: timestamp_bits=40 max_order=19 bucket_order=01625worker1 # [ 0.670674] io scheduler kyber registered1626lb # [ 0.655307] Initialise system trusted keyrings1627worker1 # [ 0.674790] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1628worker2 # [ 0.660521] Key type asymmetric registered1629lb # [ 0.658904] workingset: timestamp_bits=40 max_order=19 bucket_order=01630worker1 # [ 0.676055] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1631worker2 # [ 0.663132] Asymmetric key parser 'x509' registered1632worker2 # [ 0.664030] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1633worker1 # [ 0.681565] Linux agpgart interface v0.1031634worker2 # [ 0.667224] io scheduler mq-deadline registered1635worker1 # [ 0.682386] ACPI: bus type drm_connector registered1636worker2 # [ 0.668012] io scheduler kyber registered1637worker1 # [ 0.685191] usbcore: registered new interface driver usbserial_generic1638worker1 # [ 0.686359] usbserial: USB Serial support registered for generic1639worker2 # [ 0.672141] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1640worker2 # [ 0.673483] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1641client # [ 0.674370] Key type asymmetric registered1642worker1 # [ 0.690782] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1643client # [ 0.675090] Asymmetric key parser 'x509' registered1644worker1 # [ 0.690782] Please enable it if your BIOS has the CPPC option.1645client # [ 0.675932] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1646worker1 # [ 0.693044] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1647worker2 # [ 0.678887] Linux agpgart interface v0.1031648worker2 # [ 0.679655] ACPI: bus type drm_connector registered1649worker1 # [ 0.695914] drop_monitor: Initializing network drop monitor service1650lb # [ 0.680040] Key type asymmetric registered1651worker1 # [ 0.697137] NET: Registered PF_INET6 protocol family1652lb # [ 0.680785] Asymmetric key parser 'x509' registered1653worker2 # [ 0.682545] usbcore: registered new interface driver usbserial_generic1654client # [ 0.681268] io scheduler mq-deadline registered1655client # [ 0.682031] io scheduler kyber registered1656worker2 # [ 0.683646] usbserial: USB Serial support registered for generic1657lb # [ 0.681648] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1658worker1 # [ 0.700213] Segment Routing with IPv61659client # [ 0.684709] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1660worker2 # [ 0.686132] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1661worker2 # [ 0.686132] Please enable it if your BIOS has the CPPC option.1662worker1 # [ 0.702800] In-situ OAM (IOAM) with IPv61663client # [ 0.685955] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1664worker1 # [ 0.703826] IPI shorthand broadcast: enabled1665worker2 # [ 0.688435] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1666lb # [ 0.686963] io scheduler mq-deadline registered1667lb # [ 0.687759] io scheduler kyber registered1668client # [ 0.691452] Linux agpgart interface v0.1031669worker2 # [ 0.693259] drop_monitor: Initializing network drop monitor service1670lb # [ 0.691471] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1671client # [ 0.692287] ACPI: bus type drm_connector registered1672worker2 # [ 0.694441] NET: Registered PF_INET6 protocol family1673lb # [ 0.692704] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1674client # [ 0.695531] usbcore: registered new interface driver usbserial_generic1675worker2 # [ 0.697592] Segment Routing with IPv61676worker1 # [ 0.712022] sched_clock: Marking stable (579016882, 132746766)->(780029209, -68265561)1677client # [ 0.696604] usbserial: USB Serial support registered for generic1678lb # [ 0.696635] Linux agpgart interface v0.1031679lb # [ 0.697393] ACPI: bus type drm_connector registered1680worker2 # [ 0.700173] In-situ OAM (IOAM) with IPv61681worker1 # [ 0.716016] registered taskstats version 11682worker2 # [ 0.701262] IPI shorthand broadcast: enabled1683worker1 # [ 0.716966] Loading compiled-in X.509 certificates1684lb # [ 0.700281] usbcore: registered new interface driver usbserial_generic1685client # [ 0.700181] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1686client # [ 0.700181] Please enable it if your BIOS has the CPPC option.1687lb # [ 0.701365] usbserial: USB Serial support registered for generic1688client # [ 0.702436] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1689lb # [ 0.703881] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1690client # [ 0.705322] drop_monitor: Initializing network drop monitor service1691lb # [ 0.703881] Please enable it if your BIOS has the CPPC option.1692client # [ 0.706580] NET: Registered PF_INET6 protocol family1693lb # [ 0.706184] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1694worker2 # [ 0.709307] sched_clock: Marking stable (576014162, 133099365)->(783664561, -74551034)1695client # [ 0.709670] Segment Routing with IPv61696lb # [ 0.709995] drop_monitor: Initializing network drop monitor service1697lb # [ 0.711213] NET: Registered PF_INET6 protocol family1698client # [ 0.712191] In-situ OAM (IOAM) with IPv61699worker2 # [ 0.714323] registered taskstats version 11700client # [ 0.713281] IPI shorthand broadcast: enabled1701worker2 # [ 0.715342] Loading compiled-in X.509 certificates1702lb # [ 0.716314] Segment Routing with IPv61703lb # [ 0.717061] In-situ OAM (IOAM) with IPv61704worker1 # [ 0.734779] Demotion targets for Node 0: null1705lb # [ 0.720158] IPI shorthand broadcast: enabled1706worker1 # [ 0.737805] Key type .fscrypt registered1707worker1 # [ 0.738504] Key type fscrypt-provisioning registered1708client # [ 0.721664] sched_clock: Marking stable (589014399, 132138579)->(793076694, -71923716)1709worker1 # [ 0.739463] ima: No TPM chip found, activating TPM-bypass!1710worker1 # [ 0.741778] ima: Allocated hash algorithm: sha11711worker1 # [ 0.742582] ima: No architecture policies found1712client # [ 0.728369] registered taskstats version 11713lb # [ 0.727608] sched_clock: Marking stable (593015261, 133833561)->(799847990, -72999168)1714client # [ 0.729350] Loading compiled-in X.509 certificates1715worker2 # [ 0.731654] Demotion targets for Node 0: null1716worker1 # [ 0.746775] PM: Magic number: 10:383:8441717worker1 # [ 0.747490] PM: hash matches drivers/base/power/main.c:11171718lb # [ 0.731075] registered taskstats version 11719lb # [ 0.732106] Loading compiled-in X.509 certificates1720worker2 # [ 0.735168] Key type .fscrypt registered1721worker2 # [ 0.735871] Key type fscrypt-provisioning registered1722worker1 # [ 0.751419] RAS: Correctable Errors collector initialized.1723worker2 # [ 0.736828] ima: No TPM chip found, activating TPM-bypass!1724worker2 # [ 0.739125] ima: Allocated hash algorithm: sha11725worker2 # [ 0.739954] ima: No architecture policies found1726worker2 # [ 0.742131] PM: Magic number: 10:383:8441727worker2 # [ 0.742878] PM: hash matches drivers/base/power/main.c:11171728worker1 # [ 0.760173] clk: Disabling unused clocks1729worker2 # [ 0.746743] RAS: Correctable Errors collector initialized.1730client # [ 0.745761] Demotion targets for Node 0: null1731worker1 # [ 0.762775] PM: genpd: Disabling unused power domains1732client # [ 0.749200] Key type .fscrypt registered1733client # [ 0.749897] Key type fscrypt-provisioning registered1734client # [ 0.750873] ima: No TPM chip found, activating TPM-bypass!1735lb # [ 0.750856] Demotion targets for Node 0: null1736client # [ 0.753166] ima: Allocated hash algorithm: sha11737worker2 # [ 0.755421] clk: Disabling unused clocks1738client # [ 0.753963] ima: No architecture policies found1739lb # [ 0.753900] Key type .fscrypt registered1740lb # [ 0.754614] Key type fscrypt-provisioning registered1741lb # [ 0.755584] ima: No TPM chip found, activating TPM-bypass!1742client # [ 0.758171] PM: Magic number: 10:383:8441743lb # [ 0.757862] ima: Allocated hash algorithm: sha11744worker2 # [ 0.760139] PM: genpd: Disabling unused power domains1745lb # [ 0.758689] ima: No architecture policies found1746client # [ 0.758855] PM: hash matches drivers/base/power/main.c:11171747lb # [ 0.760865] PM: Magic number: 10:383:8441748lb # [ 0.761631] PM: hash matches drivers/base/power/main.c:11171749client # [ 0.762751] RAS: Correctable Errors collector initialized.1750lb # [ 0.765632] RAS: Correctable Errors collector initialized.1751client # [ 0.769368] clk: Disabling unused clocks1752client # [ 0.770101] PM: genpd: Disabling unused power domains1753lb # [ 0.774332] clk: Disabling unused clocks1754lb # [ 0.778874] PM: genpd: Disabling unused power domains1755worker1 # [ 0.922213] Freeing initrd memory: 29056K1756worker1 # [ 0.925488] Freeing unused decrypted memory: 2028K1757worker1 # [ 0.928194] Freeing unused kernel image (initmem) memory: 3652K1758worker1 # [ 0.929343] Write protecting the kernel read-only data: 32768k1759worker1 # [ 0.931252] Freeing unused kernel image (text/rodata gap) memory: 1184K1760worker1 # [ 0.932816] Freeing unused kernel image (rodata/data gap) memory: 720K1761worker2 # [ 0.918973] Freeing initrd memory: 29060K1762worker2 # [ 0.922133] Freeing unused decrypted memory: 2028K1763worker2 # [ 0.924802] Freeing unused kernel image (initmem) memory: 3652K1764worker2 # [ 0.925953] Write protecting the kernel read-only data: 32768k1765worker2 # [ 0.927883] Freeing unused kernel image (text/rodata gap) memory: 1184K1766worker2 # [ 0.929456] Freeing unused kernel image (rodata/data gap) memory: 720K1767client # [ 0.937691] Freeing initrd memory: 29056K1768client # [ 0.941007] Freeing unused decrypted memory: 2028K1769client # [ 0.943799] Freeing unused kernel image (initmem) memory: 3652K1770client # [ 0.944933] Write protecting the kernel read-only data: 32768k1771client # [ 0.946844] Freeing unused kernel image (text/rodata gap) memory: 1184K1772lb # [ 0.948109] Freeing initrd memory: 29064K1773client # [ 0.948437] Freeing unused kernel image (rodata/data gap) memory: 720K1774lb # [ 0.951345] Freeing unused decrypted memory: 2028K1775lb # [ 0.954073] Freeing unused kernel image (initmem) memory: 3652K1776lb # [ 0.955207] Write protecting the kernel read-only data: 32768k1777lb # [ 0.957126] Freeing unused kernel image (text/rodata gap) memory: 1184K1778lb # [ 0.958649] Freeing unused kernel image (rodata/data gap) memory: 720K1779worker1 # [ 0.983886] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker1 # [ 0.984986] Run /init as init process1781worker2 # [ 0.980351] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782worker1 # [ 0.995629] systemd[1]: Inserted module 'autofs4'1783worker2 # [ 0.981441] Run /init as init process1784worker2 # [ 0.991975] systemd[1]: Inserted module 'autofs4'1785worker1 # [ 1.015462] fuse: init (API version 7.45)1786client # [ 0.999447] x86/mm: Checked W+X mappings: passed, no W+X pages found.1787client # [ 1.000583] Run /init as init process1788worker1 # [ 1.022894] ACPI: \_SB_.GSIG: Enabled at IRQ 221789worker1 # [ 1.025207] ACPI: \_SB_.GSIH: Enabled at IRQ 231790worker2 # [ 1.011879] fuse: init (API version 7.45)1791lb # [ 1.009652] x86/mm: Checked W+X mappings: passed, no W+X pages found.1792lb # [ 1.010749] Run /init as init process1793client # [ 1.011426] systemd[1]: Inserted module 'autofs4'1794worker1 # [ 1.028304] ACPI: \_SB_.GSIE: Enabled at IRQ 201795worker1 # [ 1.030179] ACPI: \_SB_.GSIF: Enabled at IRQ 211796worker2 # [ 1.019285] ACPI: \_SB_.GSIG: Enabled at IRQ 221797worker1 # [ 1.035383] virtiofs virtio5: discovered new tag: nix-store1798worker2 # [ 1.021631] ACPI: \_SB_.GSIH: Enabled at IRQ 231799worker1 # [ 1.037000] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1800lb # [ 1.021692] systemd[1]: Inserted module 'autofs4'1801worker2 # [ 1.024593] ACPI: \_SB_.GSIE: Enabled at IRQ 201802worker2 # [ 1.026598] ACPI: \_SB_.GSIF: Enabled at IRQ 211803worker1 # [ 1.044067] virtiofs virtio6: discovered new tag: shared1804worker1 # [ 1.045740] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1805worker2 # [ 1.031721] virtiofs virtio5: discovered new tag: nix-store1806client # [ 1.031325] fuse: init (API version 7.45)1807worker2 # [ 1.033387] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1808worker1 # [ 1.048995] virtiofs virtio7: discovered new tag: xchg1809worker1 # [ 1.050442] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1810client # [ 1.038740] ACPI: \_SB_.GSIG: Enabled at IRQ 221811worker2 # [ 1.040322] virtiofs virtio6: discovered new tag: shared1812worker2 # [ 1.041914] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1813client # [ 1.041523] ACPI: \_SB_.GSIH: Enabled at IRQ 231814lb # [ 1.041501] fuse: init (API version 7.45)1815worker2 # [ 1.045017] virtiofs virtio7: discovered new tag: xchg1816client # [ 1.044689] ACPI: \_SB_.GSIE: Enabled at IRQ 201817worker2 # [ 1.046527] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1818client # [ 1.046854] ACPI: \_SB_.GSIF: Enabled at IRQ 211819lb # [ 1.049024] ACPI: \_SB_.GSIG: Enabled at IRQ 221820lb # [ 1.051784] ACPI: \_SB_.GSIH: Enabled at IRQ 231821client # [ 1.052702] virtiofs virtio5: discovered new tag: nix-store1822worker1 # [ 1.070474] systemd[1]: Successfully made /usr/ read-only.1823client # [ 1.054305] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1824lb # [ 1.055249] ACPI: \_SB_.GSIE: Enabled at IRQ 201825lb # [ 1.057509] ACPI: \_SB_.GSIF: Enabled at IRQ 211826client # [ 1.062340] virtiofs virtio6: discovered new tag: shared1827lb # [ 1.063590] virtiofs virtio5: discovered new tag: nix-store1828client # [ 1.063832] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1829worker2 # [ 1.066973] systemd[1]: Successfully made /usr/ read-only.1830lb # [ 1.065146] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1831client # [ 1.067130] virtiofs virtio7: discovered new tag: xchg1832client # [ 1.068581] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833lb # [ 1.072970] virtiofs virtio6: discovered new tag: shared1834lb # [ 1.074551] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1835lb # [ 1.077856] virtiofs virtio7: discovered new tag: xchg1836lb # [ 1.079406] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1837client # [ 1.089907] systemd[1]: Successfully made /usr/ read-only.1838lb # [ 1.100661] systemd[1]: Successfully made /usr/ read-only.1839worker1 # [ 1.406158] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840worker1 # [ 1.410965] systemd[1]: Detected virtualization kvm.1841worker1 # [ 1.411790] systemd[1]: Detected architecture x86-64.1842worker1 # [ 1.412601] systemd[1]: Running in initrd.1843worker1 # [ 1.413638] systemd[1]: Initializing machine ID from random generator.1844worker1 # [ 1.414816] systemd[1]: Hostname set to <worker1>.1845worker2 # [ 1.402479] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1846worker2 # [ 1.407576] systemd[1]: Detected virtualization kvm.1847worker2 # [ 1.408470] systemd[1]: Detected architecture x86-64.1848worker2 # [ 1.409386] systemd[1]: Running in initrd.1849worker2 # [ 1.410488] systemd[1]: Initializing machine ID from random generator.1850worker2 # [ 1.411727] systemd[1]: Hostname set to <worker2>.1851client # [ 1.425384] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1852client # [ 1.430417] systemd[1]: Detected virtualization kvm.1853client # [ 1.431295] systemd[1]: Detected architecture x86-64.1854client # [ 1.432193] systemd[1]: Running in initrd.1855client # [ 1.433264] systemd[1]: Initializing machine ID from random generator.1856client # [ 1.434503] systemd[1]: Hostname set to <client>.1857lb # [ 1.437233] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1858lb # [ 1.449131] systemd[1]: Detected virtualization kvm.1859lb # [ 1.451193] systemd[1]: Detected architecture x86-64.1860lb # [ 1.453289] systemd[1]: Running in initrd.1861lb # [ 1.455645] systemd[1]: Initializing machine ID from random generator.1862lb # [ 1.458497] systemd[1]: Hostname set to <lb>.1863worker1 # [ 1.603146] systemd[1]: bpf-restrict-fs: LSM BPF program attached1864client # [ 1.621838] systemd[1]: bpf-restrict-fs: LSM BPF program attached1865worker1 # [ 1.640295] systemd[1]: Queued start job for default target Initrd Default Target.1866worker1 # [ 1.644024] systemd[1]: Created slice Slice /system/modprobe.1867worker1 # [ 1.645196] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1868worker2 # [ 1.628924] systemd[1]: bpf-restrict-fs: LSM BPF program attached1869worker1 # [ 1.646562] systemd[1]: Expecting device /dev/disk/by-label/nixos...1870worker1 # [ 1.647685] systemd[1]: Reached target Path Units.1871worker1 # [ 1.648524] systemd[1]: Reached target Slice Units.1872worker1 # [ 1.649395] systemd[1]: Reached target Swaps.1873worker1 # [ 1.650187] systemd[1]: Reached target Timer Units.1874worker1 # [ 1.651143] systemd[1]: Listening on D-Bus System Message Bus Socket.1875worker1 # [ 1.652326] systemd[1]: Listening on Journal Socket (/dev/log).1876worker1 # [ 1.653457] systemd[1]: Listening on Journal Sockets.1877worker1 # [ 1.654441] systemd[1]: Listening on udev Control Socket.1878worker1 # [ 1.655455] systemd[1]: Listening on udev Kernel Socket.1879worker1 # [ 1.656397] systemd[1]: Reached target Socket Units.1880worker1 # [ 1.658149] systemd[1]: Starting Create List of Static Device Nodes...1881worker1 # [ 1.661830] systemd[1]: Starting Load Kernel Module configfs...1882client # [ 1.659572] systemd[1]: Queued start job for default target Initrd Default Target.1883client # [ 1.663479] systemd[1]: Created slice Slice /system/modprobe.1884client # [ 1.664701] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1885worker1 # [ 1.682361] systemd[1]: Starting Journal Service...1886client # [ 1.666163] systemd[1]: Expecting device /dev/disk/by-label/nixos...1887client # [ 1.667275] systemd[1]: Reached target Path Units.1888client # [ 1.668163] systemd[1]: Reached target Slice Units.1889client # [ 1.669030] systemd[1]: Reached target Swaps.1890client # [ 1.669860] systemd[1]: Reached target Timer Units.1891client # [ 1.670850] systemd[1]: Listening on D-Bus System Message Bus Socket.1892client # [ 1.672072] systemd[1]: Listening on Journal Socket (/dev/log).1893client # [ 1.673310] systemd[1]: Listening on Journal Sockets.1894worker1 # [ 1.690936] systemd[1]: Starting Load Kernel Modules...1895client # [ 1.674321] systemd[1]: Listening on udev Control Socket.1896client # [ 1.675357] systemd[1]: Listening on udev Kernel Socket.1897client # [ 1.676340] systemd[1]: Reached target Socket Units.1898worker1 # [ 1.693845] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1899client # [ 1.678125] systemd[1]: Starting Create List of Static Device Nodes...1900lb # [ 1.677126] systemd[1]: bpf-restrict-fs: LSM BPF program attached1901worker2 # [ 1.683299] systemd[1]: Queued start job for default target Initrd Default Target.1902worker1 # [ 1.701473] systemd[1]: Starting Coldplug All udev Devices...1903worker2 # [ 1.687502] systemd[1]: Created slice Slice /system/modprobe.1904worker2 # [ 1.688826] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1905worker2 # [ 1.690427] systemd[1]: Expecting device /dev/disk/by-label/nixos...1906worker2 # [ 1.691644] systemd[1]: Reached target Path Units.1907worker2 # [ 1.692614] systemd[1]: Reached target Slice Units.1908worker2 # [ 1.693582] systemd[1]: Reached target Swaps.1909worker2 # [ 1.694495] systemd[1]: Reached target Timer Units.1910client # [ 1.681984] systemd[1]: Starting Load Kernel Module configfs...1911worker2 # [ 1.695550] systemd[1]: Listening on D-Bus System Message Bus Socket.1912worker2 # [ 1.696850] systemd[1]: Listening on Journal Socket (/dev/log).1913worker2 # [ 1.698155] systemd[1]: Listening on Journal Sockets.1914worker1 # [ 1.713830] systemd[1]: Finished Create List of Static Device Nodes.1915worker2 # [ 1.699318] systemd[1]: Listening on udev Control Socket.1916worker2 # [ 1.700421] systemd[1]: Listening on udev Kernel Socket.1917worker2 # [ 1.701488] systemd[1]: Reached target Socket Units.1918worker2 # [ 1.703459] systemd[1]: Starting Create List of Static Device Nodes...1919client # [ 1.702762] systemd[1]: Starting Journal Service...1920worker1 # [ 1.719663] systemd[1]: modprobe@configfs.service: Deactivated successfully.1921worker1 # [ 1.726116] systemd[1]: Finished Load Kernel Module configfs.1922client # [ 1.713235] systemd[1]: Starting Load Kernel Modules...1923worker1 # [ 1.730170] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1924worker1 # [ 1.732377] systemd-journald[66]: Collecting audit messages is disabled.1925client # [ 1.715915] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1926worker2 # [ 1.708193] systemd[1]: Starting Load Kernel Module configfs...1927client # [ 1.721352] systemd[1]: Starting Coldplug All udev Devices...1928worker1 # [ 1.743830] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1929worker2 # [ 1.731388] systemd[1]: Starting Journal Service...1930lb # [ 1.729933] systemd[1]: Queued start job for default target Initrd Default Target.1931lb # [ 1.733151] systemd[1]: Created slice Slice /system/modprobe.1932client # [ 1.734543] systemd[1]: Finished Create List of Static Device Nodes.1933lb # [ 1.734352] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1934worker1 # [ 1.750666] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1935lb # [ 1.735714] systemd[1]: Expecting device /dev/disk/by-label/nixos...1936lb # [ 1.736878] systemd[1]: Reached target Path Units.1937lb # [ 1.737746] systemd[1]: Reached target Slice Units.1938lb # [ 1.738666] systemd[1]: Reached target Swaps.1939client # [ 1.738747] systemd[1]: modprobe@configfs.service: Deactivated successfully.1940lb # [ 1.739539] systemd[1]: Reached target Timer Units.1941worker2 # [ 1.742265] systemd[1]: Starting Load Kernel Modules...1942lb # [ 1.740500] systemd[1]: Listening on D-Bus System Message Bus Socket.1943lb # [ 1.741742] systemd[1]: Listening on Journal Socket (/dev/log).1944lb # [ 1.742956] systemd[1]: Listening on Journal Sockets.1945lb # [ 1.743980] systemd[1]: Listening on udev Control Socket.1946worker2 # [ 1.745249] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1947lb # [ 1.745001] systemd[1]: Listening on udev Kernel Socket.1948lb # [ 1.745991] systemd[1]: Reached target Socket Units.1949worker1 # [ 1.763083] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1950client # [ 1.747779] systemd[1]: Finished Load Kernel Module configfs.1951lb # [ 1.747738] systemd[1]: Starting Create List of Static Device Nodes...1952worker2 # [ 1.749846] systemd[1]: Starting Coldplug All udev Devices...1953client # [ 1.752625] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1954client # [ 1.758865] systemd-journald[66]: Collecting audit messages is disabled.1955worker2 # [ 1.763723] systemd[1]: Finished Create List of Static Device Nodes.1956lb # [ 1.752927] systemd[1]: Starting Load Kernel Module configfs...1957client # [ 1.766223] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1958worker2 # [ 1.768779] systemd[1]: modprobe@configfs.service: Deactivated successfully.1959worker1 # [ 1.784310] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1960lb # [ 1.773960] systemd[1]: Starting Journal Service...1961client # [ 1.774951] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1962worker1 # [ 1.792954] systemd[1]: Starting Create Static Device Nodes in /dev...1963worker2 # [ 1.778136] systemd[1]: Finished Load Kernel Module configfs.1964client # [ 1.783340] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1965worker2 # [ 1.784683] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1966lb # [ 1.783929] systemd[1]: Starting Load Kernel Modules...1967lb # [ 1.786571] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1968worker1 # [ 1.804303] systemd[1]: Finished Load Kernel Modules.1969worker2 # [ 1.789426] systemd-journald[66]: Collecting audit messages is disabled.1970lb # [ 1.792549] systemd[1]: Starting Coldplug All udev Devices...1971worker1 # [ 1.813360] systemd[1]: Starting Apply Kernel Variables...1972worker2 # [ 1.799183] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1973lb # [ 1.805910] systemd[1]: Finished Create List of Static Device Nodes.1974worker2 # [ 1.807357] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1975client # [ 1.806741] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976lb # [ 1.812472] systemd[1]: modprobe@configfs.service: Deactivated successfully.1977worker2 # [ 1.815177] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1978worker1 # [ 1.831605] systemd[1]: Finished Create Static Device Nodes in /dev.1979client # [ 1.816347] systemd[1]: Starting Create Static Device Nodes in /dev...1980lb # [ 1.819253] systemd[1]: Finished Load Kernel Module configfs.1981worker1 # [ 1.837966] systemd[1]: Reached target Preparation for Local File Systems.1982worker1 # [ 1.843846] systemd[1]: Reached target Local File Systems.1983lb # [ 1.826303] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1984client # [ 1.827696] systemd[1]: Finished Load Kernel Modules.1985lb # [ 1.828258] systemd-journald[66]: Collecting audit messages is disabled.1986worker1 # [ 1.850944] systemd[1]: Starting Rule-based Manager for Device Events and Files...1987worker2 # [ 1.836672] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1988worker1 # [ 1.715981] systemd-modules-load[67]: Inserted module 'dm_mod'1989client # [ 1.836349] systemd[1]: Starting Apply Kernel Variables...1990worker1 # [ 1.720717] systemd-modules-load[67]: Inserted module 'virtio_balloon'1991lb # [ 1.836916] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1992worker1 # [ 1.721709] systemd-modules-load[67]: Inserted module 'virtio_gpu'1993worker2 # [ 1.845319] systemd[1]: Starting Create Static Device Nodes in /dev...1994worker1 # [ 1.860953] systemd[1]: Started Journal Service.1995lb # [ 1.849395] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1996worker1 # [ 1.734890] systemd[1]: Finished Apply Kernel Variables.1997worker2 # [ 1.857185] systemd[1]: Finished Load Kernel Modules.1998client # [ 1.857226] systemd[1]: Finished Create Static Device Nodes in /dev.1999lb # [ 1.858027] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev2000client # [ 1.862652] systemd[1]: Reached target Preparation for Local File Systems.
loading 2,614 hidden lines…
4615client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.19 seconds)4616client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4617client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4618lb: must succeed: curl -sf localhost:9901/clusters4619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4620lb: must succeed: curl -sf localhost:9901/clusters4621lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4622worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4623worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4624(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.50 seconds)4625subtest: outputs read back through any worker carry the cache signature4626client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24627client # don't know how to build these paths:4628client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4629worker1 # [ 61.226408] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466164630worker2 # [ 61.214938] nix-grpc-daemon[800]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537244631worker1 # [ 61.251671] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04632worker1 # [ 61.258281] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14633worker1 # [ 61.262949] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164634worker1 # [ 61.272861] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164635client # copying 1 paths to 'lb:50051'...4636client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4637worker1 # [ 61.284593] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466164638worker1 # [ 61.295675] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4639worker1 # [ 61.314685] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=info event=niks3_push_started pid=9234640worker1 # [ 61.440284] nix-grpc-daemon[923]: time=2026-09-25T16:50:40.761Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4641worker1 # [ 61.531504] nix-daemon[864]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4642worker1 # [ 61.543860] nix-daemon[864]: reaped child process 939, status = succeeded4643lb # [ 61.535769] niks3-server[957]: 2026/09/25 16:50:40 INFO Received push request method=POST path=/api/pushes4644worker1 # [ 61.559672] nix-grpc-daemon[923]: time=2026-09-25T16:50:40.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4645worker1 # [ 61.561183] nix-grpc-daemon[923]: time=2026-09-25T16:50:40.883Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4646lb # [ 61.569758] niks3-server[957]: 2026/09/25 16:50:40 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4647lb # [ 61.576220] niks3-server[957]: 2026/09/25 16:50:40 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4648lb # [ 61.581101] niks3-server[957]: 2026/09/25 16:50:40 INFO Signed narinfos id=9 count=14649worker1 # [ 61.602040] nix-grpc-daemon[923]: time=2026-09-25T16:50:40.924Z level=INFO msg="Uploading 1 narinfos"4650lb # [ 61.598351] niks3-server[957]: 2026/09/25 16:50:40 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4651lb # [ 61.602498] niks3-server[957]: 2026/09/25 16:50:40 INFO Received complete push request method=POST path=/api/pushes/9/complete4652worker1 # [ 61.630571] nix-grpc-daemon[923]: time=2026-09-25T16:50:40.953Z level=INFO msg="Upload complete. (187ms)"4653lb # [ 61.613520] niks3-server[957]: 2026/09/25 16:50:40 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4654worker1 # [ 61.634244] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=5044655worker1 # [ 61.639166] nix-grpc-daemon[511]: ts=2026-09-25T16:50:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164656worker1 # [ 61.648596] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4657worker1 # [ 61.653177] nix-daemon[864]: reaped child process 921, status = succeeded4658worker1 # [ 61.658044] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4659client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4660client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4661client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4662client # farm-a> LOG-farm-a-sig4663worker1 # [ 62.901777] nix-daemon[864]: reaped child process 943, status = succeeded4664worker1 # [ 62.979261] nix-daemon[864]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4665worker1 # [ 62.991903] nix-daemon[864]: reaped child process 962, status = succeeded4666lb # [ 62.981347] niks3-server[957]: 2026/09/25 16:50:42 INFO Received push request method=POST path=/api/pushes4667worker1 # [ 63.007509] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.329Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4668worker1 # [ 63.008949] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.331Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4669lb # [ 63.016992] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4670lb # [ 63.031360] niks3-server[957]: 2026/09/25 16:50:42 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4671lb # [ 63.037568] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4672lb # [ 63.043320] niks3-server[957]: 2026/09/25 16:50:42 INFO Signed narinfos id=10 count=14673worker1 # [ 63.064876] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.387Z level=INFO msg="Uploading 1 narinfos"4674lb # [ 63.054401] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4675lb # [ 63.062340] niks3-server[957]: 2026/09/25 16:50:42 INFO Received complete push request method=POST path=/api/pushes/10/complete4676worker1 # [ 63.085562] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.408Z level=INFO msg="Upload complete. (219ms)"4677lb # [ 63.070740] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4678worker1 # [ 63.094415] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04679worker1 # [ 63.105580] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04680worker1 # [ 63.110212] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14681worker1 # [ 63.116184] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164682worker1 # [ 63.126845] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164683worker1 # [ 63.129119] nix-daemon[864]: reaped child process 941, status = succeeded4684worker1 # [ 63.163151] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.485Z level=INFO msg="All 1 paths already cached"4685client # copying 1 paths to 'lb:50051'...4686client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4687worker1 # [ 63.171986] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466164688worker1 # [ 63.180965] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4689worker1 # [ 63.276080] nix-daemon[864]: accepted connection from pid 968, user nix-grpc-daemon (trusted)4690worker1 # [ 63.289893] nix-daemon[864]: reaped child process 971, status = succeeded4691lb # [ 63.278355] niks3-server[957]: 2026/09/25 16:50:42 INFO Received push request method=POST path=/api/pushes4692worker1 # [ 63.304914] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.627Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4693worker1 # [ 63.306412] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.629Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4694lb # [ 63.307366] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4695lb # [ 63.319347] niks3-server[957]: 2026/09/25 16:50:42 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4696worker1 # [ 63.341851] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.664Z level=INFO msg="Uploading 1 narinfos"4697lb # [ 63.323938] niks3-server[957]: 2026/09/25 16:50:42 INFO Signed narinfos id=11 count=14698lb # [ 63.333206] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4699lb # [ 63.344740] niks3-server[957]: 2026/09/25 16:50:42 INFO Received complete push request method=POST path=/api/pushes/11/complete4700worker1 # [ 63.371569] nix-grpc-daemon[923]: time=2026-09-25T16:50:42.694Z level=INFO msg="Upload complete. (178ms)"4701lb # [ 63.354724] niks3-server[957]: 2026/09/25 16:50:42 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4702worker1 # [ 63.375296] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=6244703worker1 # [ 63.380481] nix-grpc-daemon[511]: ts=2026-09-25T16:50:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164704worker1 # [ 63.391932] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4705worker1 # [ 63.398640] nix-daemon[864]: reaped child process 966, status = succeeded4706worker1 # [ 63.403645] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4707client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4708client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4709client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4710client # farm-b> LOG-farm-b-sig4711worker1 # [ 64.018545] nix-daemon[864]: reaped child process 975, status = succeeded4712worker1 # [ 64.092525] nix-daemon[864]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4713worker1 # [ 64.106285] nix-daemon[864]: reaped child process 994, status = succeeded4714lb # [ 64.094780] niks3-server[957]: 2026/09/25 16:50:43 INFO Received push request method=POST path=/api/pushes4715worker1 # [ 64.121894] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.444Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4716worker1 # [ 64.123378] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.446Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4717lb # [ 64.133373] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4718lb # [ 64.137650] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4719lb # [ 64.145915] niks3-server[957]: 2026/09/25 16:50:43 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4720worker1 # [ 64.169362] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.491Z level=INFO msg="Uploading 1 narinfos"4721lb # [ 64.150980] niks3-server[957]: 2026/09/25 16:50:43 INFO Signed narinfos id=12 count=14722lb # [ 64.156797] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4723lb # [ 64.163055] niks3-server[957]: 2026/09/25 16:50:43 INFO Received complete push request method=POST path=/api/pushes/12/complete4724worker1 # [ 64.187574] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.510Z level=INFO msg="Upload complete. (178ms)"4725lb # [ 64.172681] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4726worker1 # [ 64.192439] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04727worker1 # [ 64.203631] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04728worker1 # [ 64.208439] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14729worker1 # [ 64.212037] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164730worker1 # [ 64.217137] nix-daemon[864]: reaped child process 973, status = succeeded4731worker1 # [ 64.218515] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164732worker1 # [ 64.224613] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.547Z level=INFO msg="All 2 paths already cached"4733client # copying 1 paths to 'lb:50051'...4734worker1 # [ 64.230952] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466164735client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4736worker1 # [ 64.242773] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4737worker1 # [ 64.327117] nix-daemon[864]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4738worker1 # [ 64.342722] nix-daemon[864]: reaped child process 1003, status = succeeded4739lb # [ 64.328874] niks3-server[957]: 2026/09/25 16:50:43 INFO Received push request method=POST path=/api/pushes4740worker1 # [ 64.357608] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.680Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4741worker1 # [ 64.359195] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.681Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4742lb # [ 64.364428] niks3-server[957]: 2026/09/25 16:50:43 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4743worker1 # [ 64.387125] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.709Z level=INFO msg="Uploading 1 narinfos"4744lb # [ 64.369090] niks3-server[957]: 2026/09/25 16:50:43 INFO Signed narinfos id=13 count=14745lb # [ 64.371447] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4746lb # [ 64.383098] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4747lb # [ 64.386562] niks3-server[957]: 2026/09/25 16:50:43 INFO Received complete push request method=POST path=/api/pushes/13/complete4748worker1 # [ 64.409141] nix-grpc-daemon[923]: time=2026-09-25T16:50:43.731Z level=INFO msg="Upload complete. (161ms)"4749lb # [ 64.392466] niks3-server[957]: 2026/09/25 16:50:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4750worker1 # [ 64.412725] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=7524751worker1 # [ 64.417439] nix-grpc-daemon[511]: ts=2026-09-25T16:50:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164752worker1 # [ 64.425277] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4753worker1 # [ 64.430868] nix-daemon[864]: reaped child process 998, status = succeeded4754worker1 # [ 64.435215] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4755client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4756client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4757client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4758client # farm-top> LOG-farm-top-sig4759worker1 # [ 65.478527] nix-daemon[864]: reaped child process 1007, status = succeeded4760worker1 # [ 65.560988] nix-daemon[864]: accepted connection from pid 1023, user nix-grpc-daemon (trusted)4761worker1 # [ 65.573045] nix-daemon[864]: reaped child process 1026, status = succeeded4762lb # [ 65.562351] niks3-server[957]: 2026/09/25 16:50:44 INFO Received push request method=POST path=/api/pushes4763worker1 # [ 65.588550] nix-grpc-daemon[923]: time=2026-09-25T16:50:44.910Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4764worker1 # [ 65.590173] nix-grpc-daemon[923]: time=2026-09-25T16:50:44.912Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4765lb # [ 65.594330] niks3-server[957]: 2026/09/25 16:50:44 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4766lb # [ 65.609989] niks3-server[957]: 2026/09/25 16:50:44 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4767lb # [ 65.615784] niks3-server[957]: 2026/09/25 16:50:44 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4768lb # [ 65.623371] niks3-server[957]: 2026/09/25 16:50:44 INFO Signed narinfos id=14 count=14769worker1 # [ 65.643890] nix-grpc-daemon[923]: time=2026-09-25T16:50:44.966Z level=INFO msg="Uploading 1 narinfos"4770lb # [ 65.631925] niks3-server[957]: 2026/09/25 16:50:44 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4771lb # [ 65.641359] niks3-server[957]: 2026/09/25 16:50:44 INFO Received complete push request method=POST path=/api/pushes/14/complete4772lb # [ 65.650327] niks3-server[957]: 2026/09/25 16:50:44 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4773worker1 # [ 65.672439] nix-grpc-daemon[923]: time=2026-09-25T16:50:44.994Z level=INFO msg="Upload complete. (229ms)"4774worker1 # [ 65.678930] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04775worker1 # [ 65.695375] nix-daemon[864]: reaped child process 1005, status = succeeded4776client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.62 seconds)4777client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4778client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4779client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4780worker1 # [ 65.938701] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:478384781worker1 # [ 65.944338] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:478384782client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.14 seconds)4783client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4784worker2 # [ 66.066442] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:409084785client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4786client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4787client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4788client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4789worker2 # [ 66.092723] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:409084790worker2 # [ 66.099413] nix-grpc-daemon[800]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4791worker2 # [ 66.113842] nix-grpc-daemon[800]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4792worker2 # [ 66.126274] nix-grpc-daemon[800]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4793client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)4794client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4795client # finding garbage collector roots...4796client # removing stale temporary roots file "/nix/var/nix/temproots/823"4797client # removing stale temporary roots file "/nix/var/nix/temproots/856"4798client # removing stale temporary roots file "/nix/var/nix/temproots/808"4799client # removing stale temporary roots file "/nix/var/nix/temproots/900"4800client # removing stale temporary roots file "/nix/var/nix/temproots/844"4801client # removing stale temporary roots file "/nix/var/nix/temproots/877"4802client # removing stale temporary roots file "/nix/var/nix/temproots/885"4803client # removing stale temporary roots file "/nix/var/nix/temproots/864"4804client # deleting specified paths...4805client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4806client # deleting unused links...4807client # note: hard linking is currently saving 0.0 KiB4808worker2 # [ 66.409215] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537244809worker1 # [ 66.428242] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164810worker1 # [ 66.437060] nix-grpc-daemon[923]: time=2026-09-25T16:50:45.759Z level=INFO msg="All 1 paths already cached"4811worker2 # [ 66.428394] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537244812worker2 # [ 66.435691] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537244813client # copying 3 paths...4814worker2 # [ 66.445517] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537244815client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4816worker2 # [ 66.462471] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537244817worker2 # [ 66.468056] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=2324818worker1 # [ 66.485138] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466164819worker1 # [ 66.490682] nix-grpc-daemon[511]: ts=2026-09-25T16:50:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_out=1764820worker2 # [ 66.476825] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537244821worker2 # [ 66.481767] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=1284822client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4823client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4824client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.35 seconds)4825(finished: subtest: outputs read back through any worker carry the cache signature, in 5.43 seconds)4826subtest: two clients wanting the same drv build it once4827client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4828client # [ 66.536274] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4829client # Running as unit: dedup1.service; invocation ID: 55478239acdf4afc900cb99e6bb1da804830client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4831client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4832worker2 # [ 66.666273] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537244833client # [ 66.671770] nix[952]: don't know how to build these paths:4834client # [ 66.672790] nix[952]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4835worker1 # [ 66.695985] nix-grpc-daemon[511]: ts=2026-09-25T16:50:46Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04836worker2 # [ 66.682342] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537244837worker1 # [ 66.700285] nix-grpc-daemon[511]: ts=2026-09-25T16:50:46Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14838worker2 # [ 66.694723] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537244839client # [ 66.703963] nix[952]: copying 1 paths to 'lb:50051'...4840client # [ 66.704798] nix[952]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4841worker2 # [ 66.705230] nix-grpc-daemon[800]: ts=2026-09-25T16:50:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537244842worker2 # [ 66.716513] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4843worker2 # [ 66.823295] nix-daemon[858]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4844worker2 # [ 66.838155] nix-daemon[858]: reaped child process 1032, status = succeeded4845lb # [ 66.841776] niks3-server[957]: 2026/09/25 16:50:46 INFO Received push request method=POST path=/api/pushes4846worker2 # [ 66.852594] nix-grpc-daemon[871]: time=2026-09-25T16:50:46.143Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4847worker2 # [ 66.854054] nix-grpc-daemon[871]: time=2026-09-25T16:50:46.143Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4848lb # [ 66.870732] niks3-server[957]: 2026/09/25 16:50:46 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4849lb # [ 66.877671] niks3-server[957]: 2026/09/25 16:50:46 INFO Signed narinfos id=15 count=14850worker2 # [ 66.883846] nix-grpc-daemon[871]: time=2026-09-25T16:50:46.175Z level=INFO msg="Uploading 1 narinfos"4851lb # [ 66.886498] niks3-server[957]: 2026/09/25 16:50:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4852lb # [ 66.892215] niks3-server[957]: 2026/09/25 16:50:46 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4853lb # [ 66.903632] niks3-server[957]: 2026/09/25 16:50:46 INFO Received complete push request method=POST path=/api/pushes/15/complete4854worker2 # [ 66.916802] nix-grpc-daemon[871]: time=2026-09-25T16:50:46.208Z level=INFO msg="Upload complete. (188ms)"4855lb # [ 66.915258] niks3-server[957]: 2026/09/25 16:50:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4856worker2 # [ 66.920944] nix-grpc-daemon[800]: ts=2026-09-25T16:50:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_in=4324857worker2 # [ 66.926288] nix-grpc-daemon[800]: ts=2026-09-25T16:50:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537244858worker2 # [ 66.935494] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4859worker2 # [ 66.944066] nix-daemon[858]: reaped child process 1027, status = succeeded4860worker2 # [ 66.947794] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4861client # [ 66.947234] nix[952]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4862lb # [ 67.329337] envoy[919]: conn peer=[2001:db8:1::1]:45318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=584863lb # [ 67.332983] envoy[919]: conn peer=[2001:db8:1::1]:45332 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4208 tx=1920 ms=20254864lb # [ 67.336545] envoy[919]: conn peer=[2001:db8:1::1]:45340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=4315 ms=44714865lb # [ 67.340128] envoy[919]: conn peer=[2001:db8:1::1]:44344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=404866lb # [ 67.344281] envoy[919]: conn peer=[2001:db8:1::1]:44328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2872 ms=964867lb # [ 67.348230] envoy[919]: conn peer=[2001:db8:1::1]:44372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=274868lb # [ 67.351673] envoy[919]: conn peer=[2001:db8:1::1]:44360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=344869client # [ 67.590685] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4870client # Running as unit: dedup2.service; invocation ID: 5a4be7a2b7ef48d6874d9f965bd6ade54871client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4872client: must succeed: sleep 24873worker2 # [ 67.770843] nix-grpc-daemon[800]: ts=2026-09-25T16:50:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537244874client # [ 67.780343] nix[968]: this derivation will be built:4875worker2 # [ 67.786259] nix-grpc-daemon[800]: ts=2026-09-25T16:50:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537244876client # [ 67.788481] nix[968]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4877worker2 # [ 67.799927] nix-grpc-daemon[800]: ts=2026-09-25T16:50:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537244878client # [ 67.801832] nix[968]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4879worker1 # [ 67.814927] nix-grpc-daemon[511]: ts=2026-09-25T16:50:47Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04880client: (finished: must succeed: sleep 2, in 2.05 seconds)4881client: waiting for success: ! systemctl is-active dedup1 dedup24882worker2 # [ 69.982449] nix-daemon[858]: reaped child process 1036, status = succeeded4883worker2 # [ 70.062667] nix-daemon[858]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)4884worker2 # [ 70.075203] nix-daemon[858]: reaped child process 1065, status = succeeded4885lb # [ 70.081306] niks3-server[957]: 2026/09/25 16:50:49 INFO Received push request method=POST path=/api/pushes4886worker2 # [ 70.166593] nix-grpc-daemon[871]: time=2026-09-25T16:50:49.457Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4887worker2 # [ 70.171836] nix-grpc-daemon[871]: time=2026-09-25T16:50:49.460Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4888lb # [ 70.211181] niks3-server[957]: 2026/09/25 16:50:49 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4889lb # [ 70.221138] niks3-server[957]: 2026/09/25 16:50:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4890lb # [ 70.228075] niks3-server[957]: 2026/09/25 16:50:49 INFO Signed narinfos id=16 count=14891worker2 # [ 70.234694] nix-grpc-daemon[871]: time=2026-09-25T16:50:49.525Z level=INFO msg="Uploading 1 narinfos"4892lb # [ 70.233417] niks3-server[957]: 2026/09/25 16:50:49 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4893lb # [ 70.238321] niks3-server[957]: 2026/09/25 16:50:49 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4894lb # [ 70.251232] niks3-server[957]: 2026/09/25 16:50:49 INFO Received complete push request method=POST path=/api/pushes/16/complete4895worker2 # [ 70.260950] nix-grpc-daemon[871]: time=2026-09-25T16:50:49.552Z level=INFO msg="Upload complete. (305ms)"4896lb # [ 70.262150] niks3-server[957]: 2026/09/25 16:50:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4897worker2 # [ 70.268702] nix-grpc-daemon[800]: ts=2026-09-25T16:50:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04898worker2 # [ 70.277285] nix-grpc-daemon[800]: ts=2026-09-25T16:50:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04899worker2 # [ 70.291303] nix-daemon[858]: reaped child process 1034, status = succeeded4900client # [ 70.312752] systemd[1]: dedup1.service: Deactivated successfully.4901client # [ 70.316115] systemd[1]: dedup1.service: Consumed 116ms CPU time over 3.778s wall clock time, 43.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4902client # [ 70.321391] systemd[1]: dedup2.service: Deactivated successfully.4903client # [ 70.324397] systemd[1]: dedup2.service: Consumed 146ms CPU time over 2.733s wall clock time, 35.4M memory peak, 6.8K incoming IP traffic, 6.1K outgoing IP traffic.4904client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)4905client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4906client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4907worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4908worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4909worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4910worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4911worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4912worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4913worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4914worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4915(finished: subtest: two clients wanting the same drv build it once, in 4.65 seconds)4916subtest: --timeout and --max-silent-time reach the builder4917worker1 # [ 71.323615] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466164918worker1 # [ 71.335963] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04919worker1 # [ 71.342687] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14920worker1 # [ 71.347427] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164921worker1 # [ 71.355185] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164922worker1 # [ 71.365826] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466164923worker1 # [ 71.376586] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4924worker1 # [ 71.472523] nix-daemon[864]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)4925worker1 # [ 71.487369] nix-daemon[864]: reaped child process 1056, status = succeeded4926lb # [ 71.475333] niks3-server[957]: 2026/09/25 16:50:50 INFO Received push request method=POST path=/api/pushes4927worker1 # [ 71.501701] nix-grpc-daemon[923]: time=2026-09-25T16:50:50.823Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4928worker1 # [ 71.503196] nix-grpc-daemon[923]: time=2026-09-25T16:50:50.823Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4929lb # [ 71.509261] niks3-server[957]: 2026/09/25 16:50:50 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4930lb # [ 71.514146] niks3-server[957]: 2026/09/25 16:50:50 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4931lb # [ 71.521311] niks3-server[957]: 2026/09/25 16:50:50 INFO Signed narinfos id=17 count=14932worker1 # [ 71.543629] nix-grpc-daemon[923]: time=2026-09-25T16:50:50.866Z level=INFO msg="Uploading 1 narinfos"4933lb # [ 71.529689] niks3-server[957]: 2026/09/25 16:50:50 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4934lb # [ 71.538711] niks3-server[957]: 2026/09/25 16:50:50 INFO Received complete push request method=POST path=/api/pushes/17/complete4935lb # [ 71.547715] niks3-server[957]: 2026/09/25 16:50:50 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4936worker1 # [ 71.564755] nix-grpc-daemon[923]: time=2026-09-25T16:50:50.887Z level=INFO msg="Upload complete. (177ms)"4937worker1 # [ 71.569381] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=4244938worker1 # [ 71.575437] nix-grpc-daemon[511]: ts=2026-09-25T16:50:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164939worker1 # [ 71.587103] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4940worker1 # [ 71.593767] nix-daemon[864]: reaped child process 1051, status = succeeded4941worker1 # [ 71.598559] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4942lb # [ 77.330326] envoy[919]: conn peer=[2001:db8:1::1]:44384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1562 ms=36154943lb # [ 77.334106] envoy[919]: conn peer=[2001:db8:1::1]:44388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1619 tx=1798 ms=25204944worker1 # [ 77.610605] nix-grpc-daemon[511]: ts=2026-09-25T16:50:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04945worker1 # [ 77.622975] nix-daemon[864]: reaped child process 1058, status = succeeded4946worker1 # [ 77.624148] nix-daemon[864]: reaped child process 1060, status = succeeded4947worker2 # [ 77.758475] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537244948worker1 # [ 77.779174] nix-grpc-daemon[511]: ts=2026-09-25T16:50:57Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04949worker1 # [ 77.781411] nix-grpc-daemon[511]: ts=2026-09-25T16:50:57Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14950worker2 # [ 77.766773] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537244951worker2 # [ 77.778648] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537244952worker2 # [ 77.789205] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537244953worker2 # [ 77.800197] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4954worker2 # [ 77.904800] nix-daemon[858]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4955worker2 # [ 77.917974] nix-daemon[858]: reaped child process 1081, status = succeeded4956lb # [ 77.924419] niks3-server[957]: 2026/09/25 16:50:57 INFO Received push request method=POST path=/api/pushes4957worker2 # [ 77.935434] nix-grpc-daemon[871]: time=2026-09-25T16:50:57.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4958worker2 # [ 77.937335] nix-grpc-daemon[871]: time=2026-09-25T16:50:57.228Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4959lb # [ 77.955455] niks3-server[957]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4960lb # [ 77.962868] niks3-server[957]: 2026/09/25 16:50:57 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4961worker2 # [ 77.970984] nix-grpc-daemon[871]: time=2026-09-25T16:50:57.262Z level=INFO msg="Uploading 1 narinfos"4962lb # [ 77.968775] niks3-server[957]: 2026/09/25 16:50:57 INFO Signed narinfos id=18 count=14963lb # [ 77.982553] niks3-server[957]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4964lb # [ 77.988069] niks3-server[957]: 2026/09/25 16:50:57 INFO Received complete push request method=POST path=/api/pushes/18/complete4965lb # [ 77.998044] niks3-server[957]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4966worker2 # [ 77.999878] nix-grpc-daemon[871]: time=2026-09-25T16:50:57.291Z level=INFO msg="Upload complete. (188ms)"4967worker2 # [ 78.004711] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_in=4244968worker2 # [ 78.010709] nix-grpc-daemon[800]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537244969worker2 # [ 78.023193] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4970worker2 # [ 78.032228] nix-daemon[858]: reaped child process 1076, status = succeeded4971worker2 # [ 78.036395] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)4972worker2 # [ 84.249629] nix-grpc-daemon[800]: ts=2026-09-25T16:51:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04973worker2 # [ 84.274668] nix-daemon[858]: reaped child process 1083, status = succeeded4974worker2 # [ 84.276668] nix-daemon[858]: reaped child process 1085, status = succeeded4975worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4976worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4977worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4978worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4979(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.30 seconds)4980subtest: a failing build is counted with its own reason4981worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4982worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4983worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4984worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4985worker1 # [ 84.705631] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466164986worker1 # [ 84.719297] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=want client=9 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv cached=04987worker1 # [ 84.725323] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=assigned drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv worker=192.168.1.3:50051 clients=14988worker1 # [ 84.730123] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466164989worker1 # [ 84.742435] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466164990worker1 # [ 84.753080] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466164991worker1 # [ 84.763960] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4992worker1 # [ 84.859268] nix-daemon[864]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)4993worker1 # [ 84.872076] nix-daemon[864]: reaped child process 1096, status = succeeded4994lb # [ 84.860813] niks3-server[957]: 2026/09/25 16:51:04 INFO Received push request method=POST path=/api/pushes4995worker1 # [ 84.886987] nix-grpc-daemon[923]: time=2026-09-25T16:51:04.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4996worker1 # [ 84.888986] nix-grpc-daemon[923]: time=2026-09-25T16:51:04.211Z level=INFO msg="Uploading j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv (400B)"4997lb # [ 84.893910] niks3-server[957]: 2026/09/25 16:51:04 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4998lb # [ 84.901515] niks3-server[957]: 2026/09/25 16:51:04 INFO Signed narinfos id=19 count=14999worker1 # [ 84.923440] nix-grpc-daemon[923]: time=2026-09-25T16:51:04.245Z level=INFO msg="Uploading 1 narinfos"5000lb # [ 84.912296] niks3-server[957]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.ls5001lb # [ 84.916848] niks3-server[957]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=nar/1awa0khyixxmb53bhxv3vk1qy0fd3xzskbp3b8g2fzdmcz3cvv97.nar.zst5002lb # [ 84.933447] niks3-server[957]: 2026/09/25 16:51:04 INFO Received complete push request method=POST path=/api/pushes/19/complete5003lb # [ 84.941541] niks3-server[957]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.narinfo5004worker1 # [ 84.959192] nix-grpc-daemon[923]: time=2026-09-25T16:51:04.281Z level=INFO msg="Upload complete. (184ms)"5005worker1 # [ 84.963950] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=4005006worker1 # [ 84.970335] nix-grpc-daemon[511]: ts=2026-09-25T16:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165007worker1 # [ 84.979992] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5008worker1 # [ 84.993277] nix-daemon[864]: reaped child process 1091, status = succeeded5009worker1 # [ 84.996840] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5010worker1 # [ 86.006525] nix-grpc-daemon[511]: ts=2026-09-25T16:51:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv assign_id=11 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05011worker1 # [ 86.031894] nix-daemon[864]: reaped child process 1098, status = succeeded5012worker1 # [ 86.034468] nix-daemon[864]: reaped child process 1100, status = succeeded5013worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5014worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5015worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5016worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5017worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5018worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5019worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5020worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5021worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5022worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5023worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5024worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5025worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5026worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5027worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5028worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5029(finished: subtest: a failing build is counted with its own reason, in 1.90 seconds)5030subtest: the first client leaving does not fail the second5031client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5032client # [ 86.408896] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5033client # Running as unit: share1.service; invocation ID: 07c74080baff42778a2bd3331816ff1c5034client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5035client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5036client # [ 86.555076] nix[1059]: don't know how to build these paths:5037client # [ 86.556472] nix[1059]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5038worker1 # [ 86.569438] nix-grpc-daemon[511]: ts=2026-09-25T16:51:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466165039worker1 # [ 86.580548] nix-grpc-daemon[511]: ts=2026-09-25T16:51:05Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05040worker1 # [ 86.585776] nix-grpc-daemon[511]: ts=2026-09-25T16:51:05Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15041worker2 # [ 86.573844] nix-grpc-daemon[800]: ts=2026-09-25T16:51:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245042worker2 # [ 86.586881] nix-grpc-daemon[800]: ts=2026-09-25T16:51:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537245043client # [ 86.595985] nix[1059]: copying 1 paths to 'lb:50051'...5044client # [ 86.596867] nix[1059]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5045worker2 # [ 86.597522] nix-grpc-daemon[800]: ts=2026-09-25T16:51:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537245046worker2 # [ 86.608121] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5047worker2 # [ 86.712423] nix-daemon[858]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5048worker2 # [ 86.724962] nix-daemon[858]: reaped child process 1129, status = succeeded5049lb # [ 86.731960] niks3-server[957]: 2026/09/25 16:51:06 INFO Received push request method=POST path=/api/pushes5050worker2 # [ 86.742594] nix-grpc-daemon[871]: time=2026-09-25T16:51:06.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5051worker2 # [ 86.744553] nix-grpc-daemon[871]: time=2026-09-25T16:51:06.035Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5052lb # [ 86.763626] niks3-server[957]: 2026/09/25 16:51:06 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5053worker2 # [ 86.772659] nix-grpc-daemon[871]: time=2026-09-25T16:51:06.064Z level=INFO msg="Uploading 1 narinfos"5054lb # [ 86.770358] niks3-server[957]: 2026/09/25 16:51:06 INFO Signed narinfos id=20 count=15055lb # [ 86.775756] niks3-server[957]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5056lb # [ 86.781657] niks3-server[957]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5057lb # [ 86.795134] niks3-server[957]: 2026/09/25 16:51:06 INFO Received complete push request method=POST path=/api/pushes/20/complete5058lb # [ 86.802879] niks3-server[957]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5059worker2 # [ 86.810999] nix-grpc-daemon[871]: time=2026-09-25T16:51:06.102Z level=INFO msg="Upload complete. (191ms)"5060worker2 # [ 86.814966] nix-grpc-daemon[800]: ts=2026-09-25T16:51:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_in=4325061worker2 # [ 86.819484] nix-grpc-daemon[800]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245062worker2 # [ 86.828176] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5063worker2 # [ 86.838098] nix-daemon[858]: reaped child process 1124, status = succeeded5064worker2 # [ 86.842996] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5065client # [ 86.841569] nix[1059]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5066lb # [ 87.330469] envoy[919]: conn peer=[2001:db8:1::1]:44402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=63045067lb # [ 87.334205] envoy[919]: conn peer=[2001:db8:1::1]:46064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=959 ms=65055068lb # [ 87.337654] envoy[919]: conn peer=[2001:db8:1::1]:43696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2866 tx=1270 ms=13195069client # [ 87.481665] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5070client # Running as unit: share2.service; invocation ID: 82b70dcf91b04f8f9d84c3def747acca5071client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.08 seconds)5072client: must succeed: sleep 2; systemctl kill -s INT share15073worker1 # [ 87.652781] nix-grpc-daemon[511]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466165074client # [ 87.643203] nix[1075]: don't know how to build these paths:5075client # [ 87.644214] nix[1075]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5076worker1 # [ 87.665735] nix-grpc-daemon[511]: ts=2026-09-25T16:51:06Z level=debug event=want client=11 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05077worker2 # [ 87.653511] nix-grpc-daemon[800]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245078client # [ 87.655793] nix[1075]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5079client # [ 89.642132] systemd[1]: share1.service: Sent signal SIGINT to main process 1059 (nix) on client request.5080client # [ 89.648822] nix[1059]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5081client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5082client: waiting for success: ! systemctl is-active share15083client # [ 89.658532] nix[1059]: error: interrupted by the user5084client # [ 89.697808] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5085client # [ 89.699322] systemd[1]: share1.service: Failed with result 'exit-code'.5086client # [ 89.701262] systemd[1]: share1.service: Consumed 117ms CPU time over 3.292s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5087client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5088client: waiting for success: ! systemctl is-active share25089worker2 # [ 91.072315] nix-daemon[858]: reaped child process 1133, status = succeeded5090worker2 # [ 91.153974] nix-daemon[858]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5091worker2 # [ 91.164818] nix-daemon[858]: reaped child process 1166, status = succeeded5092lb # [ 91.171884] niks3-server[957]: 2026/09/25 16:51:10 INFO Received push request method=POST path=/api/pushes5093worker2 # [ 91.183586] nix-grpc-daemon[871]: time=2026-09-25T16:51:10.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5094worker2 # [ 91.185303] nix-grpc-daemon[871]: time=2026-09-25T16:51:10.476Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5095lb # [ 91.217308] niks3-server[957]: 2026/09/25 16:51:10 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5096worker2 # [ 91.228856] nix-grpc-daemon[871]: time=2026-09-25T16:51:10.520Z level=INFO msg="Uploading 1 narinfos"5097lb # [ 91.226191] niks3-server[957]: 2026/09/25 16:51:10 INFO Signed narinfos id=21 count=15098lb # [ 91.234078] niks3-server[957]: 2026/09/25 16:51:10 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5099lb # [ 91.239264] niks3-server[957]: 2026/09/25 16:51:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5100lb # [ 91.250157] niks3-server[957]: 2026/09/25 16:51:10 INFO Received complete push request method=POST path=/api/pushes/21/complete5101lb # [ 91.252583] niks3-server[957]: 2026/09/25 16:51:10 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5102lb # [ 91.256543] niks3-server[957]: 2026/09/25 16:51:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5103worker2 # [ 91.262762] nix-grpc-daemon[871]: time=2026-09-25T16:51:10.554Z level=INFO msg="Upload complete. (211ms)"5104worker2 # [ 91.268847] nix-grpc-daemon[800]: ts=2026-09-25T16:51:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05105worker2 # [ 91.277299] nix-grpc-daemon[800]: ts=2026-09-25T16:51:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05106worker2 # [ 91.290909] nix-daemon[858]: reaped child process 1131, status = succeeded5107client # [ 91.299427] systemd[1]: share2.service: Deactivated successfully.5108client # [ 91.303139] systemd[1]: share2.service: Consumed 120ms CPU time over 3.821s wall clock time, 35.4M memory peak, 6.4K incoming IP traffic, 5.9K outgoing IP traffic.5109client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5110client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5111client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5112(finished: subtest: the first client leaving does not fail the second, in 5.65 seconds)5113subtest: build hook: nix-daemon with builders = grpc://lb5114client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5115client # [ 92.105462] systemd[1]: Started Nix Daemon.5116client # [ 92.193264] nix-daemon[1126]: accepted connection from pid 1123, user root (trusted)5117worker1 # [ 92.323820] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466165118worker1 # [ 92.329189] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165119worker1 # [ 92.339731] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=want client=12 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05120worker1 # [ 92.346396] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15121worker1 # [ 92.351240] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165122worker1 # [ 92.358534] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165123worker1 # [ 92.369606] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466165124worker1 # [ 92.377984] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5125worker1 # [ 92.473947] nix-daemon[864]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)5126worker1 # [ 92.486496] nix-daemon[864]: reaped child process 1151, status = succeeded5127lb # [ 92.475760] niks3-server[957]: 2026/09/25 16:51:11 INFO Received push request method=POST path=/api/pushes5128worker1 # [ 92.502298] nix-grpc-daemon[923]: time=2026-09-25T16:51:11.824Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5129worker1 # [ 92.504455] nix-grpc-daemon[923]: time=2026-09-25T16:51:11.826Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5130lb # [ 92.506667] niks3-server[957]: 2026/09/25 16:51:11 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5131lb # [ 92.517234] niks3-server[957]: 2026/09/25 16:51:11 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5132worker1 # [ 92.541077] nix-grpc-daemon[923]: time=2026-09-25T16:51:11.863Z level=INFO msg="Uploading 1 narinfos"5133lb # [ 92.522937] niks3-server[957]: 2026/09/25 16:51:11 INFO Signed narinfos id=22 count=15134lb # [ 92.529297] niks3-server[957]: 2026/09/25 16:51:11 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5135lb # [ 92.537908] niks3-server[957]: 2026/09/25 16:51:11 INFO Received complete push request method=POST path=/api/pushes/22/complete5136worker1 # [ 92.566873] nix-grpc-daemon[923]: time=2026-09-25T16:51:11.889Z level=INFO msg="Upload complete. (176ms)"5137lb # [ 92.549935] niks3-server[957]: 2026/09/25 16:51:11 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5138worker1 # [ 92.570314] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=5125139worker1 # [ 92.577233] nix-grpc-daemon[511]: ts=2026-09-25T16:51:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165140worker1 # [ 92.585847] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5141worker1 # [ 92.594461] nix-daemon[864]: reaped child process 1146, status = succeeded5142worker1 # [ 92.599147] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5143worker1 # [ 93.634509] nix-daemon[864]: reaped child process 1155, status = succeeded5144worker1 # [ 93.719086] nix-daemon[864]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5145worker1 # [ 93.731151] nix-daemon[864]: reaped child process 1174, status = succeeded5146lb # [ 93.719482] niks3-server[957]: 2026/09/25 16:51:12 INFO Received push request method=POST path=/api/pushes5147worker1 # [ 93.745286] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.067Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5148worker1 # [ 93.747144] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.069Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5149lb # [ 93.763220] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5150lb # [ 93.770826] niks3-server[957]: 2026/09/25 16:51:13 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5151lb # [ 93.776752] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5152lb # [ 93.782099] niks3-server[957]: 2026/09/25 16:51:13 INFO Signed narinfos id=23 count=15153worker1 # [ 93.803709] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.126Z level=INFO msg="Uploading 1 narinfos"5154lb # [ 93.793719] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5155lb # [ 93.798398] niks3-server[957]: 2026/09/25 16:51:13 INFO Received complete push request method=POST path=/api/pushes/23/complete5156worker1 # [ 93.820420] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.142Z level=INFO msg="Upload complete. (213ms)"5157lb # [ 93.804037] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5158worker1 # [ 93.827604] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=13 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05159worker2 # [ 93.824259] nix-grpc-daemon[800]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537245160worker1 # [ 93.843189] nix-daemon[864]: reaped child process 1153, status = succeeded5161worker2 # [ 93.833385] nix-grpc-daemon[800]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5162worker2 # [ 93.856929] nix-grpc-daemon[800]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537245163worker2 # [ 93.862269] nix-grpc-daemon[800]: ts=2026-09-25T16:51:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=1285164worker1 # [ 94.039980] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466165165worker1 # [ 94.045558] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165166worker1 # [ 94.050532] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.373Z level=INFO msg="All 1 paths already cached"5167worker1 # [ 94.064145] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=want client=13 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05168worker1 # [ 94.068870] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15169worker1 # [ 94.075311] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165170worker1 # [ 94.086693] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165171worker1 # [ 94.099624] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.422Z level=INFO msg="All 1 paths already cached"5172worker1 # [ 94.107420] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466165173worker1 # [ 94.115550] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5174worker1 # [ 94.214328] nix-daemon[864]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5175worker1 # [ 94.228170] nix-daemon[864]: reaped child process 1183, status = succeeded5176lb # [ 94.216570] niks3-server[957]: 2026/09/25 16:51:13 INFO Received push request method=POST path=/api/pushes5177worker1 # [ 94.242666] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.565Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5178worker1 # [ 94.244192] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.566Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5179lb # [ 94.248718] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5180lb # [ 94.255137] niks3-server[957]: 2026/09/25 16:51:13 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5181lb # [ 94.261649] niks3-server[957]: 2026/09/25 16:51:13 INFO Signed narinfos id=24 count=15182worker1 # [ 94.283441] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.606Z level=INFO msg="Uploading 1 narinfos"5183lb # [ 94.269286] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5184lb # [ 94.279400] niks3-server[957]: 2026/09/25 16:51:13 INFO Received complete push request method=POST path=/api/pushes/24/complete5185lb # [ 94.291079] niks3-server[957]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5186worker1 # [ 94.314192] nix-grpc-daemon[923]: time=2026-09-25T16:51:13.636Z level=INFO msg="Upload complete. (185ms)"5187worker1 # [ 94.318811] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=6245188worker1 # [ 94.324402] nix-grpc-daemon[511]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165189worker1 # [ 94.334855] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5190worker1 # [ 94.345383] nix-daemon[864]: reaped child process 1178, status = succeeded5191worker1 # [ 94.350281] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5192worker1 # [ 95.565588] nix-daemon[864]: reaped child process 1187, status = succeeded5193worker1 # [ 95.640100] nix-daemon[864]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5194worker1 # [ 95.651980] nix-daemon[864]: reaped child process 1206, status = succeeded5195lb # [ 95.642347] niks3-server[957]: 2026/09/25 16:51:14 INFO Received push request method=POST path=/api/pushes5196worker1 # [ 95.668167] nix-grpc-daemon[923]: time=2026-09-25T16:51:14.990Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5197worker1 # [ 95.670991] nix-grpc-daemon[923]: time=2026-09-25T16:51:14.992Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5198lb # [ 95.683201] niks3-server[957]: 2026/09/25 16:51:14 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5199lb # [ 95.686872] niks3-server[957]: 2026/09/25 16:51:14 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5200lb # [ 95.692848] niks3-server[957]: 2026/09/25 16:51:14 INFO Signed narinfos id=25 count=15201worker1 # [ 95.713442] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.035Z level=INFO msg="Uploading 1 narinfos"5202lb # [ 95.699112] niks3-server[957]: 2026/09/25 16:51:14 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5203lb # [ 95.710268] niks3-server[957]: 2026/09/25 16:51:14 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5204lb # [ 95.713150] niks3-server[957]: 2026/09/25 16:51:14 INFO Received complete push request method=POST path=/api/pushes/25/complete5205lb # [ 95.720148] niks3-server[957]: 2026/09/25 16:51:14 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5206worker1 # [ 95.737214] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.059Z level=INFO msg="Upload complete. (179ms)"5207worker1 # [ 95.744699] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05208worker1 # [ 95.750790] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466165209worker1 # [ 95.761336] nix-daemon[864]: reaped child process 1185, status = succeeded5210worker2 # [ 95.753269] nix-grpc-daemon[800]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537245211worker2 # [ 95.764102] nix-grpc-daemon[800]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5212worker2 # [ 95.775397] nix-grpc-daemon[800]: ts=2026-09-25T16:51:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=1765213worker1 # [ 95.956637] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466165214worker2 # [ 95.949416] nix-grpc-daemon[800]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537245215worker2 # [ 95.959491] nix-grpc-daemon[871]: time=2026-09-25T16:51:15.250Z level=INFO msg="All 2 paths already cached"5216worker1 # [ 95.989217] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=want client=14 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05217worker1 # [ 95.994596] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15218worker1 # [ 95.999877] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165219worker1 # [ 96.012133] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165220worker1 # [ 96.025486] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.347Z level=INFO msg="All 2 paths already cached"5221worker1 # [ 96.033379] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466165222worker1 # [ 96.041356] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5223worker1 # [ 96.140333] nix-daemon[864]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5224worker1 # [ 96.153047] nix-daemon[864]: reaped child process 1215, status = succeeded5225lb # [ 96.142337] niks3-server[957]: 2026/09/25 16:51:15 INFO Received push request method=POST path=/api/pushes5226worker1 # [ 96.168287] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.490Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5227worker1 # [ 96.170400] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.492Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5228lb # [ 96.170517] niks3-server[957]: 2026/09/25 16:51:15 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5229lb # [ 96.177913] niks3-server[957]: 2026/09/25 16:51:15 INFO Signed narinfos id=26 count=15230worker1 # [ 96.199580] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.522Z level=INFO msg="Uploading 1 narinfos"5231lb # [ 96.186701] niks3-server[957]: 2026/09/25 16:51:15 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5232lb # [ 96.195204] niks3-server[957]: 2026/09/25 16:51:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5233lb # [ 96.205124] niks3-server[957]: 2026/09/25 16:51:15 INFO Received complete push request method=POST path=/api/pushes/26/complete5234worker1 # [ 96.230931] nix-grpc-daemon[923]: time=2026-09-25T16:51:15.553Z level=INFO msg="Upload complete. (176ms)"5235lb # [ 96.214566] niks3-server[957]: 2026/09/25 16:51:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5236worker1 # [ 96.235424] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_in=7525237worker1 # [ 96.240694] nix-grpc-daemon[511]: ts=2026-09-25T16:51:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165238worker1 # [ 96.251972] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5239worker1 # [ 96.256565] nix-daemon[864]: reaped child process 1210, status = succeeded5240worker1 # [ 96.261166] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5241worker1 # [ 97.276233] nix-daemon[864]: reaped child process 1219, status = succeeded5242worker1 # [ 97.350162] nix-daemon[864]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5243lb # [ 97.331482] envoy[919]: conn peer=[2001:db8:1::1]:43712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=772 ms=31105244lb # [ 97.336062] envoy[919]: conn peer=[2001:db8:1::1]:43716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1470 tx=1400 ms=36465245lb # [ 97.339621] envoy[919]: conn peer=[2001:db8:1::1]:43720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3406 tx=1922 ms=15665246worker1 # [ 97.363340] nix-daemon[864]: reaped child process 1238, status = succeeded5247lb # [ 97.343565] envoy[919]: conn peer=[2001:db8:1::1]:34696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=165248lb # [ 97.347134] envoy[919]: conn peer=[2001:db8:1::1]:34706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=2139 ms=17625249lb # [ 97.351637] envoy[919]: conn peer=[2001:db8:1::1]:34710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=358 ms=355250lb # [ 97.358950] niks3-server[957]: 2026/09/25 16:51:16 INFO Received push request method=POST path=/api/pushes5251worker1 # [ 97.384800] nix-grpc-daemon[923]: time=2026-09-25T16:51:16.707Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5252worker1 # [ 97.386612] nix-grpc-daemon[923]: time=2026-09-25T16:51:16.709Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5253lb # [ 97.391085] niks3-server[957]: 2026/09/25 16:51:16 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5254lb # [ 97.392789] niks3-server[957]: 2026/09/25 16:51:16 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5255lb # [ 97.394836] niks3-server[957]: 2026/09/25 16:51:16 INFO Signed narinfos id=27 count=15256worker1 # [ 97.413217] nix-grpc-daemon[923]: time=2026-09-25T16:51:16.735Z level=INFO msg="Uploading 1 narinfos"5257lb # [ 97.400836] niks3-server[957]: 2026/09/25 16:51:16 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5258lb # [ 97.403734] niks3-server[957]: 2026/09/25 16:51:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5259lb # [ 97.410684] niks3-server[957]: 2026/09/25 16:51:16 INFO Received complete push request method=POST path=/api/pushes/27/complete5260worker1 # [ 97.434611] nix-grpc-daemon[923]: time=2026-09-25T16:51:16.757Z level=INFO msg="Upload complete. (166ms)"5261lb # [ 97.417534] niks3-server[957]: 2026/09/25 16:51:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5262worker1 # [ 97.439576] nix-grpc-daemon[511]: ts=2026-09-25T16:51:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=15 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05263worker1 # [ 97.451236] nix-daemon[864]: reaped child process 1217, status = succeeded5264worker2 # [ 97.435913] nix-grpc-daemon[800]: ts=2026-09-25T16:51:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537245265worker2 # [ 97.444860] nix-grpc-daemon[800]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5266worker2 # [ 97.468333] nix-grpc-daemon[800]: ts=2026-09-25T16:51:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537245267worker2 # [ 97.473415] nix-grpc-daemon[800]: ts=2026-09-25T16:51:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=2325268client # [ 97.542982] nix-daemon[1126]: reaped child process 1128, status = succeeded5269client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.53 seconds)5270client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5271client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5272client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5273client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5274client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5275client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5276(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.59 seconds)5277subtest: an input only one worker has still reaches the builder5278client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5279client # this derivation will be built:5280client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5281client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5282client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5283client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5284client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5285worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5286worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5287worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5288worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5289client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25290client # [ 98.201362] nix-daemon[1126]: accepted connection from pid 1209, user root (trusted)5291client # this derivation will be built:5292client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5293worker2 # [ 98.319688] nix-grpc-daemon[800]: ts=2026-09-25T16:51:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537245294client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5295worker1 # [ 98.337990] nix-grpc-daemon[511]: ts=2026-09-25T16:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165296worker1 # [ 98.444418] nix-daemon[864]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)5297worker1 # [ 98.457297] nix-daemon[864]: reaped child process 1248, status = succeeded5298lb # [ 98.446825] niks3-server[957]: 2026/09/25 16:51:17 INFO Received push request method=POST path=/api/pushes5299worker1 # [ 98.472775] nix-grpc-daemon[923]: time=2026-09-25T16:51:17.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5300worker1 # [ 98.474903] nix-grpc-daemon[923]: time=2026-09-25T16:51:17.797Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5301lb # [ 98.479606] niks3-server[957]: 2026/09/25 16:51:17 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5302lb # [ 98.484757] niks3-server[957]: 2026/09/25 16:51:17 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5303lb # [ 98.490741] niks3-server[957]: 2026/09/25 16:51:17 INFO Signed narinfos id=28 count=15304worker1 # [ 98.512639] nix-grpc-daemon[923]: time=2026-09-25T16:51:17.835Z level=INFO msg="Uploading 1 narinfos"5305lb # [ 98.498702] niks3-server[957]: 2026/09/25 16:51:17 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5306lb # [ 98.511731] niks3-server[957]: 2026/09/25 16:51:17 INFO Received complete push request method=POST path=/api/pushes/28/complete5307worker1 # [ 98.535551] nix-grpc-daemon[923]: time=2026-09-25T16:51:17.858Z level=INFO msg="Upload complete. (192ms)"5308lb # [ 98.520918] niks3-server[957]: 2026/09/25 16:51:17 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5309worker1 # [ 98.554664] nix-grpc-daemon[511]: ts=2026-09-25T16:51:17Z level=debug event=want client=15 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05310worker1 # [ 98.559842] nix-grpc-daemon[511]: ts=2026-09-25T16:51:17Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15311worker2 # [ 98.547662] nix-grpc-daemon[800]: ts=2026-09-25T16:51:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245312worker2 # [ 98.557665] nix-grpc-daemon[800]: ts=2026-09-25T16:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537245313client # copying 2 paths to 'lb:50051'...5314client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5315worker2 # [ 98.572408] nix-grpc-daemon[800]: ts=2026-09-25T16:51:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537245316client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5317worker2 # [ 98.586259] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5318worker2 # [ 98.692336] nix-daemon[858]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5319worker2 # [ 98.705042] nix-daemon[858]: reaped child process 1179, status = succeeded5320lb # [ 98.710151] niks3-server[957]: 2026/09/25 16:51:17 INFO Received push request method=POST path=/api/pushes5321worker2 # [ 98.721414] nix-grpc-daemon[871]: time=2026-09-25T16:51:18.012Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5322worker2 # [ 98.723215] nix-grpc-daemon[871]: time=2026-09-25T16:51:18.014Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5323worker2 # [ 98.724883] nix-grpc-daemon[871]: time=2026-09-25T16:51:18.014Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5324lb # [ 98.755569] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5325lb # [ 98.770124] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5326lb # [ 98.775131] niks3-server[957]: 2026/09/25 16:51:18 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5327lb # [ 98.777759] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5328lb # [ 98.784265] niks3-server[957]: 2026/09/25 16:51:18 INFO Signed narinfos id=29 count=25329worker2 # [ 98.787729] nix-grpc-daemon[871]: time=2026-09-25T16:51:18.078Z level=INFO msg="Uploading 2 narinfos"5330lb # [ 98.787570] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5331lb # [ 98.796483] niks3-server[957]: 2026/09/25 16:51:18 INFO Received complete push request method=POST path=/api/pushes/29/complete5332lb # [ 98.801204] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5333worker2 # [ 98.807271] nix-grpc-daemon[871]: time=2026-09-25T16:51:18.098Z level=INFO msg="Upload complete. (208ms)"5334lb # [ 98.807680] niks3-server[957]: 2026/09/25 16:51:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5335worker2 # [ 98.812683] nix-grpc-daemon[800]: ts=2026-09-25T16:51:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=2 nar_bytes_in=9765336worker2 # [ 98.818677] nix-grpc-daemon[800]: ts=2026-09-25T16:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245337worker2 # [ 98.828160] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5338worker2 # [ 98.839637] nix-daemon[858]: reaped child process 1174, status = succeeded5339worker2 # [ 98.845474] nix-grpc-daemon[800]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5340worker2 # [ 98.853222] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5341client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5342client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5343client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5344worker2 # [ 99.868159] nix-daemon[858]: reaped child process 1184, status = succeeded5345worker2 # [ 99.945532] nix-daemon[858]: accepted connection from pid 1200, user nix-grpc-daemon (trusted)5346worker2 # [ 99.957166] nix-daemon[858]: reaped child process 1203, status = succeeded5347lb # [ 99.962718] niks3-server[957]: 2026/09/25 16:51:19 INFO Received push request method=POST path=/api/pushes5348worker2 # [ 99.975126] nix-grpc-daemon[871]: time=2026-09-25T16:51:19.266Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5349lb # [ 99.996286] niks3-server[957]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5350lb # [ 100.002301] niks3-server[957]: 2026/09/25 16:51:19 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5351lb # [ 100.009364] niks3-server[957]: 2026/09/25 16:51:19 INFO Signed narinfos id=30 count=15352worker2 # [ 100.015391] nix-grpc-daemon[871]: time=2026-09-25T16:51:19.306Z level=INFO msg="Uploading 1 narinfos"5353lb # [ 100.017151] niks3-server[957]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5354lb # [ 100.029722] niks3-server[957]: 2026/09/25 16:51:19 INFO Received complete push request method=POST path=/api/pushes/30/complete5355lb # [ 100.037534] niks3-server[957]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5356worker2 # [ 100.043767] nix-grpc-daemon[871]: time=2026-09-25T16:51:19.335Z level=INFO msg="Upload complete. (183ms)"5357client # copying 1 paths...5358worker2 # [ 100.048367] nix-grpc-daemon[800]: ts=2026-09-25T16:51:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=1 input_bytes=120 input_s=05359worker2 # [ 100.055440] nix-grpc-daemon[800]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537245360client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5361worker2 # [ 100.065613] nix-daemon[858]: reaped child process 1181, status = succeeded5362worker2 # [ 100.074569] nix-grpc-daemon[800]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537245363worker2 # [ 100.076692] nix-grpc-daemon[800]: ts=2026-09-25T16:51:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=1205364client # [ 100.147918] nix-daemon[1126]: reaped child process 1212, status = succeeded5365client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.07 seconds)5366client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25367client # [ 100.246097] nix-daemon[1126]: accepted connection from pid 1231, user root (trusted)5368client # this derivation will be built:5369client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5370worker2 # [ 100.365590] nix-grpc-daemon[800]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537245371client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5372worker1 # [ 100.385269] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165373worker1 # [ 100.396316] nix-grpc-daemon[923]: time=2026-09-25T16:51:19.718Z level=INFO msg="All 1 paths already cached"5374worker1 # [ 100.412139] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=want client=16 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05375worker1 # [ 100.419393] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15376worker1 # [ 100.425343] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165377worker1 # [ 100.436928] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165378client # copying 1 paths to 'lb:50051'...5379client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5380worker1 # [ 100.455636] nix-grpc-daemon[511]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466165381worker1 # [ 100.467082] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5382worker1 # [ 100.481578] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 1/5)5383worker1 # [ 100.565727] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 188 ms (attempt 2/5)5384worker1 # [ 100.755867] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 3/5)5385worker1 # [ 100.830824] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 498 ms (attempt 4/5)5386worker1 # [ 101.331241] nix-grpc-daemon[511]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5387worker1 # [ 101.334557] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5388worker1 # [ 101.341290] nix-grpc-daemon[511]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5389worker1 # [ 101.456802] nix-daemon[864]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5390worker1 # [ 101.471849] nix-daemon[864]: reaped child process 1264, status = succeeded5391lb # [ 101.463402] niks3-server[957]: 2026/09/25 16:51:20 INFO Received push request method=POST path=/api/pushes5392worker1 # [ 101.488490] nix-grpc-daemon[923]: time=2026-09-25T16:51:20.810Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5393worker1 # [ 101.490650] nix-grpc-daemon[923]: time=2026-09-25T16:51:20.812Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5394lb # [ 101.495649] niks3-server[957]: 2026/09/25 16:51:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5395lb # [ 101.499127] niks3-server[957]: 2026/09/25 16:51:20 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5396lb # [ 101.507953] niks3-server[957]: 2026/09/25 16:51:20 INFO Signed narinfos id=31 count=15397worker1 # [ 101.529208] nix-grpc-daemon[923]: time=2026-09-25T16:51:20.851Z level=INFO msg="Uploading 1 narinfos"5398lb # [ 101.515315] niks3-server[957]: 2026/09/25 16:51:20 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5399lb # [ 101.527667] niks3-server[957]: 2026/09/25 16:51:20 INFO Received complete push request method=POST path=/api/pushes/31/complete5400lb # [ 101.533980] niks3-server[957]: 2026/09/25 16:51:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5401worker1 # [ 101.555373] nix-grpc-daemon[923]: time=2026-09-25T16:51:20.877Z level=INFO msg="Upload complete. (193ms)"5402worker1 # [ 101.560742] nix-grpc-daemon[511]: ts=2026-09-25T16:51:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 paths=1 nar_bytes_in=5605403worker1 # [ 101.567909] nix-grpc-daemon[511]: ts=2026-09-25T16:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466165404worker1 # [ 101.579133] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5405worker1 # [ 101.589095] nix-daemon[864]: reaped child process 1254, status = succeeded5406worker1 # [ 101.592151] nix-daemon[864]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5407client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5408client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5409client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5410worker1 # [ 102.420211] nix-daemon[864]: reaped child process 1268, status = succeeded5411worker1 # [ 102.492941] nix-daemon[864]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5412worker1 # [ 102.506400] nix-daemon[864]: reaped child process 1287, status = succeeded5413lb # [ 102.494836] niks3-server[957]: 2026/09/25 16:51:21 INFO Received push request method=POST path=/api/pushes5414worker1 # [ 102.521430] nix-grpc-daemon[923]: time=2026-09-25T16:51:21.843Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5415lb # [ 102.532261] niks3-server[957]: 2026/09/25 16:51:21 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5416lb # [ 102.537731] niks3-server[957]: 2026/09/25 16:51:21 INFO Signed narinfos id=32 count=15417worker1 # [ 102.558616] nix-grpc-daemon[923]: time=2026-09-25T16:51:21.881Z level=INFO msg="Uploading 1 narinfos"5418lb # [ 102.542169] niks3-server[957]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5419lb # [ 102.551590] niks3-server[957]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5420lb # [ 102.559190] niks3-server[957]: 2026/09/25 16:51:21 INFO Received complete push request method=POST path=/api/pushes/32/complete5421worker1 # [ 102.589605] nix-grpc-daemon[923]: time=2026-09-25T16:51:21.912Z level=INFO msg="Upload complete. (190ms)"5422lb # [ 102.571968] niks3-server[957]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5423worker1 # [ 102.592161] nix-grpc-daemon[511]: ts=2026-09-25T16:51:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05424client # copying 1 paths...5425worker1 # [ 102.598427] nix-grpc-daemon[511]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466165426worker1 # [ 102.603154] nix-daemon[864]: reaped child process 1266, status = succeeded5427client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5428worker2 # [ 102.600078] nix-grpc-daemon[800]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537245429worker2 # [ 102.609954] nix-grpc-daemon[800]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5430worker2 # [ 102.623874] nix-grpc-daemon[800]: ts=2026-09-25T16:51:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_out=1205431client # [ 102.691872] nix-daemon[1126]: reaped child process 1234, status = succeeded5432client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.54 seconds)5433client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25434client # [ 102.791410] nix-daemon[1126]: accepted connection from pid 1253, user root (trusted)5435client # this derivation will be built:5436client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5437worker1 # [ 102.922575] nix-grpc-daemon[511]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466165438worker1 # [ 102.924985] nix-grpc-daemon[511]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466165439client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5440worker1 # [ 102.932502] nix-grpc-daemon[923]: time=2026-09-25T16:51:22.255Z level=INFO msg="All 1 paths already cached"5441worker1 # [ 102.942201] nix-grpc-daemon[511]: ts=2026-09-25T16:51:22Z level=debug event=want client=17 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05442worker1 # [ 102.944367] nix-grpc-daemon[511]: ts=2026-09-25T16:51:22Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15443worker2 # [ 102.930509] nix-grpc-daemon[800]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245444worker2 # [ 102.942883] nix-grpc-daemon[800]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537245445worker2 # [ 102.956703] nix-grpc-daemon[871]: time=2026-09-25T16:51:22.247Z level=INFO msg="All 1 paths already cached"5446client # copying 1 paths to 'lb:50051'...5447client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5448worker2 # [ 102.965154] nix-grpc-daemon[800]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537245449worker2 # [ 102.976903] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5450worker2 # [ 103.083165] nix-daemon[858]: accepted connection from pid 1208, user nix-grpc-daemon (trusted)5451worker2 # [ 103.097393] nix-daemon[858]: reaped child process 1211, status = succeeded5452lb # [ 103.101125] niks3-server[957]: 2026/09/25 16:51:22 INFO Received push request method=POST path=/api/pushes5453worker2 # [ 103.112198] nix-grpc-daemon[871]: time=2026-09-25T16:51:22.403Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5454worker2 # [ 103.113542] nix-grpc-daemon[871]: time=2026-09-25T16:51:22.403Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5455lb # [ 103.128891] niks3-server[957]: 2026/09/25 16:51:22 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5456lb # [ 103.138245] niks3-server[957]: 2026/09/25 16:51:22 INFO Signed narinfos id=33 count=15457worker2 # [ 103.144066] nix-grpc-daemon[871]: time=2026-09-25T16:51:22.435Z level=INFO msg="Uploading 1 narinfos"5458lb # [ 103.144620] niks3-server[957]: 2026/09/25 16:51:22 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5459lb # [ 103.151373] niks3-server[957]: 2026/09/25 16:51:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5460lb # [ 103.161617] niks3-server[957]: 2026/09/25 16:51:22 INFO Received complete push request method=POST path=/api/pushes/33/complete5461worker2 # [ 103.172835] nix-grpc-daemon[871]: time=2026-09-25T16:51:22.464Z level=INFO msg="Upload complete. (183ms)"5462lb # [ 103.171921] niks3-server[957]: 2026/09/25 16:51:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5463worker2 # [ 103.176706] nix-grpc-daemon[800]: ts=2026-09-25T16:51:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 paths=1 nar_bytes_in=5605464worker2 # [ 103.181589] nix-grpc-daemon[800]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537245465worker2 # [ 103.191480] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5466worker2 # [ 103.198301] nix-daemon[858]: reaped child process 1206, status = succeeded5467worker2 # [ 103.203264] nix-daemon[858]: accepted connection from pid 800, user nix-grpc-daemon (trusted)5468client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5469client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5470client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5471worker2 # [ 103.834566] nix-daemon[858]: reaped child process 1215, status = succeeded5472worker2 # [ 103.911888] nix-daemon[858]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5473worker2 # [ 103.923657] nix-daemon[858]: reaped child process 1234, status = succeeded5474lb # [ 103.930917] niks3-server[957]: 2026/09/25 16:51:23 INFO Received push request method=POST path=/api/pushes5475worker2 # [ 103.941971] nix-grpc-daemon[871]: time=2026-09-25T16:51:23.233Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5476lb # [ 103.962929] niks3-server[957]: 2026/09/25 16:51:23 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5477lb # [ 103.966484] niks3-server[957]: 2026/09/25 16:51:23 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5478lb # [ 103.971641] niks3-server[957]: 2026/09/25 16:51:23 INFO Signed narinfos id=34 count=15479worker2 # [ 103.977725] nix-grpc-daemon[871]: time=2026-09-25T16:51:23.268Z level=INFO msg="Uploading 1 narinfos"5480lb # [ 103.981151] niks3-server[957]: 2026/09/25 16:51:23 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5481lb # [ 103.994354] niks3-server[957]: 2026/09/25 16:51:23 INFO Received complete push request method=POST path=/api/pushes/34/complete5482worker2 # [ 104.003976] nix-grpc-daemon[871]: time=2026-09-25T16:51:23.295Z level=INFO msg="Upload complete. (194ms)"5483lb # [ 104.007317] niks3-server[957]: 2026/09/25 16:51:23 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5484worker2 # [ 104.008664] nix-grpc-daemon[800]: ts=2026-09-25T16:51:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53724 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=18 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05485client # copying 1 paths...5486worker2 # [ 104.016936] nix-grpc-daemon[800]: ts=2026-09-25T16:51:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537245487client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5488worker2 # [ 104.026633] nix-daemon[858]: reaped child process 1213, status = succeeded5489worker1 # [ 104.051267] nix-grpc-daemon[511]: ts=2026-09-25T16:51:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466165490worker1 # [ 104.058439] nix-grpc-daemon[511]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5491worker1 # [ 104.065301] nix-grpc-daemon[511]: ts=2026-09-25T16:51:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46616 duration_s=0 paths=1 nar_bytes_out=1205492client # [ 104.118344] nix-daemon[1126]: reaped child process 1256, status = succeeded5493client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.43 seconds)5494(finished: subtest: an input only one worker has still reaches the builder, in 6.51 seconds)5495subtest: upload whose reference the worker lacks is completed from the cache5496client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5497client # this derivation will be built:5498client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5499client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5500client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5501client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5502client # this derivation will be built:5503client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5504client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5505client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5506client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5507client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5508worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5509worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.10 seconds)5510worker2 # [ 104.834944] systemd[1]: Stopping Nix worker-protocol over gRPC...5511worker2 # [ 105.733822] nix-grpc-daemon[800]: ts=2026-09-25T16:51:25Z level=info event=signal_exit5512lb # [ 107.332601] envoy[919]: conn peer=[2001:db8:1::1]:34714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4281 tx=2354 ms=15425513lb # [ 107.336486] envoy[919]: conn peer=[2001:db8:1::1]:34730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=155514lb # [ 107.339991] envoy[919]: conn peer=[2001:db8:1::1]:34742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=165515lb # [ 107.343552] envoy[919]: conn peer=[2001:db8:1::1]:34734 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3851 tx=2043 ms=17745516lb # [ 107.347108] envoy[919]: conn peer=[2001:db8:1::1]:34754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2107 ms=22705517lb # [ 107.350611] envoy[919]: conn peer=[2001:db8:1::1]:34766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=355518lb # [ 107.354935] envoy[919]: conn peer=[2001:db8:1::1]:53752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3708 tx=2089 ms=11545519lb # [ 107.358460] envoy[919]: conn peer=[2001:db8:1::1]:53756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=255520worker1 # [ 107.739424] nix-grpc-daemon[511]: ts=2026-09-25T16:51:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:354885521worker1 # [ 107.741495] nix-grpc-daemon[511]: ts=2026-09-25T16:51:27Z level=info event=worker_gone addr=192.168.1.4:500515522worker2 # [ 107.732210] nix-grpc-daemon[800]: ts=2026-09-25T16:51:27Z level=info event=scheduler_restarting5523worker2 # [ 107.742626] nix-grpc-daemon[800]: ts=2026-09-25T16:51:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5524worker2 # [ 107.762802] nix-daemon[858]: reaped child process 860, status = succeeded5525worker2 # [ 108.412174] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5526worker2 # [ 108.415927] systemd[1]: Stopped Nix worker-protocol over gRPC.5527worker2 # [ 108.420396] systemd[1]: nix-grpc-daemon.service: Consumed 2.082s CPU time over 1min 15.840s wall clock time, 69.3M memory peak, 174.7K incoming IP traffic, 344.2K outgoing IP traffic.5528lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5529lb: must succeed: curl -sf localhost:9901/clusters5530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5531lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5532client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25533worker1 # [ 108.639251] nix-grpc-daemon[511]: ts=2026-09-25T16:51:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466165534worker1 # [ 108.643553] nix-grpc-daemon[511]: ts=2026-09-25T16:51:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466165535client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5536client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5537worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5538worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5539worker2 # [ 108.709505] systemd[1]: Starting scheduler-token.service...5540worker2 # [ 108.745579] systemd[1]: scheduler-token.service: Deactivated successfully.5541worker2 # [ 108.747082] systemd[1]: Finished scheduler-token.service.5542worker2 # [ 108.748529] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 37ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5543worker2 # [ 108.752831] systemd[1]: Starting Nix worker-protocol over gRPC...5544worker2 # [ 108.831345] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5545worker2 # [ 108.833542] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5546worker2 # [ 108.836791] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15547worker2 # [ 108.841052] nix-grpc-daemon[1258]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5548worker2 # [ 108.843222] nix-grpc-daemon[1258]: E0000 00:00:1790355088.132466 1258 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5549worker2 # [ 108.846583] nix-grpc-daemon[1258]: E0000 00:00:1790355088.137902 1258 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5550worker2 # [ 108.849518] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:28Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5551worker2 # [ 108.854286] systemd[1]: Started Nix worker-protocol over gRPC.5552worker2 # [ 108.859106] nix-grpc-daemon[1258]: E0000 00:00:1790355088.150216 1301 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5553worker2 # [ 108.871608] nix-grpc-daemon[1258]: E0000 00:00:1790355088.163089 1301 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5554worker2 # [ 108.875123] nix-grpc-daemon[1258]: E0000 00:00:1790355088.165365 1301 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5555worker1 # [ 108.898747] systemd[1]: Stopping Nix worker-protocol over gRPC...5556worker2 # [ 108.885672] nix-grpc-daemon[1258]: E0000 00:00:1790355088.177112 1301 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5557worker2 # [ 108.894724] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:28Z level=info event=scheduler_connected addr=lb:500515558worker1 # [ 108.911798] nix-grpc-daemon[511]: ts=2026-09-25T16:51:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:354885559worker1 # [ 108.914102] nix-grpc-daemon[511]: ts=2026-09-25T16:51:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05560worker1 # [ 109.766090] nix-grpc-daemon[511]: ts=2026-09-25T16:51:29Z level=info event=signal_exit5561lb # [ 110.087893] niks3-server[957]: 2026/09/25 16:51:29 INFO lead: released remote=192.168.1.3:406385562worker1 # [ 111.104140] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=scheduler_restarting5563lb # [ 111.087892] envoy[919]: [2026-09-25 16:51:30.355][919][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5564worker1 # [ 111.112853] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=scheduler_restarting addr=lb:500515565worker1 # [ 111.115573] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:354885566worker2 # [ 111.099748] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:30Z level=info event=scheduler_restarting addr=lb:500515567worker1 # [ 111.117101] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=worker_gone addr=192.168.1.3:500515568worker1 # [ 111.118706] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:354885569worker1 # [ 111.120237] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=worker_gone addr=192.168.1.4:500515570worker1 # [ 111.133266] nix-daemon[864]: reaped child process 866, status = succeeded5571worker2 # [ 111.300304] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:30Z level=info event=scheduler_connected addr=lb:500515572worker1 # [ 111.319780] nix-grpc-daemon[511]: ts=2026-09-25T16:51:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5573worker2 # [ 111.304163] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5574worker2 # [ 111.505285] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:30Z level=info event=scheduler_connected addr=lb:500515575worker2 # [ 111.509190] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5576worker2 # [ 111.710278] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_connected addr=lb:500515577worker2 # [ 111.714131] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5578worker2 # [ 111.914953] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_connected addr=lb:500515579worker2 # [ 111.918828] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5580worker2 # [ 112.120741] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_connected addr=lb:500515581worker2 # [ 112.124451] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5582worker2 # [ 112.325538] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_connected addr=lb:500515583worker2 # [ 112.329390] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5584worker1 # [ 112.376340] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5585worker1 # [ 112.377954] systemd[1]: Stopped Nix worker-protocol over gRPC.5586worker1 # [ 112.379867] systemd[1]: nix-grpc-daemon.service: Consumed 2.319s CPU time over 1min 44.914s wall clock time, 62.9M memory peak, 444.6K incoming IP traffic, 436.6K outgoing IP traffic.5587lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5588lb: must succeed: curl -sf localhost:9901/clusters5589lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5590lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5591client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25592worker2 # [ 112.530429] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_connected addr=lb:500515593worker2 # [ 112.534264] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5594worker2 # [ 112.563163] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461145595worker2 # [ 112.574756] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5596worker2 # [ 112.590921] nix-grpc-daemon[1258]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5597worker2 # [ 112.624176] nix-grpc-daemon[1258]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)5598worker2 # [ 112.735325] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_connected addr=lb:500515599worker2 # [ 112.739275] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5600worker2 # [ 112.810500] nix-grpc-daemon[1258]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 3/5)5601worker2 # [ 112.865450] nix-grpc-daemon[1258]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 285 ms (attempt 4/5)5602worker2 # [ 112.940074] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_connected addr=lb:500515603worker2 # [ 112.943865] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5604worker2 # [ 113.145833] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_connected addr=lb:500515605worker2 # [ 113.149647] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5606worker2 # [ 113.155622] nix-grpc-daemon[1258]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5607worker2 # [ 113.158693] nix-grpc-daemon[1258]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5608client # copying 2 paths...5609client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5610worker2 # [ 113.181441] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461145611client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5612worker2 # [ 113.192289] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5613worker2 # [ 113.213511] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=niks3_push_started pid=13185614worker2 # [ 113.319346] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.609Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5615worker2 # [ 113.351779] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_connected addr=lb:500515616worker2 # [ 113.355143] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5617worker2 # [ 113.410126] nix-daemon[858]: accepted connection from pid 1331, user nix-grpc-daemon (trusted)5618worker2 # [ 113.422178] nix-daemon[858]: reaped child process 1334, status = succeeded5619lb # [ 113.428897] niks3-server[957]: 2026/09/25 16:51:32 INFO Received push request method=POST path=/api/pushes5620worker2 # [ 113.439890] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.731Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5621worker2 # [ 113.441438] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.732Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5622worker2 # [ 113.443379] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.734Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5623lb # [ 113.485253] niks3-server[957]: 2026/09/25 16:51:32 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5624worker2 # [ 113.495570] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.786Z level=INFO msg="Uploading 2 narinfos"5625lb # [ 113.492995] niks3-server[957]: 2026/09/25 16:51:32 INFO Signed narinfos id=35 count=25626lb # [ 113.502673] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5627lb # [ 113.506847] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5628lb # [ 113.512070] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5629lb # [ 113.518750] niks3-server[957]: 2026/09/25 16:51:32 INFO Received complete push request method=POST path=/api/pushes/35/complete5630worker2 # [ 113.527921] nix-grpc-daemon[1318]: time=2026-09-25T16:51:32.819Z level=INFO msg="Upload complete. (209ms)"5631lb # [ 113.527185] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5632lb # [ 113.530318] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5633worker2 # [ 113.533125] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=0 paths=2 nar_bytes_in=3125634lb # [ 113.531842] niks3-server[957]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5635worker2 # [ 113.544504] nix-daemon[858]: reaped child process 1316, status = succeeded5636worker2 # [ 113.553897] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_connected addr=lb:500515637worker2 # [ 113.556754] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5638client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.15 seconds)5639worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5640worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5641worker1 # [ 113.619300] systemd[1]: Starting Nix worker-protocol over gRPC...5642worker1 # [ 113.692652] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5643worker1 # [ 113.694671] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5644worker1 # [ 113.697794] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15645worker1 # [ 113.703054] nix-grpc-daemon[1313]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5646worker1 # [ 113.704476] nix-grpc-daemon[1313]: E0000 00:00:1790355093.024733 1313 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5647worker1 # [ 113.707564] nix-grpc-daemon[1313]: E0000 00:00:1790355093.030117 1313 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5648worker1 # [ 113.712070] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5649worker1 # [ 113.715631] systemd[1]: Started Nix worker-protocol over gRPC.5650lb # [ 113.701268] niks3-server[957]: 2026/09/25 16:51:32 INFO lead: acquired remote=192.168.1.3:582845651worker1 # [ 113.721077] nix-grpc-daemon[1313]: E0000 00:00:1790355093.041639 1356 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5652worker1 # [ 113.723550] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_take_over5653lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5654lb: must succeed: curl -sf localhost:9901/clusters5655worker1 # [ 113.733677] nix-grpc-daemon[1313]: E0000 00:00:1790355093.056246 1356 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5656worker1 # [ 113.736144] nix-grpc-daemon[1313]: E0000 00:00:1790355093.058418 1356 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5657worker1 # [ 113.747531] nix-grpc-daemon[1313]: E0000 00:00:1790355093.070119 1356 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5658worker1 # [ 113.759626] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515659worker1 # [ 113.761174] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5660worker2 # [ 113.757994] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515661worker2 # [ 113.761852] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5663worker1 # [ 113.961856] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515664worker1 # [ 113.963650] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5665worker2 # [ 113.962790] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515666worker2 # [ 113.966622] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5667worker1 # [ 114.164210] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515668worker1 # [ 114.165963] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5669worker2 # [ 114.169182] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515670worker2 # [ 114.172392] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5671worker1 # [ 114.366606] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515672worker1 # [ 114.368464] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5673worker2 # [ 114.370163] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515674worker2 # [ 114.373994] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5675worker1 # [ 114.568769] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515676worker1 # [ 114.572243] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:606825677worker1 # [ 114.573733] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05678worker2 # [ 114.574572] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:33Z level=info event=scheduler_connected addr=lb:500515679worker1 # [ 114.594645] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=oidc_jwks provider=mock keys=15680worker1 # [ 114.596066] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:606825681worker1 # [ 114.597711] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05682lb: must succeed: curl -sf localhost:9901/clusters5683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5684lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds)5685(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.69 seconds)5686subtest: a niks3 restart does not cost the leader its role5687lb: must succeed: curl -sf localhost:9901/clusters5688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5689lb: must succeed: curl -sf localhost:9901/clusters5690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5691lb # [ 114.883276] niks3-server[957]: 2026/09/25 16:51:34 INFO Shutdown signal received, draining in-flight requests timeout=10s5692lb # [ 114.885239] systemd[1]: Stopping niks3 server...5693worker2 # [ 114.886976] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:34Z level=info event=scheduler_lock_lost status=2005694lb # [ 114.889794] niks3-server[957]: 2026/09/25 16:51:34 INFO lead: released remote=192.168.1.3:582845695worker1 # [ 114.908308] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:34Z level=info event=scheduler_lock_lost status=2005696lb # [ 114.899192] systemd[1]: niks3.service: Deactivated successfully.5697lb # [ 114.900800] systemd[1]: Stopped niks3 server.5698lb # [ 114.902148] systemd[1]: niks3.service: Consumed 778ms CPU time over 1min 37.041s wall clock time, 28.9M memory peak, 96.8K incoming IP traffic, 139.8K outgoing IP traffic.5699lb # [ 114.908901] systemd[1]: Starting niks3 server...5700lb # [ 114.958758] niks3-server[1170]: 2026/09/25 16:51:34 goose: no migrations to run. current version: 202609231200005701lb # [ 114.966986] niks3-server[1170]: 2026/09/25 16:51:34 OK 1_commit_pending_closure.sql (6.28ms)5702lb # [ 114.970385] niks3-server[1170]: 2026/09/25 16:51:34 OK 2_object_stats_trigger.sql (3.35ms)5703lb # [ 114.973782] niks3-server[1170]: 2026/09/25 16:51:34 OK 3_commit_push.sql (2.99ms)5704lb # [ 114.974726] niks3-server[1170]: 2026/09/25 16:51:34 goose: up to current file version: 35705lb # [ 114.976078] niks3-server[1170]: 2026/09/25 16:51:34 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5706lb # [ 114.986769] niks3-server[1170]: 2026/09/25 16:51:34 INFO Read proxy enabled — serving cache objects from S35707lb # [ 114.990224] niks3-server[1170]: 2026/09/25 16:51:34 INFO Using socket-activated listener address=0.0.0.0:57515708lb # [ 114.991482] niks3-server[1170]: 2026/09/25 16:51:34 INFO Starting HTTP server address=0.0.0.0:57515709lb # [ 114.992625] systemd[1]: Started niks3 server.5710lb # [ 114.993603] niks3-server[1170]: 2026/09/25 16:51:34 INFO systemd watchdog enabled interval=15s5711lb: waiting for unit niks3.service5712lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5713lb: must succeed: curl -sf localhost:9901/clusters5714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5715worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5716worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5717lb: must succeed: curl -sf localhost:9901/clusters5718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5719lb: must succeed: curl -sf localhost:9901/clusters5720lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5721worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5722worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.02 seconds)5723client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25724worker2 # [ 115.262596] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461145725client # don't know how to build these paths:5726client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5727worker2 # [ 115.271420] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461145728worker1 # [ 115.309677] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:34Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05729worker1 # [ 115.311977] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:34Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=15730worker1 # [ 115.316260] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965731worker1 # [ 115.321344] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5732worker1 # [ 115.328366] nix-grpc-daemon[1313]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)5733worker1 # [ 115.351560] nix-grpc-daemon[1313]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 2/5)5734worker1 # [ 115.483857] nix-grpc-daemon[1313]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 344 ms (attempt 3/5)5735worker1 # [ 115.828971] nix-grpc-daemon[1313]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 603 ms (attempt 4/5)5736lb # [ 115.893973] niks3-server[1170]: 2026/09/25 16:51:35 INFO lead: acquired remote=192.168.1.3:582985737worker1 # [ 116.433662] nix-grpc-daemon[1313]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5738worker1 # [ 116.435466] nix-grpc-daemon[1313]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5739worker1 # [ 116.447096] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606965740client # copying 1 paths to 'lb:50051'...5741worker1 # [ 116.456575] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606965742client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5743worker1 # [ 116.463242] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5744worker1 # [ 116.474424] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:35Z level=info event=niks3_push_started pid=13855745worker1 # [ 116.564084] nix-grpc-daemon[1385]: time=2026-09-25T16:51:35.886Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5746worker1 # [ 116.659401] nix-daemon[864]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5747worker1 # [ 116.671720] nix-daemon[864]: reaped child process 1400, status = succeeded5748lb # [ 116.660896] niks3-server[1170]: 2026/09/25 16:51:35 INFO Received push request method=POST path=/api/pushes5749worker1 # [ 116.692223] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.014Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5750worker1 # [ 116.693719] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.016Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5751lb # [ 116.711802] niks3-server[1170]: 2026/09/25 16:51:35 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5752lb # [ 116.724770] niks3-server[1170]: 2026/09/25 16:51:35 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5753lb # [ 116.740923] niks3-server[1170]: 2026/09/25 16:51:36 INFO Signed narinfos id=36 count=15754worker1 # [ 116.760569] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.083Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 116.748548] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5756lb # [ 116.753035] niks3-server[1170]: 2026/09/25 16:51:36 INFO Received complete push request method=POST path=/api/pushes/36/complete5757lb # [ 116.760674] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5758worker1 # [ 116.782111] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.104Z level=INFO msg="Upload complete. (214ms)"5759worker1 # [ 116.784353] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=5365760worker1 # [ 116.787828] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965761worker1 # [ 116.794742] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5762worker1 # [ 116.798084] nix-daemon[864]: reaped child process 1383, status = succeeded5763worker1 # [ 116.802677] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5764client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5765client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5766client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5767lb # [ 117.333162] envoy[919]: conn peer=[2001:db8:1::4]:38510 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3017 tx=1038 ms=749755768lb # [ 117.337288] envoy[919]: conn peer=[2001:db8:1::1]:53772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=145769lb # [ 117.340715] envoy[919]: conn peer=[2001:db8:1::3]:43848 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2413 tx=1148 ms=783385770lb # [ 117.344601] envoy[919]: conn peer=[2001:db8:1::1]:53784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=282 ms=9825771client # farm-a> LOG-farm-a-after-niks3-restart5772worker1 # [ 117.414742] nix-daemon[864]: reaped child process 1404, status = succeeded5773worker1 # [ 117.488619] nix-daemon[864]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)5774worker1 # [ 117.500886] nix-daemon[864]: reaped child process 1423, status = succeeded5775lb # [ 117.489997] niks3-server[1170]: 2026/09/25 16:51:36 INFO Received push request method=POST path=/api/pushes5776worker1 # [ 117.518494] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.841Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5777worker1 # [ 117.519978] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.842Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5778lb # [ 117.531482] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5779lb # [ 117.543586] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5780lb # [ 117.553947] niks3-server[1170]: 2026/09/25 16:51:36 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5781lb # [ 117.561052] niks3-server[1170]: 2026/09/25 16:51:36 INFO Signed narinfos id=37 count=15782worker1 # [ 117.582284] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.904Z level=INFO msg="Uploading 1 narinfos"5783lb # [ 117.569646] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5784lb # [ 117.572550] niks3-server[1170]: 2026/09/25 16:51:36 INFO Received complete push request method=POST path=/api/pushes/37/complete5785lb # [ 117.579116] niks3-server[1170]: 2026/09/25 16:51:36 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5786worker1 # [ 117.600751] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.923Z level=INFO msg="Upload complete. (193ms)"5787worker1 # [ 117.604835] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05788worker1 # [ 117.611152] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05789worker1 # [ 117.613504] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=15790worker1 # [ 117.615614] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965791worker1 # [ 117.620989] nix-daemon[864]: reaped child process 1402, status = succeeded5792worker1 # [ 117.622408] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606965793worker1 # [ 117.628431] nix-grpc-daemon[1385]: time=2026-09-25T16:51:36.951Z level=INFO msg="All 1 paths already cached"5794client # copying 1 paths to 'lb:50051'...5795worker1 # [ 117.633281] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606965796client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5797worker1 # [ 117.638355] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5798worker1 # [ 117.720757] nix-daemon[864]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5799worker1 # [ 117.733584] nix-daemon[864]: reaped child process 1432, status = succeeded5800lb # [ 117.722698] niks3-server[1170]: 2026/09/25 16:51:36 INFO Received push request method=POST path=/api/pushes5801worker1 # [ 117.750729] nix-grpc-daemon[1385]: time=2026-09-25T16:51:37.073Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5802worker1 # [ 117.752757] nix-grpc-daemon[1385]: time=2026-09-25T16:51:37.074Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5803lb # [ 117.756761] niks3-server[1170]: 2026/09/25 16:51:37 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5804worker1 # [ 117.784873] nix-grpc-daemon[1385]: time=2026-09-25T16:51:37.107Z level=INFO msg="Uploading 1 narinfos"5805lb # [ 117.766921] niks3-server[1170]: 2026/09/25 16:51:37 INFO Signed narinfos id=38 count=15806lb # [ 117.773270] niks3-server[1170]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5807lb # [ 117.777876] niks3-server[1170]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5808lb # [ 117.790584] niks3-server[1170]: 2026/09/25 16:51:37 INFO Received complete push request method=POST path=/api/pushes/38/complete5809worker1 # [ 117.815712] nix-grpc-daemon[1385]: time=2026-09-25T16:51:37.138Z level=INFO msg="Upload complete. (171ms)"5810lb # [ 117.798419] niks3-server[1170]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5811worker1 # [ 117.817851] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=6565812worker1 # [ 117.820548] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965813worker1 # [ 117.825564] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5814worker1 # [ 117.830331] nix-daemon[864]: reaped child process 1427, status = succeeded5815worker1 # [ 117.835169] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5816client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5817client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5818worker1 # [ 118.835458] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:607125819client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5820client # farm-b> LOG-farm-b-after-niks3-restart5821worker1 # [ 119.250492] nix-daemon[864]: reaped child process 1436, status = succeeded5822worker1 # [ 119.324227] nix-daemon[864]: accepted connection from pid 1452, user nix-grpc-daemon (trusted)5823worker1 # [ 119.335922] nix-daemon[864]: reaped child process 1455, status = succeeded5824lb # [ 119.326895] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received push request method=POST path=/api/pushes5825worker1 # [ 119.351880] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.674Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5826worker1 # [ 119.353948] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.676Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5827lb # [ 119.362149] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5828lb # [ 119.372266] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5829worker1 # [ 119.400727] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.723Z level=INFO msg="Uploading 1 narinfos"5830lb # [ 119.382782] niks3-server[1170]: 2026/09/25 16:51:38 INFO Signed narinfos id=39 count=15831lb # [ 119.384733] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5832lb # [ 119.393052] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5833lb # [ 119.404968] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received complete push request method=POST path=/api/pushes/39/complete5834lb # [ 119.410415] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5835worker1 # [ 119.433621] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.756Z level=INFO msg="Upload complete. (192ms)"5836worker1 # [ 119.435979] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05837worker1 # [ 119.442342] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05838worker1 # [ 119.445189] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=15839worker1 # [ 119.447118] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965840worker1 # [ 119.453184] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:606965841worker1 # [ 119.455360] nix-daemon[864]: reaped child process 1434, status = succeeded5842worker1 # [ 119.459605] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.782Z level=INFO msg="All 2 paths already cached"5843client # copying 1 paths to 'lb:50051'...5844worker1 # [ 119.464760] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:606965845client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5846worker1 # [ 119.470874] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5847worker1 # [ 119.552215] nix-daemon[864]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)5848worker1 # [ 119.566922] nix-daemon[864]: reaped child process 1464, status = succeeded5849lb # [ 119.553753] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received push request method=POST path=/api/pushes5850worker1 # [ 119.580370] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5851worker1 # [ 119.582574] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.904Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5852lb # [ 119.585494] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5853worker1 # [ 119.611500] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.933Z level=INFO msg="Uploading 1 narinfos"5854lb # [ 119.593395] niks3-server[1170]: 2026/09/25 16:51:38 INFO Signed narinfos id=40 count=15855lb # [ 119.597365] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5856lb # [ 119.602242] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5857lb # [ 119.608980] niks3-server[1170]: 2026/09/25 16:51:38 INFO Received complete push request method=POST path=/api/pushes/40/complete5858worker1 # [ 119.632520] nix-grpc-daemon[1385]: time=2026-09-25T16:51:38.955Z level=INFO msg="Upload complete. (157ms)"5859lb # [ 119.616678] niks3-server[1170]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5860worker1 # [ 119.634673] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=0 paths=1 nar_bytes_in=7845861worker1 # [ 119.638387] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:606965862worker1 # [ 119.644793] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5863worker1 # [ 119.647081] nix-daemon[864]: reaped child process 1459, status = succeeded5864worker1 # [ 119.652244] nix-daemon[864]: accepted connection from pid 1313, user nix-grpc-daemon (trusted)5865client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5866client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5867client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5868client # farm-top> LOG-farm-top-after-niks3-restart5869worker1 # [ 120.667424] nix-daemon[864]: reaped child process 1468, status = succeeded5870worker1 # [ 120.741199] nix-daemon[864]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)5871worker1 # [ 120.753830] nix-daemon[864]: reaped child process 1488, status = succeeded5872lb # [ 120.741651] niks3-server[1170]: 2026/09/25 16:51:40 INFO Received push request method=POST path=/api/pushes5873worker1 # [ 120.769037] nix-grpc-daemon[1385]: time=2026-09-25T16:51:40.091Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5874worker1 # [ 120.771442] nix-grpc-daemon[1385]: time=2026-09-25T16:51:40.093Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5875lb # [ 120.784693] niks3-server[1170]: 2026/09/25 16:51:40 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5876lb # [ 120.790757] niks3-server[1170]: 2026/09/25 16:51:40 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5877lb # [ 120.796808] niks3-server[1170]: 2026/09/25 16:51:40 INFO Signed narinfos id=41 count=15878worker1 # [ 120.818659] nix-grpc-daemon[1385]: time=2026-09-25T16:51:40.141Z level=INFO msg="Uploading 1 narinfos"5879lb # [ 120.804879] niks3-server[1170]: 2026/09/25 16:51:40 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5880lb # [ 120.815706] niks3-server[1170]: 2026/09/25 16:51:40 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5881lb # [ 120.819059] niks3-server[1170]: 2026/09/25 16:51:40 INFO Received complete push request method=POST path=/api/pushes/41/complete5882lb # [ 120.824097] niks3-server[1170]: 2026/09/25 16:51:40 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5883worker1 # [ 120.845055] nix-grpc-daemon[1385]: time=2026-09-25T16:51:40.167Z level=INFO msg="Upload complete. (186ms)"5884worker1 # [ 120.847657] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60696 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05885worker1 # [ 120.856130] nix-daemon[864]: reaped child process 1466, status = succeeded5886client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.75 seconds)5887client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5888client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)5889(finished: subtest: a niks3 restart does not cost the leader its role, in 6.17 seconds)5890subtest: scheduler down: the other node takes the lock and keeps it5891lb: must succeed: curl -sf localhost:9901/clusters5892lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5893lb: must succeed: curl -sf localhost:9901/clusters5894lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5895lb: must succeed: curl -sf localhost:9901/clusters5896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5897lb: must succeed: curl -sf localhost:9901/clusters5898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5899worker1 # [ 121.131220] systemd[1]: Stopping Nix worker-protocol over gRPC...5900worker1 # [ 121.714619] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:41Z level=info event=signal_exit5901lb # [ 121.898904] niks3-server[1170]: 2026/09/25 16:51:41 INFO lead: released remote=192.168.1.3:582985902worker1 # [ 122.914505] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=scheduler_restarting5903lb # [ 122.897712] envoy[919]: [2026-09-25 16:51:42.166][919][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5904worker1 # [ 122.922406] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=scheduler_restarting addr=lb:500515905worker1 # [ 122.926135] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:606825906worker2 # [ 122.908727] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_restarting addr=lb:500515907worker1 # [ 122.927692] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=worker_gone addr=192.168.1.3:500515908worker1 # [ 122.929086] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:606825909worker1 # [ 122.930635] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=worker_gone addr=192.168.1.4:500515910worker1 # [ 122.943516] nix-daemon[864]: reaped child process 1375, status = succeeded5911worker2 # [ 123.109379] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_connected addr=lb:500515912worker1 # [ 123.128405] nix-grpc-daemon[1313]: ts=2026-09-25T16:51:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5913worker2 # [ 123.113340] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker2 # [ 123.314357] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_connected addr=lb:500515915worker2 # [ 123.318179] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5916worker2 # [ 123.519123] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_connected addr=lb:500515917worker2 # [ 123.522979] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918worker2 # [ 123.724462] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_connected addr=lb:500515919worker2 # [ 123.728337] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker2 # [ 123.931269] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 123.934565] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker2 # [ 124.132286] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_connected addr=lb:500515923worker2 # [ 124.136297] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker2 # [ 124.337469] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 124.341465] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5926worker2 # [ 124.542419] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 124.546417] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker2 # [ 124.747599] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_connected addr=lb:500515929worker2 # [ 124.751618] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930worker2 # [ 124.953337] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_connected addr=lb:500515931worker2 # [ 124.957130] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932worker1 # [ 125.096632] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5933worker1 # [ 125.098284] systemd[1]: Stopped Nix worker-protocol over gRPC.5934worker1 # [ 125.100110] systemd[1]: nix-grpc-daemon.service: Consumed 649ms CPU time over 11.482s wall clock time, 36.8M memory peak, 185.6K incoming IP traffic, 89.4K outgoing IP traffic.5935worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5936worker2 # [ 125.157483] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_connected addr=lb:500515937worker2 # [ 125.159621] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5938worker2 # [ 125.360921] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_connected addr=lb:500515939worker2 # [ 125.364832] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5940worker2 # [ 125.566255] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_connected addr=lb:500515941worker2 # [ 125.570193] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5942worker2 # [ 125.771323] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515943worker2 # [ 125.775313] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5944worker2 # [ 125.894268] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_take_over5945lb # [ 125.891761] niks3-server[1170]: 2026/09/25 16:51:45 INFO lead: acquired remote=192.168.1.4:565285946worker2 # [ 125.976794] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515947worker2 # [ 125.980588] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker2 # [ 126.181860] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515949worker2 # [ 126.186338] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.12 seconds)5951worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5952worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5953worker2 # [ 126.386845] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515954worker2 # [ 126.390772] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5955worker2 # [ 126.591686] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515956worker2 # [ 126.609159] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=oidc_jwks provider=mock keys=15957worker2 # [ 126.612602] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:543465958worker2 # [ 126.616519] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05959worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5960worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5961client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25962lb # [ 127.334282] envoy[919]: conn peer=[2001:db8:1::1]:54912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9653 tx=4392 ms=55865963lb # [ 127.338128] envoy[919]: conn peer=[2001:db8:1::3]:42326 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1968 tx=702 ms=93715964worker2 # [ 127.420857] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461145965client # don't know how to build these paths:5966client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5967worker2 # [ 127.429099] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461145968worker2 # [ 127.451162] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05969worker2 # [ 127.456621] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15970worker2 # [ 127.461984] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461145971worker2 # [ 127.473226] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461145972client # copying 1 paths to 'lb:50051'...5973client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5974worker2 # [ 127.485124] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461145975worker2 # [ 127.496237] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5976worker2 # [ 127.595192] nix-daemon[858]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)5977worker2 # [ 127.607347] nix-daemon[858]: reaped child process 1366, status = succeeded5978lb # [ 127.614325] niks3-server[1170]: 2026/09/25 16:51:46 INFO Received push request method=POST path=/api/pushes5979worker2 # [ 127.626691] nix-grpc-daemon[1318]: time=2026-09-25T16:51:46.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5980worker2 # [ 127.628938] nix-grpc-daemon[1318]: time=2026-09-25T16:51:46.919Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5981lb # [ 127.650431] niks3-server[1170]: 2026/09/25 16:51:46 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5982lb # [ 127.657323] niks3-server[1170]: 2026/09/25 16:51:46 INFO Signed narinfos id=42 count=15983worker2 # [ 127.663295] nix-grpc-daemon[1318]: time=2026-09-25T16:51:46.954Z level=INFO msg="Uploading 1 narinfos"5984lb # [ 127.661070] niks3-server[1170]: 2026/09/25 16:51:46 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5985lb # [ 127.668195] niks3-server[1170]: 2026/09/25 16:51:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5986lb # [ 127.679522] niks3-server[1170]: 2026/09/25 16:51:46 INFO Received complete push request method=POST path=/api/pushes/42/complete5987lb # [ 127.688208] niks3-server[1170]: 2026/09/25 16:51:46 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5988worker2 # [ 127.693852] nix-grpc-daemon[1318]: time=2026-09-25T16:51:46.985Z level=INFO msg="Upload complete. (187ms)"5989worker2 # [ 127.698445] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=0 paths=1 nar_bytes_in=5205990worker2 # [ 127.703839] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461145991worker2 # [ 127.715249] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5992worker2 # [ 127.721664] nix-daemon[858]: reaped child process 1361, status = succeeded5993worker2 # [ 127.727230] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5994client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5995client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5996client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5997client # farm-a> LOG-farm-a-on-standby5998worker2 # [ 128.543351] nix-daemon[858]: reaped child process 1370, status = succeeded5999worker2 # [ 128.620209] nix-daemon[858]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)6000worker2 # [ 128.632977] nix-daemon[858]: reaped child process 1389, status = succeeded6001lb # [ 128.638550] niks3-server[1170]: 2026/09/25 16:51:47 INFO Received push request method=POST path=/api/pushes6002worker2 # [ 128.649742] nix-grpc-daemon[1318]: time=2026-09-25T16:51:47.941Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6003worker2 # [ 128.651382] nix-grpc-daemon[1318]: time=2026-09-25T16:51:47.942Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6004lb # [ 128.678462] niks3-server[1170]: 2026/09/25 16:51:47 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign6005lb # [ 128.682853] niks3-server[1170]: 2026/09/25 16:51:47 INFO Signed narinfos id=43 count=16006worker2 # [ 128.688463] nix-grpc-daemon[1318]: time=2026-09-25T16:51:47.979Z level=INFO msg="Uploading 1 narinfos"6007lb # [ 128.694910] niks3-server[1170]: 2026/09/25 16:51:47 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6008lb # [ 128.705746] niks3-server[1170]: 2026/09/25 16:51:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6009lb # [ 128.711570] niks3-server[1170]: 2026/09/25 16:51:47 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6010lb # [ 128.718700] niks3-server[1170]: 2026/09/25 16:51:47 INFO Received complete push request method=POST path=/api/pushes/43/complete6011lb # [ 128.723226] niks3-server[1170]: 2026/09/25 16:51:47 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6012worker2 # [ 128.725355] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.016Z level=INFO msg="Upload complete. (190ms)"6013worker2 # [ 128.737473] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06014worker2 # [ 128.745725] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06015worker2 # [ 128.749936] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16016worker2 # [ 128.753481] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146017worker2 # [ 128.761780] nix-daemon[858]: reaped child process 1368, status = succeeded6018worker2 # [ 128.763253] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461146019worker2 # [ 128.771316] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.062Z level=INFO msg="All 1 paths already cached"6020client # copying 1 paths to 'lb:50051'...6021worker2 # [ 128.776791] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461146022client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6023worker2 # [ 128.782649] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6024worker2 # [ 128.869716] nix-daemon[858]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)6025worker2 # [ 128.882185] nix-daemon[858]: reaped child process 1398, status = succeeded6026lb # [ 128.888076] niks3-server[1170]: 2026/09/25 16:51:48 INFO Received push request method=POST path=/api/pushes6027worker2 # [ 128.898670] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6028worker2 # [ 128.900647] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.191Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6029lb # [ 128.916451] niks3-server[1170]: 2026/09/25 16:51:48 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign6030lb # [ 128.921265] niks3-server[1170]: 2026/09/25 16:51:48 INFO Signed narinfos id=44 count=16031worker2 # [ 128.927559] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.218Z level=INFO msg="Uploading 1 narinfos"6032lb # [ 128.931077] niks3-server[1170]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6033lb # [ 128.941054] niks3-server[1170]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6034lb # [ 128.949136] niks3-server[1170]: 2026/09/25 16:51:48 INFO Received complete push request method=POST path=/api/pushes/44/complete6035worker2 # [ 128.959424] nix-grpc-daemon[1318]: time=2026-09-25T16:51:48.250Z level=INFO msg="Upload complete. (172ms)"6036lb # [ 128.958978] niks3-server[1170]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6037worker2 # [ 128.962769] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=0 paths=1 nar_bytes_in=6406038worker2 # [ 128.967276] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146039worker2 # [ 128.977752] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6040worker2 # [ 128.981037] nix-daemon[858]: reaped child process 1393, status = succeeded6041worker2 # [ 128.985820] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6042client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6043client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6044client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6045client # farm-b> LOG-farm-b-on-standby6046worker2 # [ 130.222990] nix-daemon[858]: reaped child process 1402, status = succeeded6047worker2 # [ 130.300158] nix-daemon[858]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)6048worker2 # [ 130.312673] nix-daemon[858]: reaped child process 1421, status = succeeded6049lb # [ 130.318840] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received push request method=POST path=/api/pushes6050worker2 # [ 130.329905] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.621Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6051worker2 # [ 130.331726] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.622Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6052lb # [ 130.362377] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign6053lb # [ 130.371316] niks3-server[1170]: 2026/09/25 16:51:49 INFO Signed narinfos id=45 count=16054worker2 # [ 130.377521] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.668Z level=INFO msg="Uploading 1 narinfos"6055lb # [ 130.379068] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6056lb # [ 130.384936] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6057lb # [ 130.400146] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received complete push request method=POST path=/api/pushes/45/complete6058lb # [ 130.406089] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6059worker2 # [ 130.411445] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.702Z level=INFO msg="Upload complete. (216ms)"6060lb # [ 130.411095] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6061worker2 # [ 130.418790] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06062worker2 # [ 130.432190] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06063worker2 # [ 130.435561] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16064worker2 # [ 130.440204] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146065worker2 # [ 130.450553] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461146066worker2 # [ 130.453716] nix-daemon[858]: reaped child process 1400, status = succeeded6067worker2 # [ 130.457764] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.749Z level=INFO msg="All 2 paths already cached"6068client # copying 1 paths to 'lb:50051'...6069worker2 # [ 130.462871] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461146070client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6071worker2 # [ 130.469063] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6072worker2 # [ 130.553600] nix-daemon[858]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)6073worker2 # [ 130.566443] nix-daemon[858]: reaped child process 1430, status = succeeded6074lb # [ 130.571965] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received push request method=POST path=/api/pushes6075worker2 # [ 130.584700] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.876Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6076worker2 # [ 130.586769] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.877Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6077lb # [ 130.601390] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6078worker2 # [ 130.611489] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.902Z level=INFO msg="Uploading 1 narinfos"6079lb # [ 130.608651] niks3-server[1170]: 2026/09/25 16:51:49 INFO Signed narinfos id=46 count=16080lb # [ 130.612879] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6081lb # [ 130.623588] niks3-server[1170]: 2026/09/25 16:51:49 INFO Received complete push request method=POST path=/api/pushes/46/complete6082lb # [ 130.626112] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6083lb # [ 130.628571] niks3-server[1170]: 2026/09/25 16:51:49 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6084worker2 # [ 130.634458] nix-grpc-daemon[1318]: time=2026-09-25T16:51:49.925Z level=INFO msg="Upload complete. (160ms)"6085worker2 # [ 130.636767] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=0 paths=1 nar_bytes_in=7686086worker2 # [ 130.641550] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146087worker2 # [ 130.651241] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6088worker2 # [ 130.654271] nix-daemon[858]: reaped child process 1425, status = succeeded6089worker2 # [ 130.658229] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6090client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6091client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6092worker2 # [ 130.913208] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:50Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:543506093client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6094client # farm-top> LOG-farm-top-on-standby6095worker2 # [ 131.471756] nix-daemon[858]: reaped child process 1434, status = succeeded6096worker2 # [ 131.549872] nix-daemon[858]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)6097worker2 # [ 131.560807] nix-daemon[858]: reaped child process 1453, status = succeeded6098lb # [ 131.567763] niks3-server[1170]: 2026/09/25 16:51:50 INFO Received push request method=POST path=/api/pushes6099worker2 # [ 131.578915] nix-grpc-daemon[1318]: time=2026-09-25T16:51:50.870Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6100worker2 # [ 131.580708] nix-grpc-daemon[1318]: time=2026-09-25T16:51:50.871Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6101lb # [ 131.605095] niks3-server[1170]: 2026/09/25 16:51:50 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6102lb # [ 131.615847] niks3-server[1170]: 2026/09/25 16:51:50 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6103worker2 # [ 131.625298] nix-grpc-daemon[1318]: time=2026-09-25T16:51:50.916Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 131.622215] niks3-server[1170]: 2026/09/25 16:51:50 INFO Signed narinfos id=47 count=16105lb # [ 131.626433] niks3-server[1170]: 2026/09/25 16:51:50 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6106lb # [ 131.633810] niks3-server[1170]: 2026/09/25 16:51:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6107lb # [ 131.640034] niks3-server[1170]: 2026/09/25 16:51:50 INFO Received complete push request method=POST path=/api/pushes/47/complete6108worker2 # [ 131.646388] nix-grpc-daemon[1318]: time=2026-09-25T16:51:50.937Z level=INFO msg="Upload complete. (182ms)"6109lb # [ 131.645428] niks3-server[1170]: 2026/09/25 16:51:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6110worker2 # [ 131.651570] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06111worker2 # [ 131.665407] nix-daemon[858]: reaped child process 1432, status = succeeded6112client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.41 seconds)6113client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6114client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6115worker1 # [ 131.842220] systemd[1]: Starting Nix worker-protocol over gRPC...6116worker1 # [ 131.916826] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6117worker1 # [ 131.918862] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6118worker1 # [ 131.922356] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16119worker1 # [ 131.926956] nix-grpc-daemon[1506]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6120worker1 # [ 131.928344] nix-grpc-daemon[1506]: E0000 00:00:1790355111.248751 1506 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6121worker1 # [ 131.931615] nix-grpc-daemon[1506]: E0000 00:00:1790355111.254183 1506 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6122worker1 # [ 131.934750] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6123worker1 # [ 131.938679] systemd[1]: Started Nix worker-protocol over gRPC.6124worker1 # [ 131.945086] nix-grpc-daemon[1506]: E0000 00:00:1790355111.265674 1546 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6125lb: must succeed: curl -sf localhost:9901/clusters6126worker1 # [ 131.956749] nix-grpc-daemon[1506]: E0000 00:00:1790355111.279383 1546 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6127worker1 # [ 131.960151] nix-grpc-daemon[1506]: E0000 00:00:1790355111.281733 1546 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6128worker1 # [ 131.970853] nix-grpc-daemon[1506]: E0000 00:00:1790355111.293494 1546 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6129worker1 # [ 131.980518] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=info event=scheduler_connected addr=lb:500516130worker2 # [ 131.966683] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:543466131worker2 # [ 131.972257] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06132lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6133worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6134worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6135lb: must succeed: curl -sf localhost:9901/clusters6136lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6137lb: must succeed: curl -sf localhost:9901/clusters6138lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6139worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6140worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6141(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.13 seconds)6142subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6143client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6144client # [ 132.158927] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6145client # Running as unit: rs.service; invocation ID: df1fcee639dc4976a7082cb98a2fa98c6146client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6147worker1 # [ 132.304988] nix-grpc-daemon[1506]: ts=2026-09-25T16:51:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502386148worker1 # [ 132.308883] nix-daemon[864]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)6149client # [ 132.300350] nix[1380]: don't know how to build these paths:6150client # [ 132.301287] nix[1380]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6151worker2 # [ 132.309849] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06152worker2 # [ 132.316589] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16153worker2 # [ 132.321559] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146154worker2 # [ 132.332243] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461146155client # [ 132.341113] nix[1380]: copying 1 paths to 'lb:50051'...6156client # [ 132.341884] nix[1380]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6157worker2 # [ 132.342877] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461146158worker2 # [ 132.351300] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6159worker2 # [ 132.449842] nix-daemon[858]: accepted connection from pid 1466, user nix-grpc-daemon (trusted)6160worker2 # [ 132.464448] nix-daemon[858]: reaped child process 1469, status = succeeded6161lb # [ 132.468922] niks3-server[1170]: 2026/09/25 16:51:51 INFO Received push request method=POST path=/api/pushes6162worker2 # [ 132.479533] nix-grpc-daemon[1318]: time=2026-09-25T16:51:51.771Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6163worker2 # [ 132.481434] nix-grpc-daemon[1318]: time=2026-09-25T16:51:51.772Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6164lb # [ 132.496228] niks3-server[1170]: 2026/09/25 16:51:51 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6165worker2 # [ 132.504468] nix-grpc-daemon[1318]: time=2026-09-25T16:51:51.795Z level=INFO msg="Uploading 1 narinfos"6166lb # [ 132.501459] niks3-server[1170]: 2026/09/25 16:51:51 INFO Signed narinfos id=48 count=16167lb # [ 132.513661] niks3-server[1170]: 2026/09/25 16:51:51 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6168lb # [ 132.522202] niks3-server[1170]: 2026/09/25 16:51:51 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6169lb # [ 132.528212] niks3-server[1170]: 2026/09/25 16:51:51 INFO Received complete push request method=POST path=/api/pushes/48/complete6170worker2 # [ 132.538851] nix-grpc-daemon[1318]: time=2026-09-25T16:51:51.830Z level=INFO msg="Upload complete. (174ms)"6171lb # [ 132.539108] niks3-server[1170]: 2026/09/25 16:51:51 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6172worker2 # [ 132.541180] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=0 paths=1 nar_bytes_in=4166173worker2 # [ 132.545432] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461146174worker2 # [ 132.556573] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6175worker2 # [ 132.560272] nix-daemon[858]: reaped child process 1464, status = succeeded6176worker2 # [ 132.564843] nix-daemon[858]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)6177client # [ 132.564129] nix[1380]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6178lb: must succeed: curl -sf localhost:9901/clusters6179lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6180lb: must succeed: curl -sf localhost:9901/clusters6181lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6182lb: must succeed: curl -sf localhost:9901/clusters6183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6184lb: must succeed: curl -sf localhost:9901/clusters6185lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6186worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6187worker1 # [ 133.715292] systemd-journald[283]: Received client request to rotate journal, rotating.6188worker1 # [ 133.733637] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/bde8db7c60b143e1a933379f39c65834.6189worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6190worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6191worker2 # [ 133.627811] systemd[1]: Stopping Nix worker-protocol over gRPC...6192worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6193client: must succeed: sleep 5.06194worker2 # [ 133.859105] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:53Z level=info event=draining6195lb # [ 137.335308] envoy[919]: conn peer=[2001:db8:1::1]:39890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9556 tx=4363 ms=42476196client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6197worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6198worker2 # [ 139.004418] nix-daemon[858]: reaped child process 1473, status = succeeded6199worker2 # [ 139.088182] nix-daemon[858]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)6200worker2 # [ 139.100635] nix-daemon[858]: reaped child process 1513, status = succeeded6201lb # [ 139.106776] niks3-server[1170]: 2026/09/25 16:51:58 INFO Received push request method=POST path=/api/pushes6202worker2 # [ 139.117915] nix-grpc-daemon[1318]: time=2026-09-25T16:51:58.409Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6203worker2 # [ 139.119386] nix-grpc-daemon[1318]: time=2026-09-25T16:51:58.410Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6204lb # [ 139.150109] niks3-server[1170]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6205lb # [ 139.156731] niks3-server[1170]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6206lb # [ 139.163415] niks3-server[1170]: 2026/09/25 16:51:58 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6207worker2 # [ 139.173927] nix-grpc-daemon[1318]: time=2026-09-25T16:51:58.465Z level=INFO msg="Uploading 1 narinfos"6208lb # [ 139.171499] niks3-server[1170]: 2026/09/25 16:51:58 INFO Signed narinfos id=49 count=16209lb # [ 139.176240] niks3-server[1170]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6210lb # [ 139.187293] niks3-server[1170]: 2026/09/25 16:51:58 INFO Received complete push request method=POST path=/api/pushes/49/complete6211worker2 # [ 139.194360] nix-grpc-daemon[1318]: time=2026-09-25T16:51:58.485Z level=INFO msg="Upload complete. (217ms)"6212lb # [ 139.193283] niks3-server[1170]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6213worker2 # [ 139.201678] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46114 duration_s=6 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06214worker2 # [ 139.217483] nix-daemon[858]: reaped child process 1471, status = succeeded6215client # [ 139.237333] systemd[1]: rs.service: Deactivated successfully.6216client # [ 139.241672] systemd[1]: rs.service: Consumed 127ms CPU time over 7.080s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.6217worker2 # [ 139.860928] nix-grpc-daemon[1258]: ts=2026-09-25T16:51:59Z level=info event=signal_exit6218lb # [ 139.898297] niks3-server[1170]: 2026/09/25 16:51:59 INFO lead: released remote=192.168.1.4:565286219worker2 # [ 140.902448] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting6220lb # [ 140.898266] envoy[919]: [2026-09-25 16:52:00.166][919][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6221worker2 # [ 140.909453] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting addr=lb:500516222worker1 # [ 140.924885] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting addr=lb:500516223worker2 # [ 140.915671] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:543466224worker2 # [ 140.919389] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=worker_gone addr=192.168.1.3:500516225worker2 # [ 140.922929] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:543466226worker2 # [ 140.926350] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=worker_gone addr=192.168.1.4:500516227worker2 # [ 140.948667] nix-daemon[858]: reaped child process 1310, status = succeeded6228worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)6229client: waiting for success: ! systemctl is-active rs6230client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6231client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6232client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6233client: must fail: journalctl -u rs | grep -q 'reconnecting'6234client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6235lb: must succeed: curl -sf localhost:9901/clusters6236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6237worker1 # [ 141.129553] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_connected addr=lb:500516238worker1 # [ 141.133668] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6239worker2 # [ 141.122976] nix-grpc-daemon[1258]: ts=2026-09-25T16:52:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6240worker1 # [ 141.334430] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_connected addr=lb:500516241worker1 # [ 141.338543] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6242worker1 # [ 141.539443] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_connected addr=lb:500516243worker1 # [ 141.543603] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6244worker1 # [ 141.744448] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516245worker1 # [ 141.748556] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6246lb # [ 141.927340] niks3-server[1170]: 2026/09/25 16:52:01 INFO lead: acquired remote=192.168.1.3:397846247worker1 # [ 141.949321] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_take_over6248worker1 # [ 141.952919] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516249worker1 # [ 141.956783] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6250worker2 # [ 142.067194] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6251worker2 # [ 142.070165] systemd[1]: Stopped Nix worker-protocol over gRPC.6252worker2 # [ 142.072526] systemd[1]: nix-grpc-daemon.service: Consumed 1.175s CPU time over 33.313s wall clock time, 39.1M memory peak, 346.2K incoming IP traffic, 201.7K outgoing IP traffic.6253worker2 # [ 142.087448] systemd[1]: Starting scheduler-token.service...6254lb: must succeed: curl -sf localhost:9901/clusters6255worker2 # [ 142.139845] systemd[1]: scheduler-token.service: Deactivated successfully.6256worker2 # [ 142.142191] systemd[1]: Finished scheduler-token.service.6257worker2 # [ 142.142974] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 53ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6258worker1 # [ 142.157775] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516259worker2 # [ 142.146701] systemd[1]: Starting Nix worker-protocol over gRPC...6260worker1 # [ 142.161782] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6262worker2 # [ 142.226344] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6263worker2 # [ 142.228468] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6264worker2 # [ 142.231692] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16265worker2 # [ 142.236188] nix-grpc-daemon[1524]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6266worker2 # [ 142.237522] nix-grpc-daemon[1524]: E0000 00:00:1790355121.527304 1524 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6267worker2 # [ 142.241454] nix-grpc-daemon[1524]: E0000 00:00:1790355121.532855 1524 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6268worker2 # [ 142.245287] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6269worker2 # [ 142.249064] systemd[1]: Started Nix worker-protocol over gRPC.6270worker2 # [ 142.252097] nix-grpc-daemon[1524]: E0000 00:00:1790355121.543160 1566 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6271worker2 # [ 142.263791] nix-grpc-daemon[1524]: E0000 00:00:1790355121.555216 1566 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6272worker2 # [ 142.268057] nix-grpc-daemon[1524]: E0000 00:00:1790355121.557498 1566 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6273worker2 # [ 142.277744] nix-grpc-daemon[1524]: E0000 00:00:1790355121.569185 1566 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6274worker2 # [ 142.287402] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516275worker2 # [ 142.288906] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6276worker1 # [ 142.362693] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516277worker1 # [ 142.366721] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6278worker2 # [ 142.490369] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516279worker2 # [ 142.494390] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6280worker1 # [ 142.567606] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516281worker1 # [ 142.571584] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282worker2 # [ 142.695198] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500516283worker1 # [ 142.726174] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=oidc_jwks provider=mock keys=16284worker1 # [ 142.729959] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:527506285worker1 # [ 142.733972] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06286worker1 # [ 142.772175] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500516287worker1 # [ 142.776310] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:527506288worker1 # [ 142.779850] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06289lb: must succeed: curl -sf localhost:9901/clusters6290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6291worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6292worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6293(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.13 seconds)6294subtest: scheduler restart mid-build: build finishes, no double build6295client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6296client # [ 143.287679] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6297client # Running as unit: mid.service; invocation ID: 01008c53d5d744d690c4521b371872ca6298client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6299worker1 # [ 143.440603] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502386300client # [ 143.431101] nix[1416]: don't know how to build these paths:6301client # [ 143.432026] nix[1416]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6302worker1 # [ 143.454880] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06303worker1 # [ 143.459626] nix-grpc-daemon[1506]: ts=2026-09-25T16:52:02Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16304worker2 # [ 143.446730] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486266305worker2 # [ 143.457776] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6306worker2 # [ 143.473836] nix-grpc-daemon[1524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)6307worker2 # [ 143.545476] nix-grpc-daemon[1524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)6308worker2 # [ 143.595131] nix-grpc-daemon[1524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 3/5)6309worker2 # [ 143.683229] nix-grpc-daemon[1524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 355 ms (attempt 4/5)6310worker2 # [ 144.040250] nix-grpc-daemon[1524]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6311worker2 # [ 144.043978] nix-grpc-daemon[1524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6312worker2 # [ 144.061496] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486266313client # [ 144.070904] nix[1416]: copying 1 paths to 'lb:50051'...6314client # [ 144.075944] nix[1416]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6315worker2 # [ 144.076737] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486266316worker2 # [ 144.086355] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6317worker2 # [ 144.108628] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:03Z level=info event=niks3_push_started pid=15886318worker2 # [ 144.214294] nix-grpc-daemon[1588]: time=2026-09-25T16:52:03.505Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6319worker2 # [ 144.304136] nix-daemon[858]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)6320worker2 # [ 144.315814] nix-daemon[858]: reaped child process 1604, status = succeeded6321lb # [ 144.322383] niks3-server[1170]: 2026/09/25 16:52:03 INFO Received push request method=POST path=/api/pushes6322worker2 # [ 144.335576] nix-grpc-daemon[1588]: time=2026-09-25T16:52:03.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6323worker2 # [ 144.337202] nix-grpc-daemon[1588]: time=2026-09-25T16:52:03.628Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6324lb # [ 144.362684] niks3-server[1170]: 2026/09/25 16:52:03 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6325lb # [ 144.371588] niks3-server[1170]: 2026/09/25 16:52:03 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6326worker2 # [ 144.383034] nix-grpc-daemon[1588]: time=2026-09-25T16:52:03.674Z level=INFO msg="Uploading 1 narinfos"6327lb # [ 144.380279] niks3-server[1170]: 2026/09/25 16:52:03 INFO Signed narinfos id=50 count=16328lb # [ 144.402890] niks3-server[1170]: 2026/09/25 16:52:03 INFO Received complete push request method=POST path=/api/pushes/50/complete6329lb # [ 144.405857] niks3-server[1170]: 2026/09/25 16:52:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6330worker2 # [ 144.412234] nix-grpc-daemon[1588]: time=2026-09-25T16:52:03.703Z level=INFO msg="Upload complete. (194ms)"6331lb # [ 144.410591] niks3-server[1170]: 2026/09/25 16:52:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6332worker2 # [ 144.416829] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48626 duration_s=0 paths=1 nar_bytes_in=4246333worker2 # [ 144.422631] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486266334worker2 # [ 144.432512] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6335worker2 # [ 144.440411] nix-daemon[858]: reaped child process 1586, status = succeeded6336worker2 # [ 144.445259] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6337client # [ 144.445275] nix[1416]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6338lb: must succeed: curl -sf localhost:9901/clusters6339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6340lb: must succeed: curl -sf localhost:9901/clusters6341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6342worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6343worker1 # [ 145.776512] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1506 (nix-grpc-daemon) on client request.6344lb # [ 145.772824] niks3-server[1170]: 2026/09/25 16:52:05 INFO lead: released remote=192.168.1.3:397846345worker1 # [ 145.794573] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6346worker2 # [ 145.783283] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:05Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6347client # [ 145.783573] nix[1416]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6348worker1 # [ 145.812274] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6349worker1 # [ 145.813759] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6350worker1 # [ 145.814939] systemd[1]: nix-grpc-daemon.service: Consumed 215ms CPU time over 13.967s wall clock time, 13.6M memory peak, 33.2K incoming IP traffic, 41K outgoing IP traffic.6351worker1 # [ 145.828259] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6352worker1 # [ 145.829734] systemd[1]: Starting Nix worker-protocol over gRPC...6353worker1 # [ 145.834349] nix-daemon[864]: reaped child process 1565, status = succeeded6354worker1 # [ 145.905657] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6355worker1 # [ 145.907662] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6356worker1 # [ 145.910777] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16357worker1 # [ 145.915056] nix-grpc-daemon[1635]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6358worker1 # [ 145.918199] nix-grpc-daemon[1635]: E0000 00:00:1790355125.237658 1635 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6359worker1 # [ 145.920449] nix-grpc-daemon[1635]: E0000 00:00:1790355125.243021 1635 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6360worker1 # [ 145.924564] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6361worker1 # [ 145.926582] systemd[1]: Started Nix worker-protocol over gRPC.6362worker1 # [ 145.933080] nix-grpc-daemon[1635]: E0000 00:00:1790355125.254643 1677 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6363lb # [ 145.914725] niks3-server[1170]: 2026/09/25 16:52:05 INFO lead: acquired remote=192.168.1.3:444786364worker1 # [ 145.937128] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=scheduler_take_over6365worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.19 seconds)6366client: must succeed: sleep 5.06367worker1 # [ 145.945974] nix-grpc-daemon[1635]: E0000 00:00:1790355125.268616 1677 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6368worker1 # [ 145.949153] nix-grpc-daemon[1635]: E0000 00:00:1790355125.270859 1677 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6369worker1 # [ 145.960234] nix-grpc-daemon[1635]: E0000 00:00:1790355125.282834 1677 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6370worker1 # [ 145.969567] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=scheduler_connected addr=lb:500516371worker1 # [ 145.976191] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:577226372worker1 # [ 145.977732] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06373worker2 # [ 145.983469] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:05Z level=info event=scheduler_connected addr=lb:500516374worker1 # [ 146.003502] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=oidc_jwks provider=mock keys=16375worker1 # [ 146.004885] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:577226376worker1 # [ 146.006559] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16377worker1 # [ 146.635325] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:05Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:577526378lb # [ 147.335260] envoy[919]: conn peer=[2001:db8:1::1]:39896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1518 ms=69306379lb # [ 147.339191] envoy[919]: conn peer=[2001:db8:1::4]:47436 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=31014 tx=1808 ms=322216380lb # [ 147.342913] envoy[919]: conn peer=[2001:db8:1::3]:39464 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2386 tx=619 ms=138236381client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6382client: waiting for success: ! systemctl is-active mid6383worker2 # [ 151.286797] nix-daemon[858]: reaped child process 1608, status = succeeded6384worker2 # [ 151.367323] nix-daemon[858]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6385worker2 # [ 151.378629] nix-daemon[858]: reaped child process 1648, status = succeeded6386lb # [ 151.384721] niks3-server[1170]: 2026/09/25 16:52:10 INFO Received push request method=POST path=/api/pushes6387worker2 # [ 151.397322] nix-grpc-daemon[1588]: time=2026-09-25T16:52:10.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6388worker2 # [ 151.399051] nix-grpc-daemon[1588]: time=2026-09-25T16:52:10.690Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6389lb # [ 151.429244] niks3-server[1170]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6390lb # [ 151.437629] niks3-server[1170]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6391lb # [ 151.446252] niks3-server[1170]: 2026/09/25 16:52:10 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6392lb # [ 151.450475] niks3-server[1170]: 2026/09/25 16:52:10 INFO Signed narinfos id=51 count=16393worker2 # [ 151.457493] nix-grpc-daemon[1588]: time=2026-09-25T16:52:10.748Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 151.457926] niks3-server[1170]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6395lb # [ 151.467115] niks3-server[1170]: 2026/09/25 16:52:10 INFO Received complete push request method=POST path=/api/pushes/51/complete6396lb # [ 151.473166] niks3-server[1170]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6397worker2 # [ 151.474813] nix-grpc-daemon[1588]: time=2026-09-25T16:52:10.766Z level=INFO msg="Upload complete. (217ms)"6398worker2 # [ 151.480988] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48626 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06399worker2 # [ 151.496722] nix-daemon[858]: reaped child process 1606, status = succeeded6400client # [ 151.512574] systemd[1]: mid.service: Deactivated successfully.6401client # [ 151.515979] systemd[1]: mid.service: Consumed 131ms CPU time over 8.227s wall clock time, 43.4M memory peak, 7.8K incoming IP traffic, 8.8K outgoing IP traffic.6402client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6403client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6404client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6405(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.00 seconds)6406subtest: interrupting the client stops the build on the worker6407client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6408client # [ 152.261221] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6409client # Running as unit: intr.service; invocation ID: 4b3c7bc73fd340459dc2d9d9d783e6ff6410client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6411worker1 # [ 152.399577] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577546412worker1 # [ 152.410682] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6413client # [ 152.407394] nix[1452]: don't know how to build these paths:6414client # [ 152.408328] nix[1452]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6415worker1 # [ 152.432414] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:11Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06416worker1 # [ 152.437272] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:11Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16417worker1 # [ 152.444250] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546418worker1 # [ 152.453648] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 1/5)6419worker1 # [ 152.516928] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5)6420worker1 # [ 152.585983] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5)6421worker1 # [ 152.680119] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 4/5)6422worker1 # [ 152.834445] nix-grpc-daemon[1635]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6423worker1 # [ 152.837360] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6424worker1 # [ 152.855064] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546425client # [ 152.849208] nix[1452]: copying 1 paths to 'lb:50051'...6426client # [ 152.853154] nix[1452]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6427worker1 # [ 152.869345] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546428worker1 # [ 152.879162] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6429worker1 # [ 152.900200] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:12Z level=info event=niks3_push_started pid=17096430worker1 # [ 153.000289] nix-grpc-daemon[1709]: time=2026-09-25T16:52:12.322Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6431worker1 # [ 153.084704] nix-daemon[864]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6432worker1 # [ 153.095825] nix-daemon[864]: reaped child process 1725, status = succeeded6433lb # [ 153.086553] niks3-server[1170]: 2026/09/25 16:52:12 INFO Received push request method=POST path=/api/pushes6434worker1 # [ 153.115725] nix-grpc-daemon[1709]: time=2026-09-25T16:52:12.438Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6435worker1 # [ 153.117444] nix-grpc-daemon[1709]: time=2026-09-25T16:52:12.440Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6436lb # [ 153.129794] niks3-server[1170]: 2026/09/25 16:52:12 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6437lb # [ 153.139615] niks3-server[1170]: 2026/09/25 16:52:12 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6438worker1 # [ 153.164274] nix-grpc-daemon[1709]: time=2026-09-25T16:52:12.486Z level=INFO msg="Uploading 1 narinfos"6439lb # [ 153.146364] niks3-server[1170]: 2026/09/25 16:52:12 INFO Signed narinfos id=52 count=16440lb # [ 153.166271] niks3-server[1170]: 2026/09/25 16:52:12 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6441lb # [ 153.172447] niks3-server[1170]: 2026/09/25 16:52:12 INFO Received complete push request method=POST path=/api/pushes/52/complete6442lb # [ 153.179078] niks3-server[1170]: 2026/09/25 16:52:12 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6443worker1 # [ 153.200914] nix-grpc-daemon[1709]: time=2026-09-25T16:52:12.523Z level=INFO msg="Upload complete. (197ms)"6444worker1 # [ 153.205868] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=4246445worker1 # [ 153.212301] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546446worker1 # [ 153.225115] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6447worker1 # [ 153.232262] nix-daemon[864]: reaped child process 1707, status = succeeded6448worker1 # [ 153.239822] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6449client # [ 153.223553] nix[1452]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6450client: must succeed: systemctl kill -s INT intr6451client # [ 154.581799] systemd[1]: intr.service: Sent signal SIGINT to main process 1452 (nix) on client request.6452worker1 # [ 154.604641] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:13Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6453client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6454client # [ 154.590329] nix[1452]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6455client # [ 154.597406] nix[1452]: error: interrupted by the user6456client # [ 154.623691] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6457client # [ 154.626433] systemd[1]: intr.service: Failed with result 'exit-code'.6458client # [ 154.630114] systemd[1]: intr.service: Consumed 115ms CPU time over 2.367s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6459worker1 # [ 154.658969] nix-daemon[864]: reaped child process 1727, status = succeeded6460worker1 # [ 154.660547] nix-daemon[1729]: unexpected Nix daemon error: error: interrupted by the user6461worker1 # [ 154.665158] nix-daemon[864]: reaped child process 1729, status = failed with exit code 16462(finished: subtest: interrupting the client stops the build on the worker, in 2.44 seconds)6463subtest: a deploy mid-build drains: the build finishes, the new generation takes over6464client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6465client # [ 154.721970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6466client # Running as unit: sw.service; invocation ID: 1522946fc08741a3b088a48cfaedb2546467client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.05 seconds)6468worker1 # [ 154.860385] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577546469client # [ 154.850968] nix[1472]: don't know how to build these paths:6470client # [ 154.851872] nix[1472]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6471worker1 # [ 154.874824] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:14Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06472worker2 # [ 154.864456] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486266473worker1 # [ 154.879348] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:14Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16474worker2 # [ 154.875942] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486266475client # [ 154.885073] nix[1472]: copying 1 paths to 'lb:50051'...6476client # [ 154.885826] nix[1472]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6477worker2 # [ 154.886751] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486266478worker2 # [ 154.897907] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6479worker2 # [ 154.997421] nix-daemon[858]: accepted connection from pid 1672, user nix-grpc-daemon (trusted)6480worker2 # [ 155.009491] nix-daemon[858]: reaped child process 1675, status = succeeded6481lb # [ 155.016576] niks3-server[1170]: 2026/09/25 16:52:14 INFO Received push request method=POST path=/api/pushes6482worker2 # [ 155.031525] nix-grpc-daemon[1588]: time=2026-09-25T16:52:14.322Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6483worker2 # [ 155.033394] nix-grpc-daemon[1588]: time=2026-09-25T16:52:14.324Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6484lb # [ 155.049706] niks3-server[1170]: 2026/09/25 16:52:14 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6485lb # [ 155.055583] niks3-server[1170]: 2026/09/25 16:52:14 INFO Signed narinfos id=53 count=16486worker2 # [ 155.063843] nix-grpc-daemon[1588]: time=2026-09-25T16:52:14.355Z level=INFO msg="Uploading 1 narinfos"6487lb # [ 155.063533] niks3-server[1170]: 2026/09/25 16:52:14 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6488lb # [ 155.070171] niks3-server[1170]: 2026/09/25 16:52:14 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6489lb # [ 155.081275] niks3-server[1170]: 2026/09/25 16:52:14 INFO Received complete push request method=POST path=/api/pushes/53/complete6490lb # [ 155.090057] niks3-server[1170]: 2026/09/25 16:52:14 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6491worker2 # [ 155.092128] nix-grpc-daemon[1588]: time=2026-09-25T16:52:14.383Z level=INFO msg="Upload complete. (183ms)"6492worker2 # [ 155.096837] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48626 duration_s=0 paths=1 nar_bytes_in=4166493worker2 # [ 155.102685] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486266494worker2 # [ 155.112948] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6495worker2 # [ 155.120506] nix-daemon[858]: reaped child process 1670, status = succeeded6496worker2 # [ 155.125308] nix-daemon[858]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6497client # [ 155.124566] nix[1472]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6498worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26499worker2 # Checking switch inhibitors... done6500lb # [ 157.335989] envoy[919]: conn peer=[2001:db8:1::1]:32968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3074 tx=1608 ms=80706501lb # [ 157.339807] envoy[919]: conn peer=[2001:db8:1::1]:32980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=826 ms=22286502worker2 # [ 157.455853] nixos[1711]: switching to system configuration /nix/store/nwjk341l9c9p4n6yzngl25ykgzs3hzyr-nixos-system-worker2-test6503worker2 # [ 157.459026] systemd[1]: Stopped target Remote File Systems.6504worker2 # activating the configuration...6505worker2 # [ 157.461180] systemd[1]: Stopped target Local File Systems.6506worker2 # setting up /etc...6507worker2 # [ 157.882931] systemd[1]: Reload requested from client PID 1711 ('.switch-to-conf') (unit backdoor.service)...6508worker2 # [ 157.885264] systemd[1]: Reloading...6509worker2 # [ 159.233406] systemd[1]: Reloading finished in 1346 ms.6510worker2 # restarting sysinit-reactivation.target6511worker2 # [ 159.292454] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6512worker2 # [ 159.395318] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6513worker2 # [ 159.397657] systemd[1]: Reached target Reactivate sysinit units.6514worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6515worker2 # [ 159.402771] systemd[1]: Reloading D-Bus System Message Bus...6516worker2 # [ 159.406369] dbus-broker-launch[518]: Caught SIGHUP, trigger reload.6517worker2 # [ 159.407701] systemd[1]: Reloading Nix worker-protocol over gRPC...6518worker2 # [ 159.424177] dbus-broker-launch[518]: Looking up NSS user entry for 'systemd-timesync'...6519worker2 # [ 159.428058] dbus-broker-launch[518]: NSS returned no entry for 'systemd-timesync'6520worker2 # [ 159.432732] dbus-broker-launch[518]: Invalid user-name in /nix/store/9f31zf0wixjfmyvwg6m53sif40vrkysg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6521worker2 # [ 159.441645] systemd[1]: Reloaded Nix worker-protocol over gRPC.6522worker2 # [ 159.450252] systemd[1]: Reloaded D-Bus System Message Bus.6523worker2 # [ 159.457061] systemd[1]: Reached target Remote File Systems.6524worker2 # [ 159.459091] systemd[1]: Starting linger-users.service...6525worker2 # [ 159.463184] systemd[1]: Reached target Local File Systems.6526worker2 # [ 159.465075] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6527worker2 # [ 159.470129] systemd[1]: Starting Create SUID/SGID Wrappers...6528worker2 # [ 159.470979] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6529worker2 # [ 159.473039] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676530worker2 # [ 159.475158] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6531worker2 # [ 159.478988] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6532worker2 # [ 159.483541] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6533worker2 # [ 159.485815] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6534worker2 # [ 159.487064] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6535worker2 # [ 159.519468] systemd[1]: linger-users.service: Deactivated successfully.6536worker2 # [ 159.520495] systemd[1]: Finished linger-users.service.6537worker2 # [ 159.739889] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6538worker2 # [ 159.741067] systemd[1]: Finished Create SUID/SGID Wrappers.6539worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6540worker2 # [ 159.997406] nixos[1711]: finished switching to system configuration /nix/store/nwjk341l9c9p4n6yzngl25ykgzs3hzyr-nixos-system-worker2-test6541worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.91 seconds)6542worker2: must succeed: pgrep -f 'read -t [9]0 x'6543worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6544client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26545worker1 # [ 160.203321] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577546546client # don't know how to build these paths:6547client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6548worker1 # [ 160.215215] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546549worker1 # [ 160.239204] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06550worker1 # [ 160.244319] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16551worker1 # [ 160.251654] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546552worker1 # [ 160.263886] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546553worker2 # [ 160.253886] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:19Z level=info event=draining6554client # copying 1 paths to 'lb:50051'...6555worker1 # [ 160.280616] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546556client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6557worker1 # [ 160.306229] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6558worker1 # [ 160.404680] nix-daemon[864]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6559worker1 # [ 160.417508] nix-daemon[864]: reaped child process 1783, status = succeeded6560lb # [ 160.406733] niks3-server[1170]: 2026/09/25 16:52:19 INFO Received push request method=POST path=/api/pushes6561worker1 # [ 160.434285] nix-grpc-daemon[1709]: time=2026-09-25T16:52:19.756Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6562worker1 # [ 160.436152] nix-grpc-daemon[1709]: time=2026-09-25T16:52:19.758Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6563lb # [ 160.442309] niks3-server[1170]: 2026/09/25 16:52:19 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6564lb # [ 160.449109] niks3-server[1170]: 2026/09/25 16:52:19 INFO Signed narinfos id=54 count=16565worker1 # [ 160.470661] nix-grpc-daemon[1709]: time=2026-09-25T16:52:19.792Z level=INFO msg="Uploading 1 narinfos"6566lb # [ 160.457742] niks3-server[1170]: 2026/09/25 16:52:19 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6567lb # [ 160.463243] niks3-server[1170]: 2026/09/25 16:52:19 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6568lb # [ 160.472208] niks3-server[1170]: 2026/09/25 16:52:19 INFO Received complete push request method=POST path=/api/pushes/54/complete6569lb # [ 160.480120] niks3-server[1170]: 2026/09/25 16:52:19 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6570worker1 # [ 160.497524] nix-grpc-daemon[1709]: time=2026-09-25T16:52:19.820Z level=INFO msg="Upload complete. (179ms)"6571worker1 # [ 160.502406] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=5286572worker1 # [ 160.509138] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546573worker1 # [ 160.519567] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6574worker1 # [ 160.525565] nix-daemon[864]: reaped child process 1778, status = succeeded6575worker1 # [ 160.529840] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6576client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6577client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6578client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6579client # farm-a> LOG-farm-a-during-drain6580worker1 # [ 161.546192] nix-daemon[864]: reaped child process 1787, status = succeeded6581worker1 # [ 161.617769] nix-daemon[864]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6582worker1 # [ 161.630727] nix-daemon[864]: reaped child process 1806, status = succeeded6583lb # [ 161.622560] niks3-server[1170]: 2026/09/25 16:52:20 INFO Received push request method=POST path=/api/pushes6584worker1 # [ 161.647467] nix-grpc-daemon[1709]: time=2026-09-25T16:52:20.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6585worker1 # [ 161.649164] nix-grpc-daemon[1709]: time=2026-09-25T16:52:20.971Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6586lb # [ 161.661773] niks3-server[1170]: 2026/09/25 16:52:20 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6587lb # [ 161.671854] niks3-server[1170]: 2026/09/25 16:52:20 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6588worker1 # [ 161.703241] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.025Z level=INFO msg="Uploading 1 narinfos"6589lb # [ 161.685213] niks3-server[1170]: 2026/09/25 16:52:20 INFO Signed narinfos id=55 count=16590lb # [ 161.689155] niks3-server[1170]: 2026/09/25 16:52:20 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6591lb # [ 161.698147] niks3-server[1170]: 2026/09/25 16:52:20 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6592lb # [ 161.704189] niks3-server[1170]: 2026/09/25 16:52:20 INFO Received complete push request method=POST path=/api/pushes/55/complete6593lb # [ 161.710960] niks3-server[1170]: 2026/09/25 16:52:20 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6594worker1 # [ 161.727259] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.049Z level=INFO msg="Upload complete. (190ms)"6595worker1 # [ 161.736402] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06596worker1 # [ 161.748964] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06597worker1 # [ 161.754515] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16598worker1 # [ 161.759259] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546599worker1 # [ 161.769539] nix-daemon[864]: reaped child process 1785, status = succeeded6600worker1 # [ 161.771169] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546601worker1 # [ 161.779539] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.102Z level=INFO msg="All 1 paths already cached"6602client # copying 1 paths to 'lb:50051'...6603client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6604worker1 # [ 161.785314] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546605worker1 # [ 161.796603] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6606worker1 # [ 161.879189] nix-daemon[864]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6607worker1 # [ 161.891761] nix-daemon[864]: reaped child process 1815, status = succeeded6608lb # [ 161.881363] niks3-server[1170]: 2026/09/25 16:52:21 INFO Received push request method=POST path=/api/pushes6609worker1 # [ 161.909257] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.231Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 161.911494] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.233Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6611lb # [ 161.914168] niks3-server[1170]: 2026/09/25 16:52:21 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6612lb # [ 161.921609] niks3-server[1170]: 2026/09/25 16:52:21 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6613worker1 # [ 161.945772] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.268Z level=INFO msg="Uploading 1 narinfos"6614lb # [ 161.927484] niks3-server[1170]: 2026/09/25 16:52:21 INFO Signed narinfos id=56 count=16615lb # [ 161.938472] niks3-server[1170]: 2026/09/25 16:52:21 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6616lb # [ 161.943111] niks3-server[1170]: 2026/09/25 16:52:21 INFO Received complete push request method=POST path=/api/pushes/56/complete6617lb # [ 161.953477] niks3-server[1170]: 2026/09/25 16:52:21 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6618worker1 # [ 161.970391] nix-grpc-daemon[1709]: time=2026-09-25T16:52:21.292Z level=INFO msg="Upload complete. (168ms)"6619worker1 # [ 161.975058] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=6406620worker1 # [ 161.981293] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546621worker1 # [ 161.990536] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6622worker1 # [ 161.995303] nix-daemon[864]: reaped child process 1810, status = succeeded6623worker1 # [ 162.000396] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6624client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6625client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6626client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6627client # farm-b> LOG-farm-b-during-drain6628worker1 # [ 163.244468] nix-daemon[864]: reaped child process 1819, status = succeeded6629worker1 # [ 163.318829] nix-daemon[864]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6630worker1 # [ 163.331151] nix-daemon[864]: reaped child process 1838, status = succeeded6631lb # [ 163.319496] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received push request method=POST path=/api/pushes6632worker1 # [ 163.346256] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.668Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6633worker1 # [ 163.347929] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.670Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6634lb # [ 163.362647] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6635lb # [ 163.366983] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6636lb # [ 163.378569] niks3-server[1170]: 2026/09/25 16:52:22 INFO Signed narinfos id=57 count=16637worker1 # [ 163.399566] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.721Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 163.383710] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6639lb # [ 163.386959] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6640lb # [ 163.397786] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received complete push request method=POST path=/api/pushes/57/complete6641lb # [ 163.404163] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6642worker1 # [ 163.420662] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.743Z level=INFO msg="Upload complete. (212ms)"6643worker1 # [ 163.431313] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06644worker1 # [ 163.443076] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06645worker1 # [ 163.448183] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16646worker1 # [ 163.452597] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546647worker1 # [ 163.462555] nix-daemon[864]: reaped child process 1817, status = succeeded6648worker1 # [ 163.464699] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546649worker1 # [ 163.472808] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.795Z level=INFO msg="All 2 paths already cached"6650client # copying 1 paths to 'lb:50051'...6651worker1 # [ 163.478608] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546652client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6653worker1 # [ 163.494944] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6654worker1 # [ 163.579469] nix-daemon[864]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6655worker1 # [ 163.592775] nix-daemon[864]: reaped child process 1847, status = succeeded6656lb # [ 163.581400] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received push request method=POST path=/api/pushes6657worker1 # [ 163.607564] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.930Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6658worker1 # [ 163.609518] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.931Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6659lb # [ 163.615306] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6660worker1 # [ 163.639168] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.961Z level=INFO msg="Uploading 1 narinfos"6661lb # [ 163.622367] niks3-server[1170]: 2026/09/25 16:52:22 INFO Signed narinfos id=58 count=16662lb # [ 163.628172] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6663lb # [ 163.634560] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6664lb # [ 163.644084] niks3-server[1170]: 2026/09/25 16:52:22 INFO Received complete push request method=POST path=/api/pushes/58/complete6665lb # [ 163.653227] niks3-server[1170]: 2026/09/25 16:52:22 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6666worker1 # [ 163.676833] nix-grpc-daemon[1709]: time=2026-09-25T16:52:22.999Z level=INFO msg="Upload complete. (175ms)"6667worker1 # [ 163.682133] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=7686668worker1 # [ 163.688752] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546669worker1 # [ 163.701188] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6670worker1 # [ 163.709361] nix-daemon[864]: reaped child process 1842, status = succeeded6671worker1 # [ 163.713973] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6672client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6673client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6674client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6675client # farm-top> LOG-farm-top-during-drain6676worker1 # [ 164.752550] nix-daemon[864]: reaped child process 1851, status = succeeded6677worker1 # [ 164.834301] nix-daemon[864]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)6678worker1 # [ 164.847537] nix-daemon[864]: reaped child process 1870, status = succeeded6679lb # [ 164.836758] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received push request method=POST path=/api/pushes6680worker1 # [ 164.863393] nix-grpc-daemon[1709]: time=2026-09-25T16:52:24.185Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6681worker1 # [ 164.865178] nix-grpc-daemon[1709]: time=2026-09-25T16:52:24.187Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6682lb # [ 164.878524] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6683lb # [ 164.886655] niks3-server[1170]: 2026/09/25 16:52:24 INFO Signed narinfos id=59 count=16684worker1 # [ 164.910868] nix-grpc-daemon[1709]: time=2026-09-25T16:52:24.233Z level=INFO msg="Uploading 1 narinfos"6685lb # [ 164.893270] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6686lb # [ 164.898633] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6687lb # [ 164.905622] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6688lb # [ 164.915920] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received complete push request method=POST path=/api/pushes/59/complete6689lb # [ 164.922340] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6690worker1 # [ 164.939073] nix-grpc-daemon[1709]: time=2026-09-25T16:52:24.261Z level=INFO msg="Upload complete. (217ms)"6691worker1 # [ 164.945772] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06692worker1 # [ 164.961649] nix-daemon[864]: reaped child process 1849, status = succeeded6693client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.93 seconds)6694client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6695client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6696worker2: must succeed: pkill -f 'read -t [9]0 x'6697worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6698client: waiting for success: ! systemctl is-active sw6699worker2 # [ 165.149629] nix-daemon[858]: reaped child process 1679, status = succeeded6700worker2 # [ 165.224536] nix-daemon[858]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6701worker2 # [ 165.236989] nix-daemon[858]: reaped child process 1854, status = succeeded6702lb # [ 165.242535] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received push request method=POST path=/api/pushes6703worker2 # [ 165.253453] nix-grpc-daemon[1588]: time=2026-09-25T16:52:24.544Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6704worker2 # [ 165.255172] nix-grpc-daemon[1588]: time=2026-09-25T16:52:24.546Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6705lb # [ 165.279385] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6706lb # [ 165.290603] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6707worker2 # [ 165.299112] nix-grpc-daemon[1588]: time=2026-09-25T16:52:24.590Z level=INFO msg="Uploading 1 narinfos"6708lb # [ 165.296723] niks3-server[1170]: 2026/09/25 16:52:24 INFO Signed narinfos id=60 count=16709lb # [ 165.304515] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6710lb # [ 165.308909] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6711lb # [ 165.318473] niks3-server[1170]: 2026/09/25 16:52:24 INFO Received complete push request method=POST path=/api/pushes/60/complete6712lb # [ 165.326039] niks3-server[1170]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6713worker2 # [ 165.331716] nix-grpc-daemon[1588]: time=2026-09-25T16:52:24.623Z level=INFO msg="Upload complete. (190ms)"6714worker2 # [ 165.335057] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48626 duration_s=10 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06715worker2 # [ 165.342175] nix-daemon[858]: reaped child process 1677, status = succeeded6716client # [ 165.366136] systemd[1]: sw.service: Deactivated successfully.6717client # [ 165.369168] systemd[1]: sw.service: Consumed 120ms CPU time over 10.643s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic.6718client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6719client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6720client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6721worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6722worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.02 seconds)6723lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6724lb: must succeed: curl -sf localhost:9901/clusters6725worker2 # [ 166.255588] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:25Z level=info event=signal_exit6726lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6727worker1 # [ 167.279264] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:577226728worker1 # [ 167.286644] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:26Z level=info event=worker_gone addr=192.168.1.4:500516729worker2 # [ 167.263698] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:26Z level=info event=scheduler_restarting6730lb: must succeed: curl -sf localhost:9901/clusters6731worker2 # [ 167.279184] nix-grpc-daemon[1524]: ts=2026-09-25T16:52:26Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6732worker2 # [ 167.301572] nix-daemon[858]: reaped child process 1580, status = succeeded6733lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6734lb # [ 167.336476] envoy[919]: conn peer=[2001:db8:1::1]:48216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4382 ms=47696735lb # [ 167.339533] envoy[919]: conn peer=[2001:db8:1::1]:48210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1524 ms=105016736lb # [ 167.342371] envoy[919]: conn peer=[2001:db8:1::4]:33186 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4101 tx=509 ms=249886737worker2 # [ 168.280256] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6738worker2 # [ 168.284238] systemd[1]: nix-grpc-daemon.service: Consumed 638ms CPU time over 26.136s wall clock time, 35.6M memory peak, 52.2K incoming IP traffic, 63.5K outgoing IP traffic.6739lb: must succeed: curl -sf localhost:9901/clusters6740lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6741worker2 # [ 168.440942] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6742worker2 # [ 168.449742] systemd[1]: Starting scheduler-token.service...6743worker2 # [ 168.506735] systemd[1]: scheduler-token.service: Deactivated successfully.6744worker2 # [ 168.510921] systemd[1]: Finished scheduler-token.service.6745worker2 # [ 168.512465] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 60ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6746worker2 # [ 168.520514] systemd[1]: Starting Nix worker-protocol over gRPC...6747worker2 # [ 168.600680] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6748worker2 # [ 168.602724] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6749worker2 # [ 168.605898] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16750worker2 # [ 168.610051] nix-grpc-daemon[1870]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6751worker2 # [ 168.611616] nix-grpc-daemon[1870]: E0000 00:00:1790355147.901422 1870 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6752worker2 # [ 168.615397] nix-grpc-daemon[1870]: E0000 00:00:1790355147.906796 1870 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6753worker2 # [ 168.619175] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:27Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6754worker2 # [ 168.622640] systemd[1]: Started Nix worker-protocol over gRPC.6755worker2 # [ 168.626066] nix-grpc-daemon[1870]: E0000 00:00:1790355147.916823 1912 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6756worker2 # [ 168.637411] nix-grpc-daemon[1870]: E0000 00:00:1790355147.928836 1912 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6757worker2 # [ 168.641122] nix-grpc-daemon[1870]: E0000 00:00:1790355147.931137 1912 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6758worker2 # [ 168.651441] nix-grpc-daemon[1870]: E0000 00:00:1790355147.942883 1912 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6759worker2 # [ 168.660785] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:27Z level=info event=scheduler_connected addr=lb:500516760worker1 # [ 168.678876] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:577226761worker1 # [ 168.684695] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06762lb: must succeed: curl -sf localhost:9901/clusters6763lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6764lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 3.19 seconds)6765lb: must succeed: curl -sf localhost:9901/clusters6766lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6767worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6768worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6769(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.81 seconds)6770subtest: queue gauges show a burst while every build blocks6771lb: must succeed: curl -sf localhost:9901/clusters6772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6773lb: must succeed: curl -sf localhost:9901/clusters6774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6775worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6776worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6777client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6778client # [ 169.643865] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6779client # Running as unit: burst.service; invocation ID: 65104cc29c364ee19cbc65c3a4d1f20e6780client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6781lb: must succeed: curl -sf localhost:9901/clusters6782lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6783lb: must succeed: curl -sf localhost:9901/clusters6784lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6785worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6786worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6787worker1 # [ 169.793771] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577546788client # [ 169.783368] nix[1528]: don't know how to build these paths:6789client # [ 169.784298] nix[1528]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6790client # [ 169.785249] nix[1528]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6791client # [ 169.786449] nix[1528]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6792client # [ 169.787681] nix[1528]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6793worker1 # [ 169.809726] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06794worker1 # [ 169.815935] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.3:50051 clients=16795worker1 # [ 169.822231] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06796worker2 # [ 169.808422] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567526797worker1 # [ 169.825875] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.4:50051 clients=16798worker1 # [ 169.829867] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06799worker2 # [ 169.818223] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6800worker1 # [ 169.834122] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06801worker1 # [ 169.838532] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546802worker1 # [ 169.843340] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546803client # [ 169.834112] nix[1528]: copying 1 paths to 'lb:50051'...6804client # [ 169.834935] nix[1528]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6805worker1 # [ 169.850311] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546806worker2 # [ 169.834271] nix-grpc-daemon[1870]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)6807worker1 # [ 169.855545] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6808worker2 # [ 169.846570] nix-grpc-daemon[1870]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 2/5)6809worker1 # [ 169.939316] nix-daemon[864]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6810worker1 # [ 169.952943] nix-daemon[864]: reaped child process 1888, status = succeeded6811lb # [ 169.941265] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received push request method=POST path=/api/pushes6812worker1 # [ 169.969215] nix-grpc-daemon[1709]: time=2026-09-25T16:52:29.291Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6813worker1 # [ 169.971406] nix-grpc-daemon[1709]: time=2026-09-25T16:52:29.293Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6814lb # [ 169.973447] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6815lb # [ 169.980397] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6816lb # [ 169.985281] niks3-server[1170]: 2026/09/25 16:52:29 INFO Signed narinfos id=61 count=16817worker1 # [ 170.007112] nix-grpc-daemon[1709]: time=2026-09-25T16:52:29.329Z level=INFO msg="Uploading 1 narinfos"6818worker2 # [ 169.989571] nix-grpc-daemon[1870]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 3/5)6819lb # [ 170.001760] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received complete push request method=POST path=/api/pushes/61/complete6820lb # [ 170.012336] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6821lb # [ 170.020171] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6822worker1 # [ 170.043056] nix-grpc-daemon[1709]: time=2026-09-25T16:52:29.365Z level=INFO msg="Upload complete. (182ms)"6823worker1 # [ 170.049232] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=4166824worker1 # [ 170.056183] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546825worker1 # [ 170.069304] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6826worker1 # [ 170.079360] nix-daemon[864]: reaped child process 1883, status = succeeded6827worker1 # [ 170.084753] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6828client # [ 170.068744] nix[1528]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6829worker2 # [ 170.099592] nix-grpc-daemon[1870]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 192 ms (attempt 4/5)6830worker2 # [ 170.306499] nix-grpc-daemon[1870]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6831worker2 # [ 170.310133] nix-grpc-daemon[1870]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6832worker2 # [ 170.326810] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567526833client # [ 170.337370] nix[1528]: copying 1 paths to 'lb:50051'...6834client # [ 170.341217] nix[1528]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6835worker2 # [ 170.341825] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567526836worker2 # [ 170.352223] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6837worker2 # [ 170.373119] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=info event=niks3_push_started pid=19306838worker2 # [ 170.478500] nix-grpc-daemon[1930]: time=2026-09-25T16:52:29.769Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6839worker2 # [ 170.565156] nix-daemon[858]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6840worker2 # [ 170.578204] nix-daemon[858]: reaped child process 1946, status = succeeded6841lb # [ 170.582768] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received push request method=POST path=/api/pushes6842worker2 # [ 170.593262] nix-grpc-daemon[1930]: time=2026-09-25T16:52:29.884Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6843worker2 # [ 170.594770] nix-grpc-daemon[1930]: time=2026-09-25T16:52:29.886Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6844lb # [ 170.621127] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6845lb # [ 170.625290] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6846worker2 # [ 170.634486] nix-grpc-daemon[1930]: time=2026-09-25T16:52:29.925Z level=INFO msg="Uploading 1 narinfos"6847lb # [ 170.632209] niks3-server[1170]: 2026/09/25 16:52:29 INFO Signed narinfos id=62 count=16848lb # [ 170.638995] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6849lb # [ 170.651077] niks3-server[1170]: 2026/09/25 16:52:29 INFO Received complete push request method=POST path=/api/pushes/62/complete6850worker2 # [ 170.659111] nix-grpc-daemon[1930]: time=2026-09-25T16:52:29.950Z level=INFO msg="Upload complete. (177ms)"6851lb # [ 170.657258] niks3-server[1170]: 2026/09/25 16:52:29 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6852worker2 # [ 170.661329] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 paths=1 nar_bytes_in=4166853worker2 # [ 170.664442] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567526854worker2 # [ 170.671845] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6855worker2 # [ 170.675610] nix-daemon[858]: reaped child process 1928, status = succeeded6856worker2 # [ 170.680245] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6857client # [ 170.679164] nix[1528]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6858lb: must succeed: curl -sf localhost:9901/clusters6859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6860lb: must succeed: curl -sf localhost:9901/clusters6861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6862worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6863worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6864worker1 # [ 172.099907] nix-daemon[864]: reaped child process 1892, status = succeeded6865worker1 # [ 172.173450] nix-daemon[864]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6866worker1 # [ 172.185628] nix-daemon[864]: reaped child process 1923, status = succeeded6867lb # [ 172.176294] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received push request method=POST path=/api/pushes6868worker1 # [ 172.202599] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.524Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6869worker1 # [ 172.204418] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.526Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6870lb # [ 172.227077] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6871lb # [ 172.236963] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6872lb # [ 172.239955] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6873lb # [ 172.244553] niks3-server[1170]: 2026/09/25 16:52:31 INFO Signed narinfos id=63 count=16874worker1 # [ 172.262800] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.585Z level=INFO msg="Uploading 1 narinfos"6875lb # [ 172.254973] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received complete push request method=POST path=/api/pushes/63/complete6876lb # [ 172.258143] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6877lb # [ 172.263199] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6878worker1 # [ 172.279767] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.602Z level=INFO msg="Upload complete. (187ms)"6879worker1 # [ 172.289735] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=6 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06880worker1 # [ 172.297383] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.3:50051 clients=16881worker1 # [ 172.301437] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546882worker1 # [ 172.313307] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577546883worker1 # [ 172.316195] nix-daemon[864]: reaped child process 1890, status = succeeded6884client # [ 172.303809] nix[1528]: copying 1 paths to 'lb:50051'...6885worker1 # [ 172.322215] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577546886client # [ 172.307161] nix[1528]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6887worker1 # [ 172.328707] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6888worker2 # [ 172.329168] nix-daemon[858]: reaped child process 1950, status = succeeded6889worker1 # [ 172.411919] nix-daemon[864]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6890worker2 # [ 172.406085] nix-daemon[858]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)6891worker1 # [ 172.426113] nix-daemon[864]: reaped child process 1930, status = succeeded6892worker2 # [ 172.418387] nix-daemon[858]: reaped child process 1978, status = succeeded6893lb # [ 172.414141] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received push request method=POST path=/api/pushes6894lb # [ 172.421476] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received push request method=POST path=/api/pushes6895worker1 # [ 172.445801] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.768Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6896worker1 # [ 172.447561] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.770Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6897worker2 # [ 172.433428] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6898worker2 # [ 172.435316] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.726Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6899lb # [ 172.457723] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6900lb # [ 172.463394] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6901lb # [ 172.464824] niks3-server[1170]: 2026/09/25 16:52:31 INFO Signed narinfos id=64 count=16902worker1 # [ 172.484935] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.807Z level=INFO msg="Uploading 1 narinfos"6903lb # [ 172.469574] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6904lb # [ 172.472834] niks3-server[1170]: 2026/09/25 16:52:31 INFO Signed narinfos id=65 count=16905worker2 # [ 172.476808] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.768Z level=INFO msg="Uploading 1 narinfos"6906lb # [ 172.475324] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6907lb # [ 172.478441] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6908lb # [ 172.480790] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6909lb # [ 172.485574] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6910lb # [ 172.491436] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received complete push request method=POST path=/api/pushes/65/complete6911lb # [ 172.492852] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received complete push request method=POST path=/api/pushes/64/complete6912worker1 # [ 172.516125] nix-grpc-daemon[1709]: time=2026-09-25T16:52:31.838Z level=INFO msg="Upload complete. (181ms)"6913lb # [ 172.500366] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6914worker2 # [ 172.502522] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.793Z level=INFO msg="Upload complete. (214ms)"6915lb # [ 172.503093] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6916worker1 # [ 172.521739] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=4166917worker2 # [ 172.508606] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06918worker1 # [ 172.530353] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.4:50051 clients=16919worker2 # [ 172.519337] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567526920worker1 # [ 172.535687] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577546921worker2 # [ 172.528524] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567526922worker1 # [ 172.545484] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6923worker2 # [ 172.535444] nix-daemon[858]: reaped child process 1948, status = succeeded6924worker1 # [ 172.554061] nix-daemon[864]: reaped child process 1925, status = succeeded6925client # [ 172.539170] nix[1528]: copying 1 paths to 'lb:50051'...6926worker2 # [ 172.540326] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567526927worker1 # [ 172.559140] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6928client # [ 172.542252] nix[1528]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6929client # [ 172.545828] nix[1528]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6930worker2 # [ 172.554133] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6931worker2 # [ 172.641788] nix-daemon[858]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6932worker2 # [ 172.654479] nix-daemon[858]: reaped child process 1985, status = succeeded6933lb # [ 172.659161] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received push request method=POST path=/api/pushes6934worker2 # [ 172.671469] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.962Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6935worker2 # [ 172.673907] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.964Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6936lb # [ 172.688910] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6937lb # [ 172.692495] niks3-server[1170]: 2026/09/25 16:52:31 INFO Signed narinfos id=66 count=16938worker2 # [ 172.698091] nix-grpc-daemon[1930]: time=2026-09-25T16:52:31.989Z level=INFO msg="Uploading 1 narinfos"6939lb # [ 172.701607] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6940lb # [ 172.708949] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6941lb # [ 172.714229] niks3-server[1170]: 2026/09/25 16:52:31 INFO Received complete push request method=POST path=/api/pushes/66/complete6942worker2 # [ 172.721352] nix-grpc-daemon[1930]: time=2026-09-25T16:52:32.012Z level=INFO msg="Upload complete. (159ms)"6943lb # [ 172.719732] niks3-server[1170]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6944worker2 # [ 172.723346] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 paths=1 nar_bytes_in=4166945worker2 # [ 172.726712] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567526946worker2 # [ 172.734314] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6947worker2 # [ 172.736864] nix-daemon[858]: reaped child process 1980, status = succeeded6948worker2 # [ 172.740726] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6949client # [ 172.739686] nix[1528]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6950worker1 # [ 174.375317] nix-daemon[864]: reaped child process 1934, status = succeeded6951worker1 # [ 174.449084] nix-daemon[864]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)6952worker1 # [ 174.461392] nix-daemon[864]: reaped child process 1962, status = succeeded6953lb # [ 174.449907] niks3-server[1170]: 2026/09/25 16:52:33 INFO Received push request method=POST path=/api/pushes6954worker1 # [ 174.477323] nix-grpc-daemon[1709]: time=2026-09-25T16:52:33.799Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6955worker1 # [ 174.479897] nix-grpc-daemon[1709]: time=2026-09-25T16:52:33.801Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6956lb # [ 174.497456] niks3-server[1170]: 2026/09/25 16:52:33 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6957lb # [ 174.503883] niks3-server[1170]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6958worker1 # [ 174.526434] nix-grpc-daemon[1709]: time=2026-09-25T16:52:33.848Z level=INFO msg="Uploading 1 narinfos"6959lb # [ 174.508663] niks3-server[1170]: 2026/09/25 16:52:33 INFO Signed narinfos id=67 count=16960lb # [ 174.516401] niks3-server[1170]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6961lb # [ 174.520414] niks3-server[1170]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6962lb # [ 174.530666] niks3-server[1170]: 2026/09/25 16:52:33 INFO Received complete push request method=POST path=/api/pushes/67/complete6963lb # [ 174.536555] niks3-server[1170]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6964worker1 # [ 174.553730] nix-grpc-daemon[1709]: time=2026-09-25T16:52:33.876Z level=INFO msg="Upload complete. (187ms)"6965worker1 # [ 174.560690] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06966worker1 # [ 174.576699] nix-daemon[864]: reaped child process 1932, status = succeeded6967worker2 # [ 174.602118] nix-daemon[858]: reaped child process 1989, status = succeeded6968worker2 # [ 174.655468] nix-daemon[858]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6969worker2 # [ 174.667505] nix-daemon[858]: reaped child process 2017, status = succeeded6970lb # [ 174.673194] niks3-server[1170]: 2026/09/25 16:52:33 INFO Received push request method=POST path=/api/pushes6971worker2 # [ 174.686406] nix-grpc-daemon[1930]: time=2026-09-25T16:52:33.977Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6972worker2 # [ 174.687892] nix-grpc-daemon[1930]: time=2026-09-25T16:52:33.979Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6973lb # [ 174.719577] niks3-server[1170]: 2026/09/25 16:52:33 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6974worker2 # [ 174.727854] nix-grpc-daemon[1930]: time=2026-09-25T16:52:34.019Z level=INFO msg="Uploading 1 narinfos"6975lb # [ 174.724948] niks3-server[1170]: 2026/09/25 16:52:33 INFO Signed narinfos id=68 count=16976lb # [ 174.731664] niks3-server[1170]: 2026/09/25 16:52:34 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6977lb # [ 174.741940] niks3-server[1170]: 2026/09/25 16:52:34 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6978lb # [ 174.751970] niks3-server[1170]: 2026/09/25 16:52:34 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6979lb # [ 174.754905] niks3-server[1170]: 2026/09/25 16:52:34 INFO Received complete push request method=POST path=/api/pushes/68/complete6980worker2 # [ 174.764300] nix-grpc-daemon[1930]: time=2026-09-25T16:52:34.055Z level=INFO msg="Upload complete. (215ms)"6981lb # [ 174.765052] niks3-server[1170]: 2026/09/25 16:52:34 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6982worker2 # [ 174.769477] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=9 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06983worker2 # [ 174.782321] nix-daemon[858]: reaped child process 1987, status = succeeded6984client # [ 174.802252] systemd[1]: burst.service: Deactivated successfully.6985client # [ 174.804774] systemd[1]: burst.service: Consumed 155ms CPU time over 5.158s wall clock time, 53.7M memory peak, 13.9K incoming IP traffic, 18.8K outgoing IP traffic.6986client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6987client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6988lb: must succeed: curl -sf localhost:9901/clusters6989lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6990lb: must succeed: curl -sf localhost:9901/clusters6991lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6992worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6993worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6994lb: must succeed: curl -sf localhost:9901/clusters6995lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6996lb: must succeed: curl -sf localhost:9901/clusters6997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6998worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6999worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7000(finished: subtest: queue gauges show a burst while every build blocks, in 6.30 seconds)7001subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7002client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17003worker1 # [ 175.952202] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:577547004worker2 # [ 175.940269] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567527005worker1 # [ 175.993866] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07006worker1 # [ 175.999973] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17007worker1 # [ 176.006851] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07008worker1 # [ 176.011462] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07009worker1 # [ 176.016470] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547010worker1 # [ 176.025580] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547011worker1 # [ 176.037981] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547012worker1 # [ 176.049291] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7013worker1 # [ 176.141115] nix-daemon[864]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)7014worker1 # [ 176.153439] nix-daemon[864]: reaped child process 1981, status = succeeded7015lb # [ 176.143917] niks3-server[1170]: 2026/09/25 16:52:35 INFO Received push request method=POST path=/api/pushes7016worker1 # [ 176.169937] nix-grpc-daemon[1709]: time=2026-09-25T16:52:35.492Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7017worker1 # [ 176.172117] nix-grpc-daemon[1709]: time=2026-09-25T16:52:35.494Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7018lb # [ 176.173598] niks3-server[1170]: 2026/09/25 16:52:35 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign7019lb # [ 176.179141] niks3-server[1170]: 2026/09/25 16:52:35 INFO Signed narinfos id=69 count=17020worker1 # [ 176.200143] nix-grpc-daemon[1709]: time=2026-09-25T16:52:35.522Z level=INFO msg="Uploading 1 narinfos"7021lb # [ 176.187865] niks3-server[1170]: 2026/09/25 16:52:35 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7022lb # [ 176.204095] niks3-server[1170]: 2026/09/25 16:52:35 INFO Received complete push request method=POST path=/api/pushes/69/complete7023lb # [ 176.208911] niks3-server[1170]: 2026/09/25 16:52:35 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7024worker1 # [ 176.232848] nix-grpc-daemon[1709]: time=2026-09-25T16:52:35.555Z level=INFO msg="Upload complete. (173ms)"7025lb # [ 176.216380] niks3-server[1170]: 2026/09/25 16:52:35 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7026worker1 # [ 176.237444] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=5047027worker1 # [ 176.243405] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547028worker1 # [ 176.254784] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7029worker1 # [ 176.258420] nix-daemon[864]: reaped child process 1976, status = succeeded7030worker1 # [ 176.263365] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7031worker1 # [ 177.300869] nix-daemon[864]: reaped child process 1985, status = succeeded7032lb # [ 177.338144] envoy[919]: conn peer=[2001:db8:1::1]:53296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9660 tx=5179 ms=50027033worker1 # [ 177.384367] nix-daemon[864]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)7034worker1 # [ 177.396961] nix-daemon[864]: reaped child process 2004, status = succeeded7035lb # [ 177.385693] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received push request method=POST path=/api/pushes7036worker1 # [ 177.411271] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7037worker1 # [ 177.412749] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.735Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7038lb # [ 177.429437] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7039lb # [ 177.433774] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign7040lb # [ 177.438455] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7041lb # [ 177.445144] niks3-server[1170]: 2026/09/25 16:52:36 INFO Signed narinfos id=70 count=17042worker1 # [ 177.466167] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.788Z level=INFO msg="Uploading 1 narinfos"7043lb # [ 177.451902] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7044lb # [ 177.462354] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received complete push request method=POST path=/api/pushes/70/complete7045lb # [ 177.467171] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7046worker1 # [ 177.484630] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.807Z level=INFO msg="Upload complete. (214ms)"7047worker1 # [ 177.493574] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=10 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07048worker1 # [ 177.503348] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17049worker1 # [ 177.508230] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547050worker1 # [ 177.511318] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07051worker1 # [ 177.520967] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547052worker1 # [ 177.525506] nix-daemon[864]: reaped child process 1983, status = succeeded7053worker1 # [ 177.527679] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547054worker1 # [ 177.531420] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7055worker1 # [ 177.618604] nix-daemon[864]: accepted connection from pid 2010, user nix-grpc-daemon (trusted)7056worker1 # [ 177.631196] nix-daemon[864]: reaped child process 2013, status = succeeded7057lb # [ 177.620853] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received push request method=POST path=/api/pushes7058worker1 # [ 177.646492] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7059worker1 # [ 177.648433] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.970Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7060lb # [ 177.650183] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign7061lb # [ 177.654971] niks3-server[1170]: 2026/09/25 16:52:36 INFO Signed narinfos id=71 count=17062worker1 # [ 177.677170] nix-grpc-daemon[1709]: time=2026-09-25T16:52:36.999Z level=INFO msg="Uploading 1 narinfos"7063lb # [ 177.664144] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7064lb # [ 177.675924] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7065lb # [ 177.682708] niks3-server[1170]: 2026/09/25 16:52:36 INFO Received complete push request method=POST path=/api/pushes/71/complete7066lb # [ 177.690701] niks3-server[1170]: 2026/09/25 16:52:36 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7067worker1 # [ 177.712251] nix-grpc-daemon[1709]: time=2026-09-25T16:52:37.034Z level=INFO msg="Upload complete. (175ms)"7068worker1 # [ 177.716210] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=5047069worker1 # [ 177.720647] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547070worker1 # [ 177.730056] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7071worker1 # [ 177.739440] nix-daemon[864]: reaped child process 2008, status = succeeded7072worker1 # [ 177.742252] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7073worker1 # [ 178.556458] nix-daemon[864]: reaped child process 2017, status = succeeded7074worker1 # [ 178.633771] nix-daemon[864]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)7075worker1 # [ 178.646757] nix-daemon[864]: reaped child process 2036, status = succeeded7076lb # [ 178.637828] niks3-server[1170]: 2026/09/25 16:52:37 INFO Received push request method=POST path=/api/pushes7077worker1 # [ 178.662585] nix-grpc-daemon[1709]: time=2026-09-25T16:52:37.984Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7078worker1 # [ 178.664168] nix-grpc-daemon[1709]: time=2026-09-25T16:52:37.986Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7079lb # [ 178.674210] niks3-server[1170]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7080lb # [ 178.684342] niks3-server[1170]: 2026/09/25 16:52:37 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign7081lb # [ 178.687968] niks3-server[1170]: 2026/09/25 16:52:37 INFO Signed narinfos id=72 count=17082worker1 # [ 178.708832] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.031Z level=INFO msg="Uploading 1 narinfos"7083lb # [ 178.696462] niks3-server[1170]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7084lb # [ 178.700260] niks3-server[1170]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7085lb # [ 178.704595] niks3-server[1170]: 2026/09/25 16:52:37 INFO Received complete push request method=POST path=/api/pushes/72/complete7086worker1 # [ 178.729330] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.051Z level=INFO msg="Upload complete. (180ms)"7087lb # [ 178.713125] niks3-server[1170]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7088worker1 # [ 178.737685] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07089worker1 # [ 178.745236] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17090worker1 # [ 178.750281] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547091worker1 # [ 178.753328] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07092worker1 # [ 178.760258] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547093worker1 # [ 178.762463] nix-daemon[864]: reaped child process 2015, status = succeeded7094worker1 # [ 178.765866] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547095worker1 # [ 178.769750] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7096worker1 # [ 178.854636] nix-daemon[864]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7097worker1 # [ 178.867134] nix-daemon[864]: reaped child process 2045, status = succeeded7098lb # [ 178.857166] niks3-server[1170]: 2026/09/25 16:52:38 INFO Received push request method=POST path=/api/pushes7099worker1 # [ 178.883327] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.205Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7100worker1 # [ 178.884769] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.207Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7101lb # [ 178.887973] niks3-server[1170]: 2026/09/25 16:52:38 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign7102worker1 # [ 178.910613] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.233Z level=INFO msg="Uploading 1 narinfos"7103lb # [ 178.892444] niks3-server[1170]: 2026/09/25 16:52:38 INFO Signed narinfos id=73 count=17104lb # [ 178.898303] niks3-server[1170]: 2026/09/25 16:52:38 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7105lb # [ 178.908984] niks3-server[1170]: 2026/09/25 16:52:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7106lb # [ 178.911707] niks3-server[1170]: 2026/09/25 16:52:38 INFO Received complete push request method=POST path=/api/pushes/73/complete7107worker1 # [ 178.935412] nix-grpc-daemon[1709]: time=2026-09-25T16:52:38.258Z level=INFO msg="Upload complete. (160ms)"7108lb # [ 178.919859] niks3-server[1170]: 2026/09/25 16:52:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7109worker1 # [ 178.937870] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=5047110worker1 # [ 178.941967] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547111worker1 # [ 178.947564] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7112worker1 # [ 178.952109] nix-daemon[864]: reaped child process 2040, status = succeeded7113worker1 # [ 178.955392] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7114worker1 # [ 180.371652] nix-daemon[864]: reaped child process 2049, status = succeeded7115worker1 # [ 180.450086] nix-daemon[864]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7116worker1 # [ 180.463238] nix-daemon[864]: reaped child process 2068, status = succeeded7117lb # [ 180.449392] niks3-server[1170]: 2026/09/25 16:52:39 INFO Received push request method=POST path=/api/pushes7118worker1 # [ 180.476131] nix-grpc-daemon[1709]: time=2026-09-25T16:52:39.798Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7119worker1 # [ 180.478718] nix-grpc-daemon[1709]: time=2026-09-25T16:52:39.800Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7120lb # [ 180.486356] niks3-server[1170]: 2026/09/25 16:52:39 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7121lb # [ 180.499156] niks3-server[1170]: 2026/09/25 16:52:39 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7122lb # [ 180.505653] niks3-server[1170]: 2026/09/25 16:52:39 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7123worker1 # [ 180.528704] nix-grpc-daemon[1709]: time=2026-09-25T16:52:39.851Z level=INFO msg="Uploading 1 narinfos"7124lb # [ 180.510912] niks3-server[1170]: 2026/09/25 16:52:39 INFO Signed narinfos id=74 count=17125lb # [ 180.522123] niks3-server[1170]: 2026/09/25 16:52:39 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7126lb # [ 180.526746] niks3-server[1170]: 2026/09/25 16:52:39 INFO Received complete push request method=POST path=/api/pushes/74/complete7127lb # [ 180.532247] niks3-server[1170]: 2026/09/25 16:52:39 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7128worker1 # [ 180.549615] nix-grpc-daemon[1709]: time=2026-09-25T16:52:39.872Z level=INFO msg="Upload complete. (186ms)"7129worker1 # [ 180.558905] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07130worker1 # [ 180.568834] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17131worker1 # [ 180.574256] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547132worker1 # [ 180.577353] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07133worker1 # [ 180.587317] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547134worker1 # [ 180.590260] nix-daemon[864]: reaped child process 2047, status = succeeded7135worker1 # [ 180.595707] nix-grpc-daemon[1709]: time=2026-09-25T16:52:39.918Z level=INFO msg="All 1 paths already cached"7136worker1 # [ 180.600258] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547137worker1 # [ 180.604853] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7138worker1 # [ 180.686962] nix-daemon[864]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7139worker1 # [ 180.699637] nix-daemon[864]: reaped child process 2077, status = succeeded7140lb # [ 180.687243] niks3-server[1170]: 2026/09/25 16:52:39 INFO Received push request method=POST path=/api/pushes7141worker1 # [ 180.714211] nix-grpc-daemon[1709]: time=2026-09-25T16:52:40.036Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7142worker1 # [ 180.716192] nix-grpc-daemon[1709]: time=2026-09-25T16:52:40.038Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7143lb # [ 180.719198] niks3-server[1170]: 2026/09/25 16:52:39 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7144lb # [ 180.727646] niks3-server[1170]: 2026/09/25 16:52:39 INFO Signed narinfos id=75 count=17145worker1 # [ 180.748963] nix-grpc-daemon[1709]: time=2026-09-25T16:52:40.071Z level=INFO msg="Uploading 1 narinfos"7146lb # [ 180.731324] niks3-server[1170]: 2026/09/25 16:52:40 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7147lb # [ 180.740103] niks3-server[1170]: 2026/09/25 16:52:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7148lb # [ 180.750235] niks3-server[1170]: 2026/09/25 16:52:40 INFO Received complete push request method=POST path=/api/pushes/75/complete7149worker1 # [ 180.775849] nix-grpc-daemon[1709]: time=2026-09-25T16:52:40.098Z level=INFO msg="Upload complete. (166ms)"7150lb # [ 180.758549] niks3-server[1170]: 2026/09/25 16:52:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7151worker1 # [ 180.779715] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=6247152worker1 # [ 180.784409] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547153worker1 # [ 180.795113] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7154worker1 # [ 180.799501] nix-daemon[864]: reaped child process 2072, status = succeeded7155worker1 # [ 180.804193] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7156worker1 # [ 182.242698] nix-daemon[864]: reaped child process 2081, status = succeeded7157worker1 # [ 182.327087] nix-daemon[864]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7158worker1 # [ 182.340632] nix-daemon[864]: reaped child process 2100, status = succeeded7159lb # [ 182.326636] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received push request method=POST path=/api/pushes7160worker1 # [ 182.353979] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.676Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7161worker1 # [ 182.356056] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.678Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7162lb # [ 182.369654] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7163lb # [ 182.380765] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7164lb # [ 182.386737] niks3-server[1170]: 2026/09/25 16:52:41 INFO Signed narinfos id=76 count=17165worker1 # [ 182.407844] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.730Z level=INFO msg="Uploading 1 narinfos"7166lb # [ 182.394397] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7167lb # [ 182.405438] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received complete push request method=POST path=/api/pushes/76/complete7168lb # [ 182.410495] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7169worker1 # [ 182.427867] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.750Z level=INFO msg="Upload complete. (215ms)"7170lb # [ 182.413305] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7171worker1 # [ 182.438535] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=13 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07172worker1 # [ 182.447288] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17173worker1 # [ 182.451196] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547174worker1 # [ 182.454206] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07175worker1 # [ 182.463297] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547176worker1 # [ 182.466297] nix-daemon[864]: reaped child process 2079, status = succeeded7177worker1 # [ 182.470495] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.793Z level=INFO msg="All 1 paths already cached"7178worker1 # [ 182.474937] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547179worker1 # [ 182.479558] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7180worker1 # [ 182.561388] nix-daemon[864]: accepted connection from pid 2106, user nix-grpc-daemon (trusted)7181worker1 # [ 182.575519] nix-daemon[864]: reaped child process 2109, status = succeeded7182lb # [ 182.561666] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received push request method=POST path=/api/pushes7183worker1 # [ 182.587975] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.910Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7184worker1 # [ 182.589497] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.910Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7185lb # [ 182.588948] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7186lb # [ 182.593969] niks3-server[1170]: 2026/09/25 16:52:41 INFO Signed narinfos id=77 count=17187worker1 # [ 182.615591] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.938Z level=INFO msg="Uploading 1 narinfos"7188lb # [ 182.603257] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7189lb # [ 182.609340] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7190lb # [ 182.620630] niks3-server[1170]: 2026/09/25 16:52:41 INFO Received complete push request method=POST path=/api/pushes/77/complete7191worker1 # [ 182.644959] nix-grpc-daemon[1709]: time=2026-09-25T16:52:41.967Z level=INFO msg="Upload complete. (160ms)"7192lb # [ 182.628422] niks3-server[1170]: 2026/09/25 16:52:41 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7193worker1 # [ 182.648498] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=6247194worker1 # [ 182.653313] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547195worker1 # [ 182.662720] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7196worker1 # [ 182.668112] nix-daemon[864]: reaped child process 2104, status = succeeded7197worker1 # [ 182.674891] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7198worker1 # [ 183.908755] nix-daemon[864]: reaped child process 2113, status = succeeded7199worker1 # [ 183.979695] nix-daemon[864]: accepted connection from pid 2129, user nix-grpc-daemon (trusted)7200worker1 # [ 183.990782] nix-daemon[864]: reaped child process 2132, status = succeeded7201lb # [ 183.979742] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received push request method=POST path=/api/pushes7202worker1 # [ 184.007947] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.330Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7203worker1 # [ 184.009523] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.332Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7204lb # [ 184.017876] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7205lb # [ 184.030905] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7206lb # [ 184.035883] niks3-server[1170]: 2026/09/25 16:52:43 INFO Signed narinfos id=78 count=17207worker1 # [ 184.056850] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.379Z level=INFO msg="Uploading 1 narinfos"7208lb # [ 184.043250] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7209lb # [ 184.049705] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7210lb # [ 184.058224] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received complete push request method=POST path=/api/pushes/78/complete7211lb # [ 184.064884] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7212worker1 # [ 184.081302] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.403Z level=INFO msg="Upload complete. (198ms)"7213worker1 # [ 184.092456] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07214worker1 # [ 184.102330] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17215worker1 # [ 184.106397] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547216worker1 # [ 184.110182] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07217worker1 # [ 184.119520] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547218worker1 # [ 184.123548] nix-daemon[864]: reaped child process 2111, status = succeeded7219worker1 # [ 184.127932] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.450Z level=INFO msg="All 1 paths already cached"7220worker1 # [ 184.132491] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547221worker1 # [ 184.135909] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7222worker1 # [ 184.215800] nix-daemon[864]: accepted connection from pid 2138, user nix-grpc-daemon (trusted)7223worker1 # [ 184.229433] nix-daemon[864]: reaped child process 2141, status = succeeded7224lb # [ 184.216307] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received push request method=POST path=/api/pushes7225worker1 # [ 184.241850] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.564Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7226worker1 # [ 184.243749] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.565Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7227lb # [ 184.245789] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7228lb # [ 184.255633] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7229worker1 # [ 184.277873] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.600Z level=INFO msg="Uploading 1 narinfos"7230lb # [ 184.259920] niks3-server[1170]: 2026/09/25 16:52:43 INFO Signed narinfos id=79 count=17231lb # [ 184.268349] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7232lb # [ 184.279791] niks3-server[1170]: 2026/09/25 16:52:43 INFO Received complete push request method=POST path=/api/pushes/79/complete7233worker1 # [ 184.305492] nix-grpc-daemon[1709]: time=2026-09-25T16:52:43.628Z level=INFO msg="Upload complete. (164ms)"7234lb # [ 184.288408] niks3-server[1170]: 2026/09/25 16:52:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7235worker1 # [ 184.309606] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=6247236worker1 # [ 184.315287] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547237worker1 # [ 184.324699] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7238worker1 # [ 184.332469] nix-daemon[864]: reaped child process 2136, status = succeeded7239worker1 # [ 184.337092] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7240worker1 # [ 185.372564] nix-daemon[864]: reaped child process 2145, status = succeeded7241worker1 # [ 185.455359] nix-daemon[864]: accepted connection from pid 2161, user nix-grpc-daemon (trusted)7242worker1 # [ 185.468590] nix-daemon[864]: reaped child process 2164, status = succeeded7243lb # [ 185.458376] niks3-server[1170]: 2026/09/25 16:52:44 INFO Received push request method=POST path=/api/pushes7244worker1 # [ 185.484282] nix-grpc-daemon[1709]: time=2026-09-25T16:52:44.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7245worker1 # [ 185.485802] nix-grpc-daemon[1709]: time=2026-09-25T16:52:44.808Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7246lb # [ 185.500634] niks3-server[1170]: 2026/09/25 16:52:44 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7247lb # [ 185.505496] niks3-server[1170]: 2026/09/25 16:52:44 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7248lb # [ 185.513974] niks3-server[1170]: 2026/09/25 16:52:44 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7249lb # [ 185.517628] niks3-server[1170]: 2026/09/25 16:52:44 INFO Signed narinfos id=80 count=17250worker1 # [ 185.535935] nix-grpc-daemon[1709]: time=2026-09-25T16:52:44.858Z level=INFO msg="Uploading 1 narinfos"7251lb # [ 185.523703] niks3-server[1170]: 2026/09/25 16:52:44 INFO Received complete push request method=POST path=/api/pushes/80/complete7252lb # [ 185.525393] niks3-server[1170]: 2026/09/25 16:52:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7253lb # [ 185.529939] niks3-server[1170]: 2026/09/25 16:52:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7254worker1 # [ 185.551207] nix-grpc-daemon[1709]: time=2026-09-25T16:52:44.873Z level=INFO msg="Upload complete. (206ms)"7255worker1 # [ 185.557767] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07256worker1 # [ 185.569203] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17257worker1 # [ 185.573380] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547258worker1 # [ 185.576445] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07259worker1 # [ 185.585300] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547260worker1 # [ 185.587841] nix-daemon[864]: reaped child process 2143, status = succeeded7261worker1 # [ 185.591972] nix-grpc-daemon[1709]: time=2026-09-25T16:52:44.914Z level=INFO msg="All 2 paths already cached"7262worker1 # [ 185.596720] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547263worker1 # [ 185.600404] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7264worker1 # [ 185.685629] nix-daemon[864]: accepted connection from pid 2170, user nix-grpc-daemon (trusted)7265worker1 # [ 185.698423] nix-daemon[864]: reaped child process 2173, status = succeeded7266lb # [ 185.687918] niks3-server[1170]: 2026/09/25 16:52:44 INFO Received push request method=POST path=/api/pushes7267worker1 # [ 185.714177] nix-grpc-daemon[1709]: time=2026-09-25T16:52:45.036Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7268worker1 # [ 185.716120] nix-grpc-daemon[1709]: time=2026-09-25T16:52:45.038Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7269lb # [ 185.718368] niks3-server[1170]: 2026/09/25 16:52:44 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7270lb # [ 185.722293] niks3-server[1170]: 2026/09/25 16:52:44 INFO Signed narinfos id=81 count=17271worker1 # [ 185.743336] nix-grpc-daemon[1709]: time=2026-09-25T16:52:45.065Z level=INFO msg="Uploading 1 narinfos"7272lb # [ 185.731961] niks3-server[1170]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7273lb # [ 185.737626] niks3-server[1170]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7274lb # [ 185.751288] niks3-server[1170]: 2026/09/25 16:52:45 INFO Received complete push request method=POST path=/api/pushes/81/complete7275lb # [ 185.758868] niks3-server[1170]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7276worker1 # [ 185.781118] nix-grpc-daemon[1709]: time=2026-09-25T16:52:45.103Z level=INFO msg="Upload complete. (175ms)"7277worker1 # [ 185.783666] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=7527278worker1 # [ 185.789464] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547279worker1 # [ 185.800109] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7280worker1 # [ 185.807995] nix-daemon[864]: reaped child process 2168, status = succeeded7281worker1 # [ 185.812758] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7282worker1 # [ 187.450418] nix-daemon[864]: reaped child process 2177, status = succeeded7283worker1 # [ 187.533922] nix-daemon[864]: accepted connection from pid 2193, user nix-grpc-daemon (trusted)7284worker1 # [ 187.547128] nix-daemon[864]: reaped child process 2196, status = succeeded7285lb # [ 187.536594] niks3-server[1170]: 2026/09/25 16:52:46 INFO Received push request method=POST path=/api/pushes7286worker1 # [ 187.565429] nix-grpc-daemon[1709]: time=2026-09-25T16:52:46.887Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7287worker1 # [ 187.566916] nix-grpc-daemon[1709]: time=2026-09-25T16:52:46.889Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7288lb # [ 187.581370] niks3-server[1170]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7289lb # [ 187.587090] niks3-server[1170]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7290lb # [ 187.594324] niks3-server[1170]: 2026/09/25 16:52:46 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7291worker1 # [ 187.618208] nix-grpc-daemon[1709]: time=2026-09-25T16:52:46.940Z level=INFO msg="Uploading 1 narinfos"7292lb # [ 187.601126] niks3-server[1170]: 2026/09/25 16:52:46 INFO Signed narinfos id=82 count=17293lb # [ 187.605985] niks3-server[1170]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7294lb # [ 187.616358] niks3-server[1170]: 2026/09/25 16:52:46 INFO Received complete push request method=POST path=/api/pushes/82/complete7295lb # [ 187.625769] niks3-server[1170]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7296worker1 # [ 187.642326] nix-grpc-daemon[1709]: time=2026-09-25T16:52:46.964Z level=INFO msg="Upload complete. (220ms)"7297worker1 # [ 187.648705] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=16 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07298worker1 # [ 187.658268] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:46Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17299worker1 # [ 187.663324] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547300worker1 # [ 187.674605] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547301worker1 # [ 187.681069] nix-daemon[864]: reaped child process 2175, status = succeeded7302worker1 # [ 187.688207] nix-grpc-daemon[1709]: time=2026-09-25T16:52:47.010Z level=INFO msg="All 2 paths already cached"7303worker1 # [ 187.695774] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547304worker1 # [ 187.705427] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7305worker1 # [ 187.793799] nix-daemon[864]: accepted connection from pid 2200, user nix-grpc-daemon (trusted)7306worker1 # [ 187.806051] nix-daemon[864]: reaped child process 2203, status = succeeded7307lb # [ 187.796397] niks3-server[1170]: 2026/09/25 16:52:47 INFO Received push request method=POST path=/api/pushes7308worker1 # [ 187.823465] nix-grpc-daemon[1709]: time=2026-09-25T16:52:47.146Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7309worker1 # [ 187.824931] nix-grpc-daemon[1709]: time=2026-09-25T16:52:47.147Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7310lb # [ 187.833354] niks3-server[1170]: 2026/09/25 16:52:47 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7311lb # [ 187.839814] niks3-server[1170]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7312worker1 # [ 187.865838] nix-grpc-daemon[1709]: time=2026-09-25T16:52:47.188Z level=INFO msg="Uploading 1 narinfos"7313lb # [ 187.847312] niks3-server[1170]: 2026/09/25 16:52:47 INFO Signed narinfos id=83 count=17314lb # [ 187.852870] niks3-server[1170]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7315lb # [ 187.866160] niks3-server[1170]: 2026/09/25 16:52:47 INFO Received complete push request method=POST path=/api/pushes/83/complete7316worker1 # [ 187.891289] nix-grpc-daemon[1709]: time=2026-09-25T16:52:47.213Z level=INFO msg="Upload complete. (175ms)"7317lb # [ 187.874880] niks3-server[1170]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7318worker1 # [ 187.895340] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=7527319worker1 # [ 187.900112] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547320worker1 # [ 187.909702] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7321worker1 # [ 187.918403] nix-daemon[864]: reaped child process 2198, status = succeeded7322worker1 # [ 187.925056] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7323worker1 # [ 189.366309] nix-daemon[864]: reaped child process 2207, status = succeeded7324worker1 # [ 189.441906] nix-daemon[864]: accepted connection from pid 2223, user nix-grpc-daemon (trusted)7325worker1 # [ 189.454860] nix-daemon[864]: reaped child process 2226, status = succeeded7326lb # [ 189.446267] niks3-server[1170]: 2026/09/25 16:52:48 INFO Received push request method=POST path=/api/pushes7327worker1 # [ 189.472918] nix-grpc-daemon[1709]: time=2026-09-25T16:52:48.795Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7328worker1 # [ 189.474627] nix-grpc-daemon[1709]: time=2026-09-25T16:52:48.797Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7329lb # [ 189.490707] niks3-server[1170]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7330lb # [ 189.497988] niks3-server[1170]: 2026/09/25 16:52:48 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7331worker1 # [ 189.521867] nix-grpc-daemon[1709]: time=2026-09-25T16:52:48.844Z level=INFO msg="Uploading 1 narinfos"7332lb # [ 189.505282] niks3-server[1170]: 2026/09/25 16:52:48 INFO Signed narinfos id=84 count=17333lb # [ 189.515412] niks3-server[1170]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7334lb # [ 189.523899] niks3-server[1170]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7335lb # [ 189.533335] niks3-server[1170]: 2026/09/25 16:52:48 INFO Received complete push request method=POST path=/api/pushes/84/complete7336lb # [ 189.540638] niks3-server[1170]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7337worker1 # [ 189.562005] nix-grpc-daemon[1709]: time=2026-09-25T16:52:48.884Z level=INFO msg="Upload complete. (229ms)"7338worker1 # [ 189.569707] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=17 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07339worker1 # [ 189.578248] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:48Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17340worker1 # [ 189.582901] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547341worker1 # [ 189.594502] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547342worker1 # [ 189.600953] nix-daemon[864]: reaped child process 2205, status = succeeded7343worker1 # [ 189.607204] nix-grpc-daemon[1709]: time=2026-09-25T16:52:48.929Z level=INFO msg="All 2 paths already cached"7344worker1 # [ 189.614391] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:577547345worker1 # [ 189.622128] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7346worker1 # [ 189.708256] nix-daemon[864]: accepted connection from pid 2230, user nix-grpc-daemon (trusted)7347worker1 # [ 189.721481] nix-daemon[864]: reaped child process 2233, status = succeeded7348lb # [ 189.710860] niks3-server[1170]: 2026/09/25 16:52:48 INFO Received push request method=POST path=/api/pushes7349worker1 # [ 189.738179] nix-grpc-daemon[1709]: time=2026-09-25T16:52:49.060Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7350worker1 # [ 189.740251] nix-grpc-daemon[1709]: time=2026-09-25T16:52:49.062Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7351lb # [ 189.744997] niks3-server[1170]: 2026/09/25 16:52:49 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7352lb # [ 189.753599] niks3-server[1170]: 2026/09/25 16:52:49 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7353lb # [ 189.759562] niks3-server[1170]: 2026/09/25 16:52:49 INFO Signed narinfos id=85 count=17354worker1 # [ 189.783173] nix-grpc-daemon[1709]: time=2026-09-25T16:52:49.105Z level=INFO msg="Uploading 1 narinfos"7355lb # [ 189.767170] niks3-server[1170]: 2026/09/25 16:52:49 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7356lb # [ 189.783606] niks3-server[1170]: 2026/09/25 16:52:49 INFO Received complete push request method=POST path=/api/pushes/85/complete7357worker1 # [ 189.809667] nix-grpc-daemon[1709]: time=2026-09-25T16:52:49.132Z level=INFO msg="Upload complete. (180ms)"7358lb # [ 189.792394] niks3-server[1170]: 2026/09/25 16:52:49 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7359worker1 # [ 189.814582] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=0 paths=1 nar_bytes_in=7527360worker1 # [ 189.820448] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:577547361worker1 # [ 189.831234] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7362worker1 # [ 189.841798] nix-daemon[864]: reaped child process 2228, status = succeeded7363worker1 # [ 189.849651] nix-daemon[864]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7364worker1 # [ 191.080419] nix-daemon[864]: reaped child process 2237, status = succeeded7365worker1 # [ 191.156424] nix-daemon[864]: accepted connection from pid 2253, user nix-grpc-daemon (trusted)7366worker1 # [ 191.168338] nix-daemon[864]: reaped child process 2256, status = succeeded7367lb # [ 191.157348] niks3-server[1170]: 2026/09/25 16:52:50 INFO Received push request method=POST path=/api/pushes7368worker1 # [ 191.185983] nix-grpc-daemon[1709]: time=2026-09-25T16:52:50.508Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7369worker1 # [ 191.187498] nix-grpc-daemon[1709]: time=2026-09-25T16:52:50.510Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7370lb # [ 191.198305] niks3-server[1170]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7371lb # [ 191.207446] niks3-server[1170]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7372lb # [ 191.214753] niks3-server[1170]: 2026/09/25 16:52:50 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7373worker1 # [ 191.236425] nix-grpc-daemon[1709]: time=2026-09-25T16:52:50.558Z level=INFO msg="Uploading 1 narinfos"7374lb # [ 191.218255] niks3-server[1170]: 2026/09/25 16:52:50 INFO Signed narinfos id=86 count=17375lb # [ 191.223764] niks3-server[1170]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7376lb # [ 191.227316] niks3-server[1170]: 2026/09/25 16:52:50 INFO Received complete push request method=POST path=/api/pushes/86/complete7377worker1 # [ 191.250195] nix-grpc-daemon[1709]: time=2026-09-25T16:52:50.572Z level=INFO msg="Upload complete. (193ms)"7378lb # [ 191.233632] niks3-server[1170]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7379worker1 # [ 191.256141] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57754 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=18 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07380worker1 # [ 191.269411] nix-daemon[864]: reaped child process 2235, status = succeeded7381client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.49 seconds)7382worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7383worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7384client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7385client # [ 191.369738] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7386client # Running as unit: gpu.service; invocation ID: df21da13d65a4e42886a1ecb006e47287387client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7388lb: must succeed: curl -sf localhost:9901/clusters7389lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7390lb: must succeed: curl -sf localhost:9901/clusters7391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7392worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7393worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7394worker2 # [ 191.503847] nix-grpc-daemon[1870]: ts=2026-09-25T16:52:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567527395client # [ 191.509660] nix[1598]: don't know how to build these paths:7396client # [ 191.510623] nix[1598]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7397worker1 # [ 191.526740] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:577547398client # [ 191.530218] nix[1598]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7399worker1 # [ 191.545419] nix-grpc-daemon[1635]: ts=2026-09-25T16:52:50Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07400lb: must succeed: curl -sf localhost:9901/clusters7401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7402lb: must succeed: curl -sf localhost:9901/clusters7403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7405worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7406client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7407lb # [ 197.338322] envoy[919]: conn peer=[2001:db8:1::1]:49532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27098 tx=12298 ms=153217408client # [ 221.531572] nix[1598]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7409client # [ 221.540849] nix[1598]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7410client # [ 221.571704] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7411client # [ 221.574727] systemd[1]: gpu.service: Failed with result 'exit-code'.7412client # [ 221.579088] systemd[1]: gpu.service: Consumed 154ms CPU time over 30.202s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7413client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.40 seconds)7414client: must succeed: journalctl -u gpu -o cat7415client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7416lb: must succeed: curl -sf localhost:9901/clusters7417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7418lb: must succeed: curl -sf localhost:9901/clusters7419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7420worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7421worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7422(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.37 seconds)7423subtest: low disk drains a worker and builds go to the other7424worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7425worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7426worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7427worker1 # [ 223.954640] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:23Z level=info event=unhealthy reason=min_free7428worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)7429client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27430client # don't know how to build these paths:7431client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7432worker2 # [ 224.301846] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567527433worker2 # [ 224.308462] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567527434worker1 # [ 224.344556] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:23Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07435worker1 # [ 224.349284] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:23Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17436worker2 # [ 224.336126] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527437worker2 # [ 224.348127] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567527438client # copying 1 paths to 'lb:50051'...7439client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7440worker2 # [ 224.361530] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567527441worker2 # [ 224.372316] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7442worker2 # [ 224.477023] nix-daemon[858]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7443worker2 # [ 224.489372] nix-daemon[858]: reaped child process 2028, status = succeeded7444lb # [ 224.494363] niks3-server[1170]: 2026/09/25 16:53:23 INFO Received push request method=POST path=/api/pushes7445worker2 # [ 224.508880] nix-grpc-daemon[1930]: time=2026-09-25T16:53:23.800Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7446worker2 # [ 224.510523] nix-grpc-daemon[1930]: time=2026-09-25T16:53:23.801Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7447lb # [ 224.534070] niks3-server[1170]: 2026/09/25 16:53:23 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7448lb # [ 224.549164] niks3-server[1170]: 2026/09/25 16:53:23 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7449lb # [ 224.554238] niks3-server[1170]: 2026/09/25 16:53:23 INFO Signed narinfos id=87 count=17450worker2 # [ 224.560887] nix-grpc-daemon[1930]: time=2026-09-25T16:53:23.851Z level=INFO msg="Uploading 1 narinfos"7451lb # [ 224.572262] niks3-server[1170]: 2026/09/25 16:53:23 INFO Received complete push request method=POST path=/api/pushes/87/complete7452lb # [ 224.576216] niks3-server[1170]: 2026/09/25 16:53:23 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7453lb # [ 224.583191] niks3-server[1170]: 2026/09/25 16:53:23 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7454worker2 # [ 224.585366] nix-grpc-daemon[1930]: time=2026-09-25T16:53:23.876Z level=INFO msg="Upload complete. (198ms)"7455worker2 # [ 224.590823] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 paths=1 nar_bytes_in=5127456worker2 # [ 224.597636] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527457worker2 # [ 224.611108] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7458worker2 # [ 224.620355] nix-daemon[858]: reaped child process 2023, status = succeeded7459client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7460worker2 # [ 224.628319] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7461client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7462client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7463client # farm-a> LOG-farm-a-drain7464worker2 # [ 225.268611] nix-daemon[858]: reaped child process 2032, status = succeeded7465worker2 # [ 225.353891] nix-daemon[858]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)7466worker2 # [ 225.366464] nix-daemon[858]: reaped child process 2051, status = succeeded7467lb # [ 225.370625] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received push request method=POST path=/api/pushes7468worker2 # [ 225.383303] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.674Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7469worker2 # [ 225.385160] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.676Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7470lb # [ 225.410500] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7471lb # [ 225.424108] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7472lb # [ 225.435104] niks3-server[1170]: 2026/09/25 16:53:24 INFO Signed narinfos id=88 count=17473worker2 # [ 225.440049] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.731Z level=INFO msg="Uploading 1 narinfos"7474lb # [ 225.437358] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7475lb # [ 225.440867] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7476lb # [ 225.447079] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received complete push request method=POST path=/api/pushes/88/complete7477lb # [ 225.453997] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7478worker2 # [ 225.455692] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.747Z level=INFO msg="Upload complete. (218ms)"7479worker2 # [ 225.462803] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=19 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07480worker1 # [ 225.484829] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:24Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07481worker2 # [ 225.473406] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527482worker1 # [ 225.489233] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:24Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17483worker2 # [ 225.485507] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567527484worker2 # [ 225.492266] nix-daemon[858]: reaped child process 2030, status = succeeded7485worker2 # [ 225.498376] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.789Z level=INFO msg="All 1 paths already cached"7486client # copying 1 paths to 'lb:50051'...7487client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7488worker2 # [ 225.505480] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567527489worker2 # [ 225.513806] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7490worker2 # [ 225.596075] nix-daemon[858]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)7491worker2 # [ 225.608956] nix-daemon[858]: reaped child process 2058, status = succeeded7492lb # [ 225.613580] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received push request method=POST path=/api/pushes7493worker2 # [ 225.625245] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.916Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7494worker2 # [ 225.626966] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.918Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7495lb # [ 225.645804] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7496worker2 # [ 225.654930] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.946Z level=INFO msg="Uploading 1 narinfos"7497lb # [ 225.651927] niks3-server[1170]: 2026/09/25 16:53:24 INFO Signed narinfos id=89 count=17498lb # [ 225.656066] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7499lb # [ 225.663193] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7500lb # [ 225.670727] niks3-server[1170]: 2026/09/25 16:53:24 INFO Received complete push request method=POST path=/api/pushes/89/complete7501worker2 # [ 225.679873] nix-grpc-daemon[1930]: time=2026-09-25T16:53:24.971Z level=INFO msg="Upload complete. (160ms)"7502lb # [ 225.678224] niks3-server[1170]: 2026/09/25 16:53:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7503worker2 # [ 225.682960] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 paths=1 nar_bytes_in=6247504worker2 # [ 225.688209] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527505worker2 # [ 225.697629] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7506worker2 # [ 225.701465] nix-daemon[858]: reaped child process 2053, status = succeeded7507worker2 # [ 225.706295] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7508client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7509client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7510client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7511client # farm-b> LOG-farm-b-drain7512worker2 # [ 226.519701] nix-daemon[858]: reaped child process 2062, status = succeeded7513worker2 # [ 226.595084] nix-daemon[858]: accepted connection from pid 2078, user nix-grpc-daemon (trusted)7514worker2 # [ 226.606653] nix-daemon[858]: reaped child process 2081, status = succeeded7515lb # [ 226.610360] niks3-server[1170]: 2026/09/25 16:53:25 INFO Received push request method=POST path=/api/pushes7516worker2 # [ 226.622536] nix-grpc-daemon[1930]: time=2026-09-25T16:53:25.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7517worker2 # [ 226.623978] nix-grpc-daemon[1930]: time=2026-09-25T16:53:25.915Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7518lb # [ 226.650921] niks3-server[1170]: 2026/09/25 16:53:25 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7519lb # [ 226.653883] niks3-server[1170]: 2026/09/25 16:53:25 INFO Signed narinfos id=90 count=17520worker2 # [ 226.656450] nix-grpc-daemon[1930]: time=2026-09-25T16:53:25.947Z level=INFO msg="Uploading 1 narinfos"7521lb # [ 226.656815] niks3-server[1170]: 2026/09/25 16:53:25 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7522lb # [ 226.659671] niks3-server[1170]: 2026/09/25 16:53:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7523lb # [ 226.662152] niks3-server[1170]: 2026/09/25 16:53:25 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7524lb # [ 226.668662] niks3-server[1170]: 2026/09/25 16:53:25 INFO Received complete push request method=POST path=/api/pushes/90/complete7525worker2 # [ 226.674348] nix-grpc-daemon[1930]: time=2026-09-25T16:53:25.965Z level=INFO msg="Upload complete. (162ms)"7526lb # [ 226.673473] niks3-server[1170]: 2026/09/25 16:53:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7527worker2 # [ 226.676805] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=20 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07528worker2 # [ 226.684214] nix-daemon[858]: reaped child process 2060, status = succeeded7529worker2 # [ 226.687029] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527530worker1 # [ 226.701356] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:26Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07531worker2 # [ 226.692983] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567527532worker1 # [ 226.705667] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:26Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17533worker2 # [ 226.698771] nix-grpc-daemon[1930]: time=2026-09-25T16:53:25.990Z level=INFO msg="All 2 paths already cached"7534client # copying 1 paths to 'lb:50051'...7535worker2 # [ 226.703865] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567527536client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7537worker2 # [ 226.709646] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7538worker2 # [ 226.793451] nix-daemon[858]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)7539worker2 # [ 226.805717] nix-daemon[858]: reaped child process 2088, status = succeeded7540lb # [ 226.810822] niks3-server[1170]: 2026/09/25 16:53:26 INFO Received push request method=POST path=/api/pushes7541worker2 # [ 226.822228] nix-grpc-daemon[1930]: time=2026-09-25T16:53:26.113Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7542worker2 # [ 226.824214] nix-grpc-daemon[1930]: time=2026-09-25T16:53:26.115Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7543lb # [ 226.845841] niks3-server[1170]: 2026/09/25 16:53:26 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7544lb # [ 226.850756] niks3-server[1170]: 2026/09/25 16:53:26 INFO Signed narinfos id=91 count=17545worker2 # [ 226.857784] nix-grpc-daemon[1930]: time=2026-09-25T16:53:26.149Z level=INFO msg="Uploading 1 narinfos"7546lb # [ 226.857301] niks3-server[1170]: 2026/09/25 16:53:26 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7547lb # [ 226.870980] niks3-server[1170]: 2026/09/25 16:53:26 INFO Received complete push request method=POST path=/api/pushes/91/complete7548worker2 # [ 226.882575] nix-grpc-daemon[1930]: time=2026-09-25T16:53:26.174Z level=INFO msg="Upload complete. (167ms)"7549lb # [ 226.882821] niks3-server[1170]: 2026/09/25 16:53:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7550lb # [ 226.885686] niks3-server[1170]: 2026/09/25 16:53:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7551worker2 # [ 226.886582] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=0 paths=1 nar_bytes_in=7527552worker2 # [ 226.891466] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567527553worker2 # [ 226.900539] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7554worker2 # [ 226.907541] nix-daemon[858]: reaped child process 2083, status = succeeded7555worker2 # [ 226.912498] nix-daemon[858]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7556client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7557client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7558lb # [ 227.340266] envoy[919]: conn peer=[2001:db8:1::1]:49948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300487559client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7560client # farm-top> LOG-farm-top-drain7561worker2 # [ 228.355757] nix-daemon[858]: reaped child process 2092, status = succeeded7562worker2 # [ 228.439086] nix-daemon[858]: accepted connection from pid 2108, user nix-grpc-daemon (trusted)7563worker2 # [ 228.451784] nix-daemon[858]: reaped child process 2111, status = succeeded7564lb # [ 228.456659] niks3-server[1170]: 2026/09/25 16:53:27 INFO Received push request method=POST path=/api/pushes7565worker2 # [ 228.468515] nix-grpc-daemon[1930]: time=2026-09-25T16:53:27.759Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7566worker2 # [ 228.470306] nix-grpc-daemon[1930]: time=2026-09-25T16:53:27.761Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7567lb # [ 228.503860] niks3-server[1170]: 2026/09/25 16:53:27 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7568worker2 # [ 228.511365] nix-grpc-daemon[1930]: time=2026-09-25T16:53:27.802Z level=INFO msg="Uploading 1 narinfos"7569lb # [ 228.509663] niks3-server[1170]: 2026/09/25 16:53:27 INFO Signed narinfos id=92 count=17570lb # [ 228.517144] niks3-server[1170]: 2026/09/25 16:53:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7571lb # [ 228.524248] niks3-server[1170]: 2026/09/25 16:53:27 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7572lb # [ 228.530939] niks3-server[1170]: 2026/09/25 16:53:27 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7573lb # [ 228.539112] niks3-server[1170]: 2026/09/25 16:53:27 INFO Received complete push request method=POST path=/api/pushes/92/complete7574worker2 # [ 228.547939] nix-grpc-daemon[1930]: time=2026-09-25T16:53:27.839Z level=INFO msg="Upload complete. (227ms)"7575lb # [ 228.547791] niks3-server[1170]: 2026/09/25 16:53:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7576worker2 # [ 228.555281] nix-grpc-daemon[1870]: ts=2026-09-25T16:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56752 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=21 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07577worker2 # [ 228.570157] nix-daemon[858]: reaped child process 2090, status = succeeded7578client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.43 seconds)7579client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7580client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7581worker1: must succeed: rm /nix/.rw-store/fill7582worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7583worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7584worker1 # [ 228.960355] nix-grpc-daemon[1635]: ts=2026-09-25T16:53:28Z level=info event=healthy reason=min_free7585worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7586(finished: subtest: low disk drains a worker and builds go to the other, in 7.78 seconds)7587(finished: run the VM test script, in 230.91 seconds)7588test script finished in 231.09s7589cleanup7590kill QemuMachine (pid 49)7591client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7592client # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7593client # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7594client # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7595kill QemuMachine (pid 51)7596lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7597lb # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7598lb # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7599lb # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7600kill QemuMachine (pid 50)7601worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7602worker1 # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7603worker1 # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7604worker1 # [2026-09-25T16:53:30Z INFO virtiofsd] Client disconnected, shutting down7605kill QemuMachine (pid 52)7606worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7607worker2 # [2026-09-25T16:53:31Z INFO virtiofsd] Client disconnected, shutting down7608worker2 # [2026-09-25T16:53:31Z INFO virtiofsd] Client disconnected, shutting down7609worker2 # [2026-09-25T16:53:31Z INFO virtiofsd] Client disconnected, shutting down7610(finished: cleanup, in 1.05 seconds)7611additionally exposed symbols:7612 client, lb, worker1, worker2,7613 vlan1,7614 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh