tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker1: QEMU running (pid 52) worker1 # Disk image does not exist, creating the virtualisation disk image... worker1 # Formatting '/build/vm-state-worker1/tmp.91TojKDAU9', fmt=raw size=1073741824 worker2 # Disk image does not exist, creating the virtualisation disk image... worker1 # mke2fs 1.47.4 (6-Mar-2025) worker1 # Discarding device blocks: 0/262144 done worker2 # Formatting '/build/vm-state-worker2/tmp.uiyAb6cUP4', fmt=raw size=1073741824 worker2: QEMU running (pid 51) worker2 # mke2fs 1.47.4 (6-Mar-2025) worker2 # Discarding device blocks: 0/262144 done worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes client # Disk image does not exist, creating the virtualisation disk image... worker1 # Filesystem UUID: 8851ecf9-41c2-47fe-beec-3e4751d76772 client: QEMU running (pid 49) worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # Formatting '/build/vm-state-client/tmp.xJcHzaOym9', fmt=raw size=1073741824 worker2 # Filesystem UUID: ed1e66d5-357a-4014-b002-15e4458b76d3 client # mke2fs 1.47.4 (6-Mar-2025) worker2 # Superblock backups stored on blocks: client # Discarding device blocks: 0/262144 done worker2 # 32768, 98304, 163840, 229376 client # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # client # Filesystem UUID: 76799aa6-4ca2-4066-b0be-376a60231b64 worker2 # Allocating group tables: 0/8 done client # Superblock backups stored on blocks: worker2 # Writing inode tables: 0/8 done client # 32768, 98304, 163840, 229376 worker2 # Creating journal (8192 blocks): done client # worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Allocating group tables: 0/8 done worker2 # client # Writing inode tables: 0/8 done worker2 # Virtualisation disk image created. client # Creating journal (8192 blocks): done worker2 # Starting virtiofs daemons... client # Writing superblocks and filesystem accounting information: 0/8 done worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # client # Virtualisation disk image created. worker1 # Superblock backups stored on blocks: client # Starting virtiofs daemons... worker1 # 32768, 98304, 163840, 229376 client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # Allocating group tables: 0/8 done client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # Writing inode tables: 0/8 done client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # Creating journal (8192 blocks): done client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # Writing superblocks and filesystem accounting information: 0/8 done client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Disk image does not exist, creating the virtualisation disk image... worker1 # Virtualisation disk image created. client # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Formatting '/build/vm-state-lb/tmp.wDsZraxWWG', fmt=raw size=1073741824 worker1 # Starting virtiofs daemons... client # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # Discarding device blocks: 0/262144 done worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # Filesystem UUID: e136e456-ff5d-48ae-b492-c371e2be8835 worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb: QEMU running (pid 50) lb # Superblock backups stored on blocks: worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # 32768, 98304, 163840, 229376 worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # (finished: start all VMs, in 0.22 seconds) worker1 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Allocating group tables: 0/8 done client: waiting for unit network-addresses-eth1.service worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Writing inode tables: 0/8 done client: waiting for the VM to finish booting worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Creating journal (8192 blocks): done worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # worker2 # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # Virtualisation disk image created. lb # Starting virtiofs daemons... lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:08Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:08Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T16:12:08Z INFO virtiofsd] Client connected, servicing requests client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) client # client # worker2 # worker2 # client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 client # Press Ctrl-B to configure iPXE (PCI 00:02.0)... client # client # client # client # worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker2 # worker2 # worker2 # worker2 # client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 client # Press Ctrl-B to configure iPXE (PCI 00:05.0)... client # client # worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker2 # worker2 # worker1 # worker1 # worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 lb # lb # worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker1 # worker1 # worker1 # worker1 # lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)... lb # lb # lb # lb # worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # worker1 # lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 client # Booting from ROM... worker2 # Booting from ROM... lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)... lb # lb # worker1 # Booting from ROM... lb # Booting from ROM... client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test/init regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wdxq8bvyk71y06pp5xincwaxz514413f-nixos-system-worker2-test/init regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved client # [ 0.000000] NX (Execute Disable) protection: active worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved client # [ 0.000000] APIC: Static calls initialized client # [ 0.000000] SMBIOS 2.8 present. worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker2 # [ 0.000000] NX (Execute Disable) protection: active worker2 # [ 0.000000] APIC: Static calls initialized client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker2 # [ 0.000000] SMBIOS 2.8 present. client # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.000000] Hypervisor detected: KVM worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker2 # [ 0.000000] Hypervisor detected: KVM client # [ 0.000000] kvm-clock: using sched offset of 446635667 cycles worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] kvm-clock: using sched offset of 471439904 cycles client # [ 0.000005] tsc: Detected 2400.012 MHz processor client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.002776] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs client # [ 0.002787] Using GB pages for direct mapping worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT client # [ 0.002913] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker2 # [ 0.002742] found SMP MP-table at [mem 0x000f5450-0x000f545f] client # [ 0.002923] ACPI: Early table checksum verification disabled worker2 # [ 0.002754] Using GB pages for direct mapping client # [ 0.002927] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker2 # [ 0.002848] RAMDISK: [mem 0x5e36f000-0x5ffcffff] worker2 # [ 0.002859] ACPI: Early table checksum verification disabled client # [ 0.002930] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mjibqls45h3zals0927jhbwnzswp7x1m-nixos-system-worker1-test/init regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.000000] BIOS-provided physical RAM map: worker2 # [ 0.002862] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.002934] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable worker2 # [ 0.002866] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002941] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.002943] ACPI: FACS 0x000000005FFE0000 000040 worker2 # [ 0.002870] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved worker2 # [ 0.002878] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002945] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker2 # [ 0.002880] ACPI: FACS 0x000000005FFE0000 000040 lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.002947] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.000000] BIOS-provided physical RAM map: worker2 # [ 0.002882] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.002948] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable worker2 # [ 0.002883] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.002950] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved client # [ 0.002951] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker2 # [ 0.002886] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002952] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker2 # [ 0.002888] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.002953] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker2 # [ 0.002889] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.000000] APIC: Static calls initialized client # [ 0.002953] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker1 # [ 0.000000] SMBIOS 2.8 present. lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker2 # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] client # [ 0.002953] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved worker2 # [ 0.002890] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.002954] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.000000] DMI: Memory slots populated: 1/1 lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker2 # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.002954] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker1 # [ 0.000000] Hypervisor detected: KVM client # [ 0.003186] No NUMA configuration found lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.002891] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] lb # [ 0.000000] NX (Execute Disable) protection: active client # [ 0.003186] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 lb # [ 0.000000] APIC: Static calls initialized worker2 # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.003189] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.000000] SMBIOS 2.8 present. worker2 # [ 0.003127] No NUMA configuration found worker1 # [ 0.000001] kvm-clock: using sched offset of 482185618 cycles client # [ 0.005851] Zone ranges: worker2 # [ 0.003128] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] client # [ 0.005852] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.003130] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] client # [ 0.005854] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor worker2 # [ 0.005854] Zone ranges: client # [ 0.005855] Normal empty lb # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.005856] Device empty lb # [ 0.000000] Hypervisor detected: KVM worker2 # [ 0.005855] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.005857] Movable zone start for each node worker2 # [ 0.005857] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.005858] Early memory node ranges worker1 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker2 # [ 0.005858] Normal empty worker2 # [ 0.005859] Device empty lb # [ 0.000000] kvm-clock: using sched offset of 475440794 cycles client # [ 0.005858] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker2 # [ 0.005859] Movable zone start for each node client # [ 0.005859] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.005860] Early memory node ranges worker1 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f] lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.002739] Using GB pages for direct mapping worker2 # [ 0.005861] node 0: [mem 0x0000000000001000-0x000000000009efff] client # [ 0.005860] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] lb # [ 0.000004] tsc: Detected 2400.012 MHz processor worker1 # [ 0.002899] RAMDISK: [mem 0x5e370000-0x5ffcffff] client # [ 0.005882] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.005862] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] lb # [ 0.000803] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.002906] ACPI: Early table checksum verification disabled client # [ 0.006159] On node 0, zone DMA: 97 pages in unavailable ranges worker2 # [ 0.005863] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.002909] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) lb # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs client # [ 0.033190] On node 0, zone DMA32: 40 pages in unavailable ranges worker2 # [ 0.005883] On node 0, zone DMA: 1 pages in unavailable ranges lb # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT client # [ 0.033659] ACPI: PM-Timer IO Port: 0x608 worker1 # [ 0.002913] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.006140] On node 0, zone DMA: 97 pages in unavailable ranges lb # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f] client # [ 0.033671] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.032380] On node 0, zone DMA32: 40 pages in unavailable ranges worker1 # [ 0.002917] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002733] Using GB pages for direct mapping client # [ 0.033699] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 worker2 # [ 0.032842] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.002855] RAMDISK: [mem 0x5e36e000-0x5ffcffff] worker1 # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.033701] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker2 # [ 0.032854] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) lb # [ 0.002862] ACPI: Early table checksum verification disabled worker1 # [ 0.002925] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.033703] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) lb # [ 0.002864] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker2 # [ 0.032883] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 worker1 # [ 0.002927] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.033704] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker2 # [ 0.032886] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) lb # [ 0.002868] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.033705] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.032887] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002872] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.033706] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker2 # [ 0.032888] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker1 # [ 0.002930] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002878] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.033709] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.032889] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) lb # [ 0.002880] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.033709] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker1 # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.032890] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) client # [ 0.033713] TSC deadline timer available lb # [ 0.002881] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.033717] CPU topo: Max. logical packages: 1 worker2 # [ 0.032893] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.032894] ACPI: HPET id: 0x8086a201 base: 0xfed00000 client # [ 0.033718] CPU topo: Max. logical dies: 1 worker1 # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] lb # [ 0.002883] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.032897] TSC deadline timer available client # [ 0.033718] CPU topo: Max. dies per package: 1 worker1 # [ 0.002934] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker2 # [ 0.032901] CPU topo: Max. logical packages: 1 client # [ 0.033722] CPU topo: Max. threads per core: 1 lb # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.032902] CPU topo: Max. logical dies: 1 client # [ 0.033722] CPU topo: Num. cores per package: 1 worker1 # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker2 # [ 0.032902] CPU topo: Max. dies per package: 1 client # [ 0.033723] CPU topo: Num. threads per package: 1 lb # [ 0.002886] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.032905] CPU topo: Max. threads per core: 1 worker1 # [ 0.002935] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.033723] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs lb # [ 0.002887] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker2 # [ 0.032906] CPU topo: Num. cores per package: 1 worker1 # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] client # [ 0.033742] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker2 # [ 0.032906] CPU topo: Num. threads per package: 1 lb # [ 0.002888] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.002936] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker2 # [ 0.032906] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs client # [ 0.033773] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.003155] No NUMA configuration found lb # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker2 # [ 0.032924] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.033775] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.002889] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker1 # [ 0.003156] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] client # [ 0.033776] [mem 0x60000000-0xafffffff] available for PCI devices worker2 # [ 0.032953] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.003158] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker1 # [ 0.005368] Zone ranges: client # [ 0.033778] Booting paravirtualized kernel on KVM worker2 # [ 0.032955] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.002890] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker1 # [ 0.005369] DMA [mem 0x0000000000001000-0x0000000000ffffff] worker2 # [ 0.032956] [mem 0x60000000-0xafffffff] available for PCI devices worker1 # [ 0.005370] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] lb # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.033781] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.005371] Normal empty worker2 # [ 0.032958] Booting paravirtualized kernel on KVM lb # [ 0.003120] No NUMA configuration found worker1 # [ 0.005372] Device empty client # [ 0.038251] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.005372] Movable zone start for each node lb # [ 0.003121] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker2 # [ 0.032961] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns client # [ 0.040393] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker1 # [ 0.005373] Early memory node ranges lb # [ 0.003124] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.005268] Zone ranges: client # [ 0.040448] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.005373] node 0: [mem 0x0000000000001000-0x000000000009efff] worker2 # [ 0.037441] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 lb # [ 0.005269] DMA [mem 0x0000000000001000-0x0000000000ffffff] worker1 # [ 0.005374] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.039593] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 lb # [ 0.005270] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] worker2 # [ 0.039638] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.005376] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] lb # [ 0.005271] Normal empty lb # [ 0.005272] Device empty worker1 # [ 0.005397] On node 0, zone DMA: 1 pages in unavailable ranges lb # [ 0.005272] Movable zone start for each node worker1 # [ 0.005652] On node 0, zone DMA: 97 pages in unavailable ranges lb # [ 0.005273] Early memory node ranges client # [ 0.040449] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test/init regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.030835] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.005273] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.031282] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.005274] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker1 # [ 0.031292] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.039640] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wdxq8bvyk71y06pp5xincwaxz514413f-nixos-system-worker2-test/init regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.040544] Unknown kernel command line parameters "regInfo=/nix/store/mfd16v4b7h8ls3x1mqxsddn2cfna0fbk-closure-info/registration", will be passed to user space. lb # [ 0.005275] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] client # [ 0.040556] random: crng init done worker1 # [ 0.031319] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.005293] On node 0, zone DMA: 1 pages in unavailable ranges worker1 # [ 0.031321] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) client # [ 0.040557] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes lb # [ 0.005543] On node 0, zone DMA: 97 pages in unavailable ranges worker2 # [ 0.039733] Unknown kernel command line parameters "regInfo=/nix/store/qzdpdlggq3mji3l0y7vsv5dazbj5alv6-closure-info/registration", will be passed to user space. worker1 # [ 0.031322] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker2 # [ 0.039745] random: crng init done lb # [ 0.030610] On node 0, zone DMA32: 40 pages in unavailable ranges client # [ 0.042650] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.031069] ACPI: PM-Timer IO Port: 0x608 worker1 # [ 0.031324] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker2 # [ 0.039746] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes client # [ 0.043700] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.031079] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) client # [ 0.043730] Fallback order for Node 0: 0 worker1 # [ 0.031324] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.041863] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.031106] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 client # [ 0.043733] Built 1 zonelists, mobility grouping on. Total pages: 393078 worker1 # [ 0.031325] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) client # [ 0.043733] Policy zone: DMA32 worker2 # [ 0.042930] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.031108] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.031327] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.042963] Fallback order for Node 0: 0 client # [ 0.046220] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.031110] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.031328] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker2 # [ 0.042966] Built 1 zonelists, mobility grouping on. Total pages: 393078 worker1 # [ 0.031332] TSC deadline timer available client # [ 0.049581] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.031111] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker2 # [ 0.042967] Policy zone: DMA32 worker1 # [ 0.031336] CPU topo: Max. logical packages: 1 client # [ 0.052848] allocated 3145728 bytes of page_ext lb # [ 0.031112] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.045637] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.031336] CPU topo: Max. logical dies: 1 client # [ 0.062707] ftrace: allocating 48787 entries in 192 pages worker1 # [ 0.031337] CPU topo: Max. dies per package: 1 worker2 # [ 0.048757] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.062710] ftrace: allocated 192 pages with 2 groups lb # [ 0.031113] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker1 # [ 0.031340] CPU topo: Max. threads per core: 1 client # [ 0.063531] Dynamic Preempt: lazy worker2 # [ 0.052080] allocated 3145728 bytes of page_ext lb # [ 0.031115] ACPI: Using ACPI (MADT) for SMP configuration information worker1 # [ 0.031341] CPU topo: Num. cores per package: 1 client # [ 0.063679] rcu: Preemptible hierarchical RCU implementation. worker2 # [ 0.061933] ftrace: allocating 48787 entries in 192 pages lb # [ 0.031116] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker1 # [ 0.031341] CPU topo: Num. threads per package: 1 client # [ 0.063680] rcu: RCU event tracing is enabled. worker2 # [ 0.061935] ftrace: allocated 192 pages with 2 groups lb # [ 0.031119] TSC deadline timer available worker2 # [ 0.062774] Dynamic Preempt: lazy worker1 # [ 0.031341] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs lb # [ 0.031123] CPU topo: Max. logical packages: 1 client # [ 0.063680] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.062950] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.031124] CPU topo: Max. logical dies: 1 client # [ 0.063682] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.031358] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker2 # [ 0.062951] rcu: RCU event tracing is enabled. lb # [ 0.031124] CPU topo: Max. dies per package: 1 client # [ 0.063682] Rude variant of Tasks RCU enabled. worker1 # [ 0.031386] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] lb # [ 0.031128] CPU topo: Max. threads per core: 1 worker2 # [ 0.062951] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.063682] Tracing variant of Tasks RCU enabled. lb # [ 0.031128] CPU topo: Num. cores per package: 1 worker2 # [ 0.062953] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.031388] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] client # [ 0.063683] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.031129] CPU topo: Num. threads per package: 1 worker2 # [ 0.062953] Rude variant of Tasks RCU enabled. worker1 # [ 0.031389] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.063684] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.031129] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.062954] Tracing variant of Tasks RCU enabled. worker1 # [ 0.031390] Booting paravirtualized kernel on KVM worker2 # [ 0.062954] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.031145] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.063705] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.031392] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker2 # [ 0.062955] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.031174] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] client # [ 0.063706] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.035865] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 lb # [ 0.031175] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker2 # [ 0.062964] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.063707] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.037905] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 lb # [ 0.031177] [mem 0x60000000-0xafffffff] available for PCI devices worker2 # [ 0.062966] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.068057] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 lb # [ 0.031178] Booting paravirtualized kernel on KVM worker1 # [ 0.037948] kvm-guest: PV spinlocks disabled, single CPU client # [ 0.068328] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.062967] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.031181] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker2 # [ 0.067346] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 client # [ 0.068336] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns lb # [ 0.035637] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker2 # [ 0.067624] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.037706] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 client # [ 0.068436] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.037748] kvm-guest: PV spinlocks disabled, single CPU client # [ 0.071969] Console: colour VGA+ 80x25 worker2 # [ 0.067632] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.037949] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mjibqls45h3zals0927jhbwnzswp7x1m-nixos-system-worker1-test/init regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.071971] printk: legacy console [tty0] enabled client # [ 0.101567] printk: legacy console [ttyS0] enabled worker2 # [ 0.067736] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.071271] Console: colour VGA+ 80x25 client # [ 0.199286] ACPI: Core revision 20250807 worker1 # [ 0.038038] Unknown kernel command line parameters "regInfo=/nix/store/2iln1bcw2hgn0z2cnks06n8y84wjkj86-closure-info/registration", will be passed to user space. worker2 # [ 0.071274] printk: legacy console [tty0] enabled worker1 # [ 0.038051] random: crng init done worker2 # [ 0.100827] printk: legacy console [ttyS0] enabled worker1 # [ 0.038052] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes client # [ 0.200231] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns lb # [ 0.037749] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.198699] ACPI: Core revision 20250807 worker1 # [ 0.040211] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.201760] APIC: Switch to symmetric I/O mode setup worker1 # [ 0.041220] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.199586] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns client # [ 0.202739] x2apic enabled lb # [ 0.037839] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space. worker1 # [ 0.041249] Fallback order for Node 0: 0 lb # [ 0.037852] random: crng init done worker2 # [ 0.201111] APIC: Switch to symmetric I/O mode setup client # [ 0.203523] APIC: Switched APIC routing to: physical x2apic worker1 # [ 0.041252] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.037852] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker1 # [ 0.041253] Policy zone: DMA32 worker2 # [ 0.202095] x2apic enabled lb # [ 0.040040] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.043639] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.202884] APIC: Switched APIC routing to: physical x2apic client # [ 0.205314] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker1 # [ 0.046509] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.041054] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.049654] allocated 3145728 bytes of page_ext lb # [ 0.041083] Fallback order for Node 0: 0 worker1 # [ 0.058976] ftrace: allocating 48787 entries in 192 pages lb # [ 0.041086] Built 1 zonelists, mobility grouping on. Total pages: 393078 worker2 # [ 0.204684] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.206282] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.041087] Policy zone: DMA32 worker1 # [ 0.058977] ftrace: allocated 192 pages with 2 groups worker1 # [ 0.059969] Dynamic Preempt: lazy lb # [ 0.043437] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.060134] rcu: Preemptible hierarchical RCU implementation. client # [ 0.207926] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker2 # [ 0.205649] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.046349] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.060135] rcu: RCU event tracing is enabled. lb # [ 0.049479] allocated 3145728 bytes of page_ext client # [ 0.209012] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.060136] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.058866] ftrace: allocating 48787 entries in 192 pages worker2 # [ 0.207292] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker1 # [ 0.060137] Trampoline variant of Tasks RCU enabled. lb # [ 0.058869] ftrace: allocated 192 pages with 2 groups client # [ 0.210072] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker1 # [ 0.060137] Rude variant of Tasks RCU enabled. lb # [ 0.059842] Dynamic Preempt: lazy worker2 # [ 0.208378] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.060137] Tracing variant of Tasks RCU enabled. lb # [ 0.060001] rcu: Preemptible hierarchical RCU implementation. client # [ 0.210923] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 lb # [ 0.060002] rcu: RCU event tracing is enabled. worker2 # [ 0.209418] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker1 # [ 0.060138] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.060003] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.060139] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.210289] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 client # [ 0.211927] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto lb # [ 0.060004] Trampoline variant of Tasks RCU enabled. lb # [ 0.060004] Rude variant of Tasks RCU enabled. worker1 # [ 0.060158] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.213923] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl lb # [ 0.060005] Tracing variant of Tasks RCU enabled. worker2 # [ 0.211293] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker1 # [ 0.060160] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.060005] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. client # [ 0.214923] Transient Scheduler Attacks: Vulnerable: No microcode worker2 # [ 0.213289] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker1 # [ 0.060161] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.060006] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.215923] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker1 # [ 0.064469] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker2 # [ 0.214290] Transient Scheduler Attacks: Vulnerable: No microcode lb # [ 0.060022] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.216923] Speculative Return Stack Overflow: Mitigation: Safe RET worker1 # [ 0.064744] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.215289] Spectre V2 : Mitigation: Enhanced / Automatic IBRS lb # [ 0.060024] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.064751] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.216289] Speculative Return Stack Overflow: Mitigation: Safe RET lb # [ 0.060024] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.064850] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.064313] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.068343] Console: colour VGA+ 80x25 lb # [ 0.064589] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.068346] printk: legacy console [tty0] enabled client # [ 0.217923] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker1 # [ 0.099947] printk: legacy console [ttyS0] enabled lb # [ 0.064595] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns client # [ 0.219931] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.196550] ACPI: Core revision 20250807 worker2 # [ 0.217289] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization lb # [ 0.064693] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.220924] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier lb # [ 0.068368] Console: colour VGA+ 80x25 worker2 # [ 0.219297] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.197412] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns lb # [ 0.068370] printk: legacy console [tty0] enabled client # [ 0.221924] active return thunk: srso_alias_return_thunk lb # [ 0.098765] printk: legacy console [ttyS0] enabled worker1 # [ 0.198904] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.220290] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier lb # [ 0.194176] ACPI: Core revision 20250807 worker1 # [ 0.199887] x2apic enabled client # [ 0.222944] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.221290] active return thunk: srso_alias_return_thunk client # [ 0.223923] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.200664] APIC: Switched APIC routing to: physical x2apic lb # [ 0.195074] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker2 # [ 0.222312] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' client # [ 0.224923] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' lb # [ 0.196575] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.223289] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.202523] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.225923] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' lb # [ 0.197552] x2apic enabled worker2 # [ 0.225026] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' client # [ 0.226923] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' lb # [ 0.198339] APIC: Switched APIC routing to: physical x2apic worker2 # [ 0.226029] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker1 # [ 0.203502] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns client # [ 0.227923] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.227058] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' lb # [ 0.200145] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker1 # [ 0.205197] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) client # [ 0.228923] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.228056] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' client # [ 0.230808] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker1 # [ 0.206280] x86/cpu: User Mode Instruction Prevention (UMIP) activated lb # [ 0.201105] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.229072] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' client # [ 0.231625] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker1 # [ 0.208108] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker2 # [ 0.230290] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 client # [ 0.232623] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 lb # [ 0.202738] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker1 # [ 0.208882] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker2 # [ 0.231289] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 client # [ 0.233627] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.203822] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker2 # [ 0.232231] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.209947] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto client # [ 0.233923] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.204861] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker2 # [ 0.232992] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.205736] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker1 # [ 0.211194] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl client # [ 0.234923] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.233998] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker1 # [ 0.212195] Transient Scheduler Attacks: Vulnerable: No microcode lb # [ 0.206739] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker2 # [ 0.235013] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker1 # [ 0.213926] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker1 # [ 0.214891] Speculative Return Stack Overflow: Mitigation: Safe RET lb # [ 0.208736] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl lb # [ 0.209736] Transient Scheduler Attacks: Vulnerable: No microcode lb # [ 0.210735] Spectre V2 : Mitigation: Enhanced / Automatic IBRS lb # [ 0.212492] Speculative Return Stack Overflow: Mitigation: Safe RET worker1 # [ 0.215194] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker1 # [ 0.217201] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.218752] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker1 # [ 0.219195] active return thunk: srso_alias_return_thunk lb # [ 0.213523] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization lb # [ 0.214743] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.221097] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' lb # [ 0.216292] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker1 # [ 0.222194] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' lb # [ 0.216736] active return thunk: srso_alias_return_thunk worker1 # [ 0.223194] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' lb # [ 0.218617] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker1 # [ 0.224194] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' lb # [ 0.219735] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.225193] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' lb # [ 0.220735] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker1 # [ 0.226194] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' lb # [ 0.221736] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker1 # [ 0.227194] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' lb # [ 0.222735] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker1 # [ 0.228195] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 lb # [ 0.223735] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker1 # [ 0.229948] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker1 # [ 0.230954] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 lb # [ 0.224735] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker1 # [ 0.231948] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.225735] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker1 # [ 0.232955] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.226735] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 lb # [ 0.227735] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.233949] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. lb # [ 0.228735] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.229735] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.230735] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. client # [ 0.269760] Freeing SMP alternatives memory: 44K client # [ 0.270501] pid_max: default: 32768 minimum: 301 worker2 # [ 0.269444] Freeing SMP alternatives memory: 44K client # [ 0.271032] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.272025] landlock: Up and running. worker2 # [ 0.270243] pid_max: default: 32768 minimum: 301 client # [ 0.272924] Yama: becoming mindful. worker2 # [ 0.270945] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.274015] LSM support for eBPF active worker2 # [ 0.272186] landlock: Up and running. worker2 # [ 0.272755] Yama: becoming mindful. client # [ 0.274775] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.273503] LSM support for eBPF active client # [ 0.275965] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.274411] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.275331] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.278337] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.279511] Performance Events: Fam17h+ core perfctr, AMD PMU driver. client # [ 0.279932] ... version: 2 worker2 # [ 0.278526] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.280655] ... bit width: 48 client # [ 0.280926] ... generic counters: 6 worker2 # [ 0.279947] Performance Events: Fam17h+ core perfctr, AMD PMU driver. client # [ 0.281633] ... generic bitmap: 000000000000003f worker2 # [ 0.280298] ... version: 2 client # [ 0.281925] ... fixed-purpose counters: 0 worker2 # [ 0.281039] ... bit width: 48 client # [ 0.282653] ... fixed-purpose bitmap: 0000000000000000 worker2 # [ 0.281292] ... generic counters: 6 client # [ 0.282925] ... value mask: 0000ffffffffffff worker2 # [ 0.282054] ... generic bitmap: 000000000000003f worker2 # [ 0.282292] ... fixed-purpose counters: 0 client # [ 0.283836] ... max period: 00007fffffffffff worker2 # [ 0.283038] ... fixed-purpose bitmap: 0000000000000000 client # [ 0.284637] ... global_ctrl mask: 000000000000003f worker2 # [ 0.283292] ... value mask: 0000ffffffffffff client # [ 0.285037] signal: max sigframe size: 3376 client # [ 0.285826] rcu: Hierarchical SRCU implementation. worker2 # [ 0.284237] ... max period: 00007fffffffffff client # [ 0.286530] rcu: Max phase no-delay instances is 400. worker2 # [ 0.285027] ... global_ctrl mask: 000000000000003f worker2 # [ 0.285405] signal: max sigframe size: 3376 worker2 # [ 0.286254] rcu: Hierarchical SRCU implementation. worker2 # [ 0.286916] rcu: Max phase no-delay instances is 400. worker1 # [ 0.268176] Freeing SMP alternatives memory: 44K worker1 # [ 0.268785] pid_max: default: 32768 minimum: 301 worker1 # [ 0.269897] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.291434] smp: Bringing up secondary CPUs ... client # [ 0.291940] smp: Brought up 1 node, 1 CPU worker1 # [ 0.270276] landlock: Up and running. lb # [ 0.265352] Freeing SMP alternatives memory: 44K worker1 # [ 0.271618] Yama: becoming mindful. lb # [ 0.265738] pid_max: default: 32768 minimum: 301 client # [ 0.292587] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker1 # [ 0.272405] LSM support for eBPF active lb # [ 0.266831] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.291836] smp: Bringing up secondary CPUs ... lb # [ 0.267833] landlock: Up and running. worker2 # [ 0.292308] smp: Brought up 1 node, 1 CPU worker1 # [ 0.273298] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.293070] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) lb # [ 0.268423] Yama: becoming mindful. worker2 # [ 0.292943] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) client # [ 0.294188] devtmpfs: initialized lb # [ 0.269353] LSM support for eBPF active worker1 # [ 0.274231] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.294936] x86/mm: Memory block size: 128MB lb # [ 0.269848] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.293444] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved) worker1 # [ 0.276817] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) lb # [ 0.270772] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.294556] devtmpfs: initialized client # [ 0.296685] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.295386] x86/mm: Memory block size: 128MB worker1 # [ 0.277697] Performance Events: Fam17h+ core perfctr, AMD PMU driver. client # [ 0.297811] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.278200] ... version: 2 lb # [ 0.273671] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.299005] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.278921] ... bit width: 48 worker2 # [ 0.297208] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.279197] ... generic counters: 6 lb # [ 0.274222] Performance Events: Fam17h+ core perfctr, AMD PMU driver. client # [ 0.300191] PM: RTC time: 16:12:08, date: 2026-09-23 lb # [ 0.274740] ... version: 2 worker1 # [ 0.279881] ... generic bitmap: 000000000000003f worker2 # [ 0.298198] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.275417] ... bit width: 48 worker1 # [ 0.280246] ... fixed-purpose counters: 0 worker2 # [ 0.299371] pinctrl core: initialized pinctrl subsystem lb # [ 0.275738] ... generic counters: 6 worker1 # [ 0.280990] ... fixed-purpose bitmap: 0000000000000000 lb # [ 0.276444] ... generic bitmap: 000000000000003f worker2 # [ 0.300485] PM: RTC time: 16:12:08, date: 2026-09-23 worker1 # [ 0.281893] ... value mask: 0000ffffffffffff lb # [ 0.276781] ... fixed-purpose counters: 0 client # [ 0.303829] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.282197] ... max period: 00007fffffffffff lb # [ 0.277468] ... fixed-purpose bitmap: 0000000000000000 worker1 # [ 0.283089] ... global_ctrl mask: 000000000000003f lb # [ 0.277738] ... value mask: 0000ffffffffffff client # [ 0.305158] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.284033] signal: max sigframe size: 3376 lb # [ 0.278590] ... max period: 00007fffffffffff worker2 # [ 0.304045] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.284830] rcu: Hierarchical SRCU implementation. client # [ 0.305947] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.279387] ... global_ctrl mask: 000000000000003f worker1 # [ 0.285200] rcu: Max phase no-delay instances is 400. lb # [ 0.279844] signal: max sigframe size: 3376 worker2 # [ 0.304788] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.307197] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.280574] rcu: Hierarchical SRCU implementation. client # [ 0.307936] audit: initializing netlink subsys (disabled) lb # [ 0.281307] rcu: Max phase no-delay instances is 400. worker2 # [ 0.305317] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.309057] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.306571] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.309059] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.307302] audit: initializing netlink subsys (disabled) client # [ 0.309927] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.290504] smp: Bringing up secondary CPUs ... worker2 # [ 0.308526] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.310890] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.291211] smp: Brought up 1 node, 1 CPU worker2 # [ 0.308528] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.291867] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) lb # [ 0.286185] smp: Bringing up secondary CPUs ... client # [ 0.311673] audit: type=2000 audit(1790179929.076:1): state=initialized audit_enabled=0 res=1 lb # [ 0.286751] smp: Brought up 1 node, 1 CPU worker2 # [ 0.309294] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.312928] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.287407] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.310258] thermal_sys: Registered thermal governor 'user_space' client # [ 0.312941] cpuidle: using governor menu worker1 # [ 0.292323] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker1 # [ 0.293398] devtmpfs: initialized worker2 # [ 0.311015] audit: type=2000 audit(1790179929.078:1): state=initialized audit_enabled=0 res=1 client # [ 0.315134] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 lb # [ 0.287932] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved) worker1 # [ 0.294222] x86/mm: Memory block size: 128MB worker2 # [ 0.312294] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.288970] devtmpfs: initialized worker2 # [ 0.312308] cpuidle: using governor menu client # [ 0.316153] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] lb # [ 0.289780] x86/mm: Memory block size: 128MB worker1 # [ 0.295980] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.316929] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.314517] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 client # [ 0.317938] PCI: Using configuration type 1 for base access worker1 # [ 0.297092] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.291525] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.315516] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.298270] pinctrl core: initialized pinctrl subsystem client # [ 0.319077] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.316295] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry lb # [ 0.292611] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.299416] PM: RTC time: 16:12:08, date: 2026-09-23 worker2 # [ 0.317303] PCI: Using configuration type 1 for base access lb # [ 0.293804] pinctrl core: initialized pinctrl subsystem lb # [ 0.294941] PM: RTC time: 16:12:08, date: 2026-09-23 worker2 # [ 0.318469] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker1 # [ 0.302821] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.324044] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.298368] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.303671] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.324927] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.325919] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.304217] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.299226] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.323517] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.299757] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.305467] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.324293] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.306206] audit: initializing netlink subsys (disabled) lb # [ 0.301004] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.301747] audit: initializing netlink subsys (disabled) worker1 # [ 0.307198] audit: type=2000 audit(1790179929.099:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.308364] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.302740] audit: type=2000 audit(1790179929.103:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.308366] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.303832] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.332928] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.309198] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.329293] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.303834] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.310177] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.330292] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.304738] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.310915] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.305634] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.311211] cpuidle: using governor menu lb # [ 0.306434] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.306751] cpuidle: using governor menu worker1 # [ 0.313443] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 lb # [ 0.308949] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.314433] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] lb # [ 0.309964] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.315199] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry client # [ 0.341352] ACPI: Added _OSI(Module Device) worker1 # [ 0.316208] PCI: Using configuration type 1 for base access lb # [ 0.310740] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry client # [ 0.341928] ACPI: Added _OSI(Processor Device) lb # [ 0.311750] PCI: Using configuration type 1 for base access client # [ 0.342627] ACPI: Added _OSI(Processor Aggregator Device) worker1 # [ 0.317332] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. lb # [ 0.312834] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.341632] ACPI: Added _OSI(Module Device) worker2 # [ 0.342294] ACPI: Added _OSI(Processor Device) worker2 # [ 0.343050] ACPI: Added _OSI(Processor Aggregator Device) worker1 # [ 0.324219] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.319760] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.325197] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.320738] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.351469] ACPI: 1 ACPI AML tables successfully acquired and loaded worker2 # [ 0.350886] ACPI: 1 ACPI AML tables successfully acquired and loaded worker1 # [ 0.330198] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.355384] ACPI: Interpreter enabled lb # [ 0.325739] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.331197] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.355941] ACPI: PM: (supports S0 S3 S4 S5) lb # [ 0.326738] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.356670] ACPI: Using IOAPIC for interrupt routing client # [ 0.356986] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug worker2 # [ 0.354543] ACPI: Interpreter enabled worker2 # [ 0.355244] ACPI: PM: (supports S0 S3 S4 S5) client # [ 0.357927] PCI: Using E820 reservations for host bridge windows worker2 # [ 0.355840] ACPI: Using IOAPIC for interrupt routing worker2 # [ 0.356337] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.361064] ACPI: Enabled 2 GPEs in block 00 to 3F worker2 # [ 0.357293] PCI: Using E820 reservations for host bridge windows worker2 # [ 0.360430] ACPI: Enabled 2 GPEs in block 00 to 3F worker1 # [ 0.342421] ACPI: Added _OSI(Module Device) lb # [ 0.337957] ACPI: Added _OSI(Module Device) worker1 # [ 0.343198] ACPI: Added _OSI(Processor Device) lb # [ 0.338662] ACPI: Added _OSI(Processor Device) worker1 # [ 0.343920] ACPI: Added _OSI(Processor Aggregator Device) lb # [ 0.339266] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.369740] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.370748] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.372299] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker2 # [ 0.369144] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker1 # [ 0.349576] ACPI: 1 ACPI AML tables successfully acquired and loaded lb # [ 0.344968] ACPI: 1 ACPI AML tables successfully acquired and loaded client # [ 0.373049] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker2 # [ 0.370098] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.374264] PCI host bridge to bus 0000:00 client # [ 0.374930] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] worker2 # [ 0.371820] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] client # [ 0.375926] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker1 # [ 0.353599] ACPI: Interpreter enabled worker2 # [ 0.372415] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] lb # [ 0.349139] ACPI: Interpreter enabled worker1 # [ 0.354213] ACPI: PM: (supports S0 S3 S4 S5) worker2 # [ 0.373715] PCI host bridge to bus 0000:00 lb # [ 0.349752] ACPI: PM: (supports S0 S3 S4 S5) client # [ 0.376926] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker1 # [ 0.354891] ACPI: Using IOAPIC for interrupt routing lb # [ 0.350421] ACPI: Using IOAPIC for interrupt routing worker2 # [ 0.374297] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] client # [ 0.377927] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker2 # [ 0.375292] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.378926] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.376292] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.379931] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.357249] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug lb # [ 0.352804] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.380955] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.377292] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.358197] PCI: Using E820 reservations for host bridge windows lb # [ 0.353738] PCI: Using E820 reservations for host bridge windows worker2 # [ 0.378292] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] client # [ 0.381884] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.379293] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.361265] ACPI: Enabled 2 GPEs in block 00 to 3F client # [ 0.383374] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.380292] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.356795] ACPI: Enabled 2 GPEs in block 00 to 3F worker2 # [ 0.381322] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.382729] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker1 # [ 0.368648] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) lb # [ 0.363975] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.386070] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker1 # [ 0.369203] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] lb # [ 0.364744] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.386938] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] client # [ 0.387881] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.365814] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.370277] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] lb # [ 0.366859] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker1 # [ 0.371320] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] client # [ 0.388847] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker2 # [ 0.385448] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.368096] PCI host bridge to bus 0000:00 worker1 # [ 0.372534] PCI host bridge to bus 0000:00 worker2 # [ 0.386304] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.368742] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] worker1 # [ 0.373201] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] client # [ 0.390666] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.387381] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.369738] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker1 # [ 0.374197] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] lb # [ 0.370738] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker1 # [ 0.375197] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker2 # [ 0.388655] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] lb # [ 0.371738] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] client # [ 0.392964] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] worker1 # [ 0.376197] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] client # [ 0.393832] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.372738] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.391014] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.377197] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] client # [ 0.394648] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] lb # [ 0.373738] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.378197] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] client # [ 0.394932] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] lb # [ 0.374739] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.379197] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.394331] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.375724] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.380181] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint client # [ 0.396481] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.395199] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.377107] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.396066] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.381624] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.397131] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] client # [ 0.397944] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] client # [ 0.398791] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.399140] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.399719] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.401343] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.402176] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] lb # [ 0.379777] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker2 # [ 0.403146] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker1 # [ 0.384293] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.380751] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] worker2 # [ 0.404039] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] worker1 # [ 0.385288] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.381685] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] client # [ 0.403976] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.386266] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker2 # [ 0.404891] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.405549] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] lb # [ 0.382642] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker1 # [ 0.387504] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] client # [ 0.405948] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] lb # [ 0.384627] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.407504] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.388926] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.407300] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker2 # [ 0.409012] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] lb # [ 0.387745] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.408958] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.391237] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.388554] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker2 # [ 0.409314] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] client # [ 0.409851] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker1 # [ 0.392098] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.389431] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] client # [ 0.410660] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.392942] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker2 # [ 0.410857] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.389744] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker1 # [ 0.393203] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] client # [ 0.410932] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.391207] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.394748] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.413271] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] client # [ 0.412480] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.413946] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker2 # [ 0.414321] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.396204] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] client # [ 0.413960] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] lb # [ 0.393746] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker1 # [ 0.397165] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] client # [ 0.414948] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker2 # [ 0.415298] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.394553] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker1 # [ 0.397905] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.395427] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.416502] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.416867] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.398763] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.396288] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.417969] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.418337] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker2 # [ 0.419314] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] client # [ 0.418948] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] lb # [ 0.398746] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.401246] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] lb # [ 0.399570] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker1 # [ 0.402204] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker2 # [ 0.420842] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.420504] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.403143] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] lb # [ 0.400445] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] client # [ 0.421969] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.422304] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker1 # [ 0.404690] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.422948] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] lb # [ 0.401282] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.423315] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker1 # [ 0.406233] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] client # [ 0.424464] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.424867] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.403578] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.407074] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] lb # [ 0.404341] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker1 # [ 0.407898] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] client # [ 0.425971] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.404760] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker2 # [ 0.426305] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker1 # [ 0.408203] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.405744] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] client # [ 0.426941] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.427265] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker1 # [ 0.409942] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.407233] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.428461] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.428694] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.412008] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker1 # [ 0.412976] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] client # [ 0.429977] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.430305] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.409750] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] client # [ 0.430859] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker1 # [ 0.414709] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.410690] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker2 # [ 0.431314] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] client # [ 0.431659] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker2 # [ 0.432860] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.412094] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.416240] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] client # [ 0.432498] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.417151] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] lb # [ 0.413750] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.435121] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] lb # [ 0.414743] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker1 # [ 0.418797] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.435963] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] client # [ 0.433934] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] client # [ 0.435567] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker2 # [ 0.436313] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] lb # [ 0.416308] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.420062] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] client # [ 0.435949] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker1 # [ 0.420910] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker2 # [ 0.437827] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.437518] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.421743] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.418750] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] lb # [ 0.419678] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] client # [ 0.438664] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.439988] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.423182] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] client # [ 0.439123] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.440299] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.421141] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.423950] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.441237] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] client # [ 0.440600] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] lb # [ 0.422750] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker1 # [ 0.425536] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.442775] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.423701] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] client # [ 0.441132] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.427249] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.444045] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] lb # [ 0.425113] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.442647] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker1 # [ 0.428123] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker2 # [ 0.444500] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.428952] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.429749] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.427745] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.446709] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] lb # [ 0.428590] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker2 # [ 0.447486] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.429493] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.431233] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker2 # [ 0.449032] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] lb # [ 0.430278] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.432203] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.433158] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] lb # [ 0.431747] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.434668] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.432612] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.435764] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] client # [ 0.443151] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.433486] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker1 # [ 0.436424] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint client # [ 0.445568] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] lb # [ 0.434301] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.437755] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] client # [ 0.446212] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint lb # [ 0.435555] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker1 # [ 0.438398] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint client # [ 0.447217] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO lb # [ 0.436555] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.439711] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] client # [ 0.448212] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint lb # [ 0.438446] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker2 # [ 0.449513] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.438930] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint client # [ 0.449954] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker2 # [ 0.452051] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.450847] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] lb # [ 0.440453] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker2 # [ 0.452570] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.452070] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.453580] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.453570] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.454545] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.456320] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker2 # [ 0.457253] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.457057] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.458033] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker1 # [ 0.440460] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.458348] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint client # [ 0.459023] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker1 # [ 0.442758] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.460034] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker2 # [ 0.460007] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.461024] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker1 # [ 0.443448] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.462021] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.444517] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO lb # [ 0.441703] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.463023] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.464043] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.445446] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.463400] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.464991] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.464430] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker1 # [ 0.447001] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] lb # [ 0.444385] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.465941] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker2 # [ 0.465399] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker1 # [ 0.447813] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.466941] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 lb # [ 0.445021] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker2 # [ 0.466398] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 client # [ 0.467942] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.448506] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.467389] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 lb # [ 0.446025] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.468914] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker2 # [ 0.468389] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.449810] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.469663] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.446983] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.469393] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.469941] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 client # [ 0.470910] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.470401] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker2 # [ 0.471337] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 lb # [ 0.448767] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker1 # [ 0.453268] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.472846] iommu: Default domain type: Translated worker2 # [ 0.472308] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 lb # [ 0.449579] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.473531] iommu: DMA domain TLB invalidation policy: lazy mode worker1 # [ 0.454330] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker2 # [ 0.473222] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 client # [ 0.474188] ACPI: bus type USB registered worker2 # [ 0.474013] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.455298] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 lb # [ 0.451131] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint client # [ 0.474944] usbcore: registered new interface driver usbfs worker2 # [ 0.474304] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker1 # [ 0.456294] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 client # [ 0.475842] usbcore: registered new interface driver hub lb # [ 0.452556] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.475273] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 worker1 # [ 0.457296] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 client # [ 0.476556] usbcore: registered new device driver usb worker2 # [ 0.476042] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker1 # [ 0.458296] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 client # [ 0.477766] NetLabel: Initializing worker2 # [ 0.476304] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker1 # [ 0.459297] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.478367] NetLabel: domain hash size = 128 worker1 # [ 0.460294] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 client # [ 0.478926] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker2 # [ 0.478340] iommu: Default domain type: Translated lb # [ 0.456297] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker1 # [ 0.461232] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 client # [ 0.479842] NetLabel: unlabeled traffic allowed by default worker2 # [ 0.479141] iommu: DMA domain TLB invalidation policy: lazy mode lb # [ 0.456845] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 client # [ 0.480569] PCI: Using ACPI for IRQ routing worker1 # [ 0.462185] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker2 # [ 0.480280] ACPI: bus type USB registered lb # [ 0.457836] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker1 # [ 0.463214] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker2 # [ 0.480894] usbcore: registered new interface driver usbfs lb # [ 0.458833] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker2 # [ 0.481307] usbcore: registered new interface driver hub worker1 # [ 0.464213] ACPI: PCI: Interrupt link GSID configured for IRQ 19 lb # [ 0.459833] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker2 # [ 0.482206] usbcore: registered new device driver usb worker1 # [ 0.465192] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 lb # [ 0.460837] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.466157] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 worker2 # [ 0.483865] NetLabel: Initializing lb # [ 0.461837] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker1 # [ 0.466997] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker2 # [ 0.484293] NetLabel: domain hash size = 128 lb # [ 0.462836] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.467951] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.485055] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.463774] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.485324] NetLabel: unlabeled traffic allowed by default lb # [ 0.464741] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker2 # [ 0.486251] PCI: Using ACPI for IRQ routing worker1 # [ 0.469176] iommu: Default domain type: Translated lb # [ 0.465681] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker1 # [ 0.469795] iommu: DMA domain TLB invalidation policy: lazy mode lb # [ 0.466474] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.470462] ACPI: bus type USB registered lb # [ 0.466749] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker1 # [ 0.471255] usbcore: registered new interface driver usbfs lb # [ 0.467744] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 worker1 # [ 0.472188] usbcore: registered new interface driver hub worker1 # [ 0.472886] usbcore: registered new device driver usb lb # [ 0.468715] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.469680] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker1 # [ 0.473997] NetLabel: Initializing worker1 # [ 0.474203] NetLabel: domain hash size = 128 lb # [ 0.471507] iommu: Default domain type: Translated worker1 # [ 0.474979] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.471745] iommu: DMA domain TLB invalidation policy: lazy mode worker1 # [ 0.475959] NetLabel: unlabeled traffic allowed by default lb # [ 0.472980] ACPI: bus type USB registered worker1 # [ 0.476209] PCI: Using ACPI for IRQ routing lb # [ 0.473717] usbcore: registered new interface driver usbfs lb # [ 0.474460] usbcore: registered new interface driver hub lb # [ 0.474747] usbcore: registered new device driver usb lb # [ 0.476464] NetLabel: Initializing lb # [ 0.476744] NetLabel: domain hash size = 128 lb # [ 0.477615] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.478576] NetLabel: unlabeled traffic allowed by default lb # [ 0.479462] PCI: Using ACPI for IRQ routing client # [ 0.524745] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.524922] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.524922] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.524931] vgaarb: loaded client # [ 0.525674] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.525928] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker2 # [ 0.530927] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker2 # [ 0.531288] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.532008] clocksource: Switched to clocksource kvm-clock worker2 # [ 0.531288] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker2 # [ 0.531297] vgaarb: loaded worker2 # [ 0.532068] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.535532] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.532294] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.536283] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.537707] pnp: PnP ACPI init client # [ 0.538518] ACPI: IRQ 4 override to edge(!), high(!) worker1 # [ 0.520900] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker1 # [ 0.521193] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.539507] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker1 # [ 0.521193] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.541170] pnp: PnP ACPI: found 5 devices worker2 # [ 0.538380] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.521201] vgaarb: loaded worker1 # [ 0.521962] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker1 # [ 0.522197] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker2 # [ 0.541945] VFS: Disk quotas dquot_6.6.0 lb # [ 0.522091] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker2 # [ 0.542729] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.522734] pci 0000:00:01.0: vgaarb: bridge control possible worker2 # [ 0.544301] pnp: PnP ACPI init lb # [ 0.522734] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker2 # [ 0.545106] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.522743] vgaarb: loaded worker1 # [ 0.527287] clocksource: Switched to clocksource kvm-clock client # [ 0.548851] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns lb # [ 0.523455] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker2 # [ 0.546093] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved client # [ 0.550248] clocksource: Switched to clocksource acpi_pm lb # [ 0.523738] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker2 # [ 0.547582] pnp: PnP ACPI: found 5 devices client # [ 0.551209] NET: Registered PF_INET protocol family worker1 # [ 0.530726] VFS: Disk quotas dquot_6.6.0 client # [ 0.552430] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.531428] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.532750] pnp: PnP ACPI init worker1 # [ 0.533564] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.527828] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.534529] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker1 # [ 0.536083] pnp: PnP ACPI: found 5 devices lb # [ 0.531303] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.555138] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns lb # [ 0.531993] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.556674] clocksource: Switched to clocksource acpi_pm lb # [ 0.533275] pnp: PnP ACPI init worker2 # [ 0.557649] NET: Registered PF_INET protocol family lb # [ 0.534038] ACPI: IRQ 4 override to edge(!), high(!) worker2 # [ 0.558902] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.535004] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.536547] pnp: PnP ACPI: found 5 devices worker1 # [ 0.543681] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.545191] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.546217] NET: Registered PF_INET protocol family worker1 # [ 0.547482] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.568897] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.570403] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.571767] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.544183] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns lb # [ 0.545784] clocksource: Switched to clocksource acpi_pm client # [ 0.573154] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.546729] NET: Registered PF_INET protocol family client # [ 0.574427] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.547865] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.575576] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.576877] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.578059] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.575354] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.579314] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.580310] NET: Registered PF_XDP protocol family worker2 # [ 0.577003] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.581192] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.578389] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.582244] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.579783] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.583305] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.581069] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.584414] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker2 # [ 0.582216] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.585546] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.563990] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.583469] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.586644] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.565526] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.584553] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.588357] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.566810] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.585808] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.586805] NET: Registered PF_XDP protocol family worker1 # [ 0.568195] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.590661] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.587630] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker1 # [ 0.569465] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.588634] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] lb # [ 0.563953] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.570539] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.592842] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.589652] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.565425] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.571777] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.590743] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker1 # [ 0.572856] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.566672] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.594816] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.591862] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.574050] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.567957] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.596447] PCI: CLS 0 bytes, default 64 worker1 # [ 0.575027] NET: Registered PF_XDP protocol family worker2 # [ 0.592969] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] lb # [ 0.569175] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.575861] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.594624] ACPI: \_SB_.GSIA: Enabled at IRQ 16 client # [ 0.597409] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.570248] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.576873] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] client # [ 0.599128] Trying to unpack rootfs image as initramfs... lb # [ 0.571421] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.577854] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.572460] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.597296] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.578937] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] lb # [ 0.573650] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.580011] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] lb # [ 0.574609] NET: Registered PF_XDP protocol family worker2 # [ 0.599198] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.581145] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] lb # [ 0.575429] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker1 # [ 0.582841] ACPI: \_SB_.GSIA: Enabled at IRQ 16 lb # [ 0.576431] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.601087] ACPI: \_SB_.GSID: Enabled at IRQ 19 lb # [ 0.577416] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.602794] PCI: CLS 0 bytes, default 64 worker1 # [ 0.584977] ACPI: \_SB_.GSIB: Enabled at IRQ 17 lb # [ 0.578532] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker2 # [ 0.603780] Trying to unpack rootfs image as initramfs... lb # [ 0.579612] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.587105] ACPI: \_SB_.GSIC: Enabled at IRQ 18 lb # [ 0.580684] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] lb # [ 0.582362] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.589038] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.606119] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker1 # [ 0.590728] PCI: CLS 0 bytes, default 64 lb # [ 0.584559] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.591629] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.586684] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.593391] Trying to unpack rootfs image as initramfs... lb # [ 0.588561] ACPI: \_SB_.GSID: Enabled at IRQ 19 lb # [ 0.590585] PCI: CLS 0 bytes, default 64 lb # [ 0.591467] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.593279] Trying to unpack rootfs image as initramfs... client # [ 0.643544] Initialise system trusted keyrings client # [ 0.644532] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker2 # [ 0.649087] Initialise system trusted keyrings worker2 # [ 0.650110] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker1 # [ 0.635864] Initialise system trusted keyrings worker1 # [ 0.639519] workingset: timestamp_bits=40 max_order=19 bucket_order=0 lb # [ 0.636547] Initialise system trusted keyrings lb # [ 0.637490] workingset: timestamp_bits=40 max_order=19 bucket_order=0 client # [ 0.668748] Key type asymmetric registered client # [ 0.669456] Asymmetric key parser 'x509' registered client # [ 0.670368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) client # [ 0.673731] io scheduler mq-deadline registered client # [ 0.674545] io scheduler kyber registered worker2 # [ 0.673813] Key type asymmetric registered worker2 # [ 0.674569] Asymmetric key parser 'x509' registered client # [ 0.678821] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.675543] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) client # [ 0.680068] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.660636] Key type asymmetric registered worker1 # [ 0.661367] Asymmetric key parser 'x509' registered worker2 # [ 0.680819] io scheduler mq-deadline registered worker1 # [ 0.662262] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker2 # [ 0.681611] io scheduler kyber registered client # [ 0.685405] Linux agpgart interface v0.103 client # [ 0.686204] ACPI: bus type drm_connector registered worker2 # [ 0.684219] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.661255] Key type asymmetric registered worker1 # [ 0.667563] io scheduler mq-deadline registered worker2 # [ 0.685513] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A client # [ 0.689052] usbcore: registered new interface driver usbserial_generic lb # [ 0.661927] Asymmetric key parser 'x509' registered worker1 # [ 0.668280] io scheduler kyber registered client # [ 0.690108] usbserial: USB Serial support registered for generic lb # [ 0.662737] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker1 # [ 0.671050] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.666297] io scheduler mq-deadline registered worker2 # [ 0.691099] Linux agpgart interface v0.103 worker1 # [ 0.672379] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A lb # [ 0.667044] io scheduler kyber registered worker2 # [ 0.691878] ACPI: bus type drm_connector registered client # [ 0.694630] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. client # [ 0.694630] Please enable it if your BIOS has the CPPC option. client # [ 0.696906] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled lb # [ 0.669714] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.695120] usbcore: registered new interface driver usbserial_generic lb # [ 0.670928] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.677864] Linux agpgart interface v0.103 worker2 # [ 0.696213] usbserial: USB Serial support registered for generic client # [ 0.699762] drop_monitor: Initializing network drop monitor service worker1 # [ 0.678615] ACPI: bus type drm_connector registered client # [ 0.700875] NET: Registered PF_INET6 protocol family worker2 # [ 0.698697] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.698697] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.681888] usbcore: registered new interface driver usbserial_generic client # [ 0.704037] Segment Routing with IPv6 lb # [ 0.676434] Linux agpgart interface v0.103 worker2 # [ 0.700990] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.682988] usbserial: USB Serial support registered for generic lb # [ 0.677179] ACPI: bus type drm_connector registered client # [ 0.706653] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.703805] drop_monitor: Initializing network drop monitor service client # [ 0.707767] IPI shorthand broadcast: enabled worker2 # [ 0.705044] NET: Registered PF_INET6 protocol family lb # [ 0.680386] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.686485] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. lb # [ 0.681420] usbserial: USB Serial support registered for generic worker1 # [ 0.686485] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.688719] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled lb # [ 0.684176] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.710132] Segment Routing with IPv6 lb # [ 0.684176] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.691590] drop_monitor: Initializing network drop monitor service worker2 # [ 0.710834] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.692762] NET: Registered PF_INET6 protocol family lb # [ 0.686361] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker2 # [ 0.712956] IPI shorthand broadcast: enabled client # [ 0.715884] sched_clock: Marking stable (580020248, 135585922)->(790455437, -74849267) lb # [ 0.689320] drop_monitor: Initializing network drop monitor service worker1 # [ 0.696047] Segment Routing with IPv6 lb # [ 0.690466] NET: Registered PF_INET6 protocol family client # [ 0.719868] registered taskstats version 1 worker1 # [ 0.698509] In-situ OAM (IOAM) with IPv6 client # [ 0.720833] Loading compiled-in X.509 certificates worker1 # [ 0.699534] IPI shorthand broadcast: enabled lb # [ 0.693595] Segment Routing with IPv6 lb # [ 0.696215] In-situ OAM (IOAM) with IPv6 lb # [ 0.697275] IPI shorthand broadcast: enabled worker2 # [ 0.721316] sched_clock: Marking stable (585014818, 135656381)->(796467777, -75796578) worker1 # [ 0.707724] sched_clock: Marking stable (571014967, 136442578)->(777735591, -70278046) worker2 # [ 0.726896] registered taskstats version 1 worker2 # [ 0.727892] Loading compiled-in X.509 certificates lb # [ 0.704527] sched_clock: Marking stable (570014122, 134139379)->(775138804, -70985303) worker1 # [ 0.711696] registered taskstats version 1 worker1 # [ 0.712668] Loading compiled-in X.509 certificates lb # [ 0.710340] registered taskstats version 1 client # [ 0.738625] Demotion targets for Node 0: null lb # [ 0.711328] Loading compiled-in X.509 certificates client # [ 0.741652] Key type .fscrypt registered client # [ 0.742372] Key type fscrypt-provisioning registered client # [ 0.743369] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.744302] Demotion targets for Node 0: null client # [ 0.747622] ima: Allocated hash algorithm: sha1 client # [ 0.748416] ima: No architecture policies found client # [ 0.749418] PM: Magic number: 10:612:237 worker2 # [ 0.747720] Key type .fscrypt registered worker2 # [ 0.748419] Key type fscrypt-provisioning registered worker2 # [ 0.749429] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.731472] Demotion targets for Node 0: null client # [ 0.753119] RAS: Correctable Errors collector initialized. worker1 # [ 0.732308] Key type .fscrypt registered worker2 # [ 0.751683] ima: Allocated hash algorithm: sha1 worker2 # [ 0.752526] ima: No architecture policies found worker1 # [ 0.735461] Key type fscrypt-provisioning registered lb # [ 0.730167] Demotion targets for Node 0: null worker1 # [ 0.736388] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.731029] Key type .fscrypt registered worker1 # [ 0.737351] ima: Allocated hash algorithm: sha1 worker1 # [ 0.738190] ima: No architecture policies found worker2 # [ 0.756693] PM: Magic number: 10:612:237 worker2 # [ 0.758349] RAS: Correctable Errors collector initialized. worker1 # [ 0.740468] PM: Magic number: 10:612:237 lb # [ 0.734158] Key type fscrypt-provisioning registered client # [ 0.762777] clk: Disabling unused clocks lb # [ 0.735040] ima: No TPM chip found, activating TPM-bypass! client # [ 0.763453] PM: genpd: Disabling unused power domains worker1 # [ 0.741974] RAS: Correctable Errors collector initialized. lb # [ 0.735934] ima: Allocated hash algorithm: sha1 lb # [ 0.736756] ima: No architecture policies found lb # [ 0.739166] PM: Magic number: 10:612:237 lb # [ 0.740720] RAS: Correctable Errors collector initialized. worker2 # [ 0.766368] clk: Disabling unused clocks worker1 # [ 0.751160] clk: Disabling unused clocks worker2 # [ 0.769720] PM: genpd: Disabling unused power domains worker1 # [ 0.755484] PM: genpd: Disabling unused power domains lb # [ 0.750822] clk: Disabling unused clocks lb # [ 0.753187] PM: genpd: Disabling unused power domains client # [ 0.924497] Freeing initrd memory: 29056K client # [ 0.927774] Freeing unused decrypted memory: 2028K client # [ 0.930423] Freeing unused kernel image (initmem) memory: 3652K client # [ 0.931540] Write protecting the kernel read-only data: 32768k client # [ 0.933441] Freeing unused kernel image (text/rodata gap) memory: 1184K worker2 # [ 0.931310] Freeing initrd memory: 29060K worker1 # [ 0.913516] Freeing initrd memory: 29056K client # [ 0.934989] Freeing unused kernel image (rodata/data gap) memory: 720K worker2 # [ 0.934497] Freeing unused decrypted memory: 2028K worker1 # [ 0.916599] Freeing unused decrypted memory: 2028K worker2 # [ 0.937144] Freeing unused kernel image (initmem) memory: 3652K worker1 # [ 0.919122] Freeing unused kernel image (initmem) memory: 3652K lb # [ 0.913603] Freeing initrd memory: 29064K worker2 # [ 0.938216] Write protecting the kernel read-only data: 32768k worker1 # [ 0.920225] Write protecting the kernel read-only data: 32768k worker2 # [ 0.940026] Freeing unused kernel image (text/rodata gap) memory: 1184K worker1 # [ 0.922082] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.916655] Freeing unused decrypted memory: 2028K worker2 # [ 0.941547] Freeing unused kernel image (rodata/data gap) memory: 720K worker1 # [ 0.923627] Freeing unused kernel image (rodata/data gap) memory: 720K lb # [ 0.919177] Freeing unused kernel image (initmem) memory: 3652K lb # [ 0.920262] Write protecting the kernel read-only data: 32768k lb # [ 0.922065] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.923566] Freeing unused kernel image (rodata/data gap) memory: 720K client # [ 0.986911] x86/mm: Checked W+X mappings: passed, no W+X pages found. client # [ 0.988052] Run /init as init process worker2 # [ 0.992458] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker1 # [ 0.974527] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker2 # [ 0.993568] Run /init as init process worker1 # [ 0.975550] Run /init as init process client # [ 0.999138] systemd[1]: Inserted module 'autofs4' lb # [ 0.975533] x86/mm: Checked W+X mappings: passed, no W+X pages found. lb # [ 0.976612] Run /init as init process worker1 # [ 0.985983] systemd[1]: Inserted module 'autofs4' worker2 # [ 1.004453] systemd[1]: Inserted module 'autofs4' lb # [ 0.986992] systemd[1]: Inserted module 'autofs4' client # [ 1.018999] fuse: init (API version 7.45) client # [ 1.026376] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.005751] fuse: init (API version 7.45) worker2 # [ 1.024369] fuse: init (API version 7.45) client # [ 1.028823] ACPI: \_SB_.GSIH: Enabled at IRQ 23 client # [ 1.031881] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.033863] ACPI: \_SB_.GSIF: Enabled at IRQ 21 lb # [ 1.006605] fuse: init (API version 7.45) worker1 # [ 1.013139] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker2 # [ 1.031767] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.015485] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker2 # [ 1.034402] ACPI: \_SB_.GSIH: Enabled at IRQ 23 client # [ 1.039211] virtiofs virtio5: discovered new tag: nix-store worker1 # [ 1.018356] ACPI: \_SB_.GSIE: Enabled at IRQ 20 worker2 # [ 1.037762] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.040875] virtiofs virtio5: virtio_fs_setup_dax: No cache capability lb # [ 1.014016] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.020368] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker2 # [ 1.039928] ACPI: \_SB_.GSIF: Enabled at IRQ 21 lb # [ 1.016351] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker1 # [ 1.025380] virtiofs virtio5: discovered new tag: nix-store lb # [ 1.019408] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.048014] virtiofs virtio6: discovered new tag: shared worker1 # [ 1.026932] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker2 # [ 1.045551] virtiofs virtio5: discovered new tag: nix-store lb # [ 1.021291] ACPI: \_SB_.GSIF: Enabled at IRQ 21 client # [ 1.049715] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.047073] virtiofs virtio5: virtio_fs_setup_dax: No cache capability client # [ 1.052840] virtiofs virtio7: discovered new tag: xchg lb # [ 1.025848] virtiofs virtio5: discovered new tag: nix-store client # [ 1.054276] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 1.033681] virtiofs virtio6: discovered new tag: shared lb # [ 1.027892] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker1 # [ 1.035196] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.054569] virtiofs virtio6: discovered new tag: shared worker2 # [ 1.056077] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 1.038423] virtiofs virtio7: discovered new tag: xchg worker1 # [ 1.039922] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.034466] virtiofs virtio6: discovered new tag: shared worker2 # [ 1.060507] virtiofs virtio7: discovered new tag: xchg lb # [ 1.035969] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.062403] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.039786] virtiofs virtio7: discovered new tag: xchg lb # [ 1.041319] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 1.073402] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 1.061619] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.084476] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.063864] systemd[1]: Successfully made /usr/ read-only. client # [ 1.408889] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.414021] systemd[1]: Detected virtualization kvm. client # [ 1.414909] systemd[1]: Detected architecture x86-64. client # [ 1.415790] systemd[1]: Running in initrd. client # [ 1.416855] systemd[1]: Initializing machine ID from random generator. client # [ 1.418071] systemd[1]: Hostname set to . worker1 # [ 1.397232] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.402412] systemd[1]: Detected virtualization kvm. worker1 # [ 1.403307] systemd[1]: Detected architecture x86-64. worker1 # [ 1.404194] systemd[1]: Running in initrd. worker2 # [ 1.419960] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.405351] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.424703] systemd[1]: Detected virtualization kvm. worker1 # [ 1.406695] systemd[1]: Hostname set to . worker2 # [ 1.425515] systemd[1]: Detected architecture x86-64. worker2 # [ 1.426343] systemd[1]: Running in initrd. lb # [ 1.398902] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.427354] systemd[1]: Initializing machine ID from random generator. lb # [ 1.403778] systemd[1]: Detected virtualization kvm. worker2 # [ 1.428544] systemd[1]: Hostname set to . lb # [ 1.404618] systemd[1]: Detected architecture x86-64. lb # [ 1.405490] systemd[1]: Running in initrd. lb # [ 1.406547] systemd[1]: Initializing machine ID from random generator. lb # [ 1.407736] systemd[1]: Hostname set to . client # [ 1.606958] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.596118] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.607380] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.635924] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.644436] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.648883] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.650096] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.651560] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.652744] systemd[1]: Reached target Path Units. client # [ 1.653635] systemd[1]: Reached target Slice Units. client # [ 1.654510] systemd[1]: Reached target Swaps. client # [ 1.655335] systemd[1]: Reached target Timer Units. client # [ 1.656319] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.657555] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.635973] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.658715] systemd[1]: Listening on Journal Sockets. client # [ 1.659667] systemd[1]: Listening on udev Control Socket. client # [ 1.660641] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.639857] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.661529] systemd[1]: Reached target Socket Units. worker1 # [ 1.641072] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.663334] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.642559] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.643688] systemd[1]: Reached target Path Units. worker1 # [ 1.644604] systemd[1]: Reached target Slice Units. worker1 # [ 1.645489] systemd[1]: Reached target Swaps. worker1 # [ 1.646285] systemd[1]: Reached target Timer Units. client # [ 1.667674] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 1.647275] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.648555] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.649729] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.650732] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.651807] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.652809] systemd[1]: Reached target Socket Units. worker1 # [ 1.654802] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.659526] systemd[1]: Starting Load Kernel Module configfs... lb # [ 1.653408] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.657399] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.658569] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.676268] systemd[1]: Starting Journal Service... lb # [ 1.659986] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.661057] systemd[1]: Reached target Path Units. lb # [ 1.661938] systemd[1]: Reached target Slice Units. lb # [ 1.662832] systemd[1]: Reached target Swaps. lb # [ 1.663650] systemd[1]: Reached target Timer Units. worker2 # [ 1.688762] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.664625] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.665861] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.667055] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.691962] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.668054] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.693107] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.669067] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.670037] systemd[1]: Reached target Socket Units. worker2 # [ 1.694494] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.695570] systemd[1]: Reached target Path Units. worker2 # [ 1.696440] systemd[1]: Reached target Slice Units. lb # [ 1.671768] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.700226] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.697304] systemd[1]: Reached target Swaps. worker2 # [ 1.698094] systemd[1]: Reached target Timer Units. worker1 # [ 1.669004] systemd[1]: Starting Journal Service... worker2 # [ 1.699051] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.700243] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.676275] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 1.701356] systemd[1]: Listening on Journal Sockets. client # [ 1.704687] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.702309] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.703296] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.704237] systemd[1]: Reached target Socket Units. worker2 # [ 1.706019] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.711678] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.709744] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 1.695170] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.696285] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.684222] systemd[1]: Starting Journal Service... client # [ 1.724380] systemd-journald[66]: Collecting audit messages is disabled. client # [ 1.726105] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.732124] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 1.718876] systemd[1]: Starting Journal Service... worker1 # [ 1.712546] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.709359] systemd[1]: Starting Load Kernel Modules... client # [ 1.740011] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.713249] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.720888] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.719911] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.747038] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker2 # [ 1.744862] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.728828] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.748775] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.731974] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 1.757668] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.755854] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.739891] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.733620] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.734929] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.744905] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config lb # [ 1.739733] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 1.767380] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.770061] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.746557] systemd[1]: Finished Load Kernel Module configfs. worker2 # [ 1.772729] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.775801] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.757684] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.752648] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker2 # [ 1.778235] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 1.761915] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.786053] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.763221] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.790132] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.774667] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.799175] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.772984] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.802746] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.807824] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.782298] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.808502] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.819700] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.819916] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.827882] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.810105] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.804716] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.818692] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.814348] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.843281] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.712422] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.849206] systemd[1]: Started Journal Service. worker1 # [ 1.829035] systemd[1]: Finished Load Kernel Modules. client # lb # [ 1.825675] systemd[1]: Finished Load Kernel Modules. client # [ 1.716316] systemd-modules-load[67]: Inserted module 'virtio_balloon' client # [ 1.719364] systemd-modules-load[67]: Inserted module 'virtio_gpu' worker2 # [ 1.854040] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.724088] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.701706] systemd-modules-load[67]: Inserted module 'dm_mod' lb # [ 1.834371] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.704962] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.706141] systemd-modules-load[67]: Inserted module 'virtio_gpu' client # [ 1.728862] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.843710] systemd[1]: Starting Apply Kernel Variables... client # [ 1.731099] systemd[1]: Reached target Local File Systems. worker2 # [ 1.864225] systemd[1]: Finished Load Kernel Modules. client # [ 1.733317] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.850791] systemd[1]: Started Journal Service. client # [ 1.738266] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.875878] systemd[1]: Starting Apply Kernel Variables... lb # [ 1.852554] systemd[1]: Started Journal Service. client # [ 1.745669] systemd[1]: Finished Apply Kernel Variables. lb # [ 1.717824] systemd-modules-load[67]: Inserted module 'dm_mod' lb # [ 1.723248] systemd-modules-load[67]: Inserted module 'virtio_balloon' lb # [ 1.726273] systemd-modules-load[67]: Inserted module 'virtio_gpu' lb # [ 1.727903] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.729164] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.735118] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.736246] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.733115] systemd[1]: Reached target Local File Systems. worker1 # [ 1.737227] systemd[1]: Reached target Local File Systems. lb # [ 1.733968] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.894003] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.759797] systemd-modules-load[67]: Inserted module 'dm_mod' client # [ 1.766321] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.763144] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.744136] systemd[1]: Starting Create System Files and Directories... lb # [ 1.741890] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.765286] systemd-modules-load[67]: Inserted module 'virtio_gpu' worker2 # [ 1.902023] systemd[1]: Started Journal Service. lb # [ 1.745163] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.749652] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.753192] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.779116] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.780168] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.781161] systemd[1]: Reached target Local File Systems. worker2 # [ 1.783872] systemd[1]: Starting Create System Files and Directories... client # [ 1.790917] systemd-udevd[80]: Using default interface naming scheme 'v261'. worker2 # [ 1.788789] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.771562] systemd[1]: Finished Create System Files and Directories. worker1 # [ 1.784547] systemd[1]: Finished Create System Files and Directories. client # [ 1.814793] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.815921] systemd[1]: Finished Create System Files and Directories. lb # [ 1.794307] systemd-udevd[75]: Using default interface naming scheme 'v261'. worker1 # [ 1.804411] systemd-udevd[80]: Using default interface naming scheme 'v261'. worker2 # [ 1.838414] systemd-udevd[81]: Using default interface naming scheme 'v261'. lb # [ 1.816642] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.829200] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.862073] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 1.868084] systemd[1]: Finished Coldplug All udev Devices. client # [ 1.868938] systemd[1]: Reached target System Initialization. client # [ 1.869791] systemd[1]: Reached target Basic System. lb # [ 1.868925] systemd[1]: Finished Coldplug All udev Devices. lb # [ 1.870064] systemd[1]: Reached target System Initialization. lb # [ 1.871185] systemd[1]: Reached target Basic System. worker1 # [ 1.880154] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 1.882779] systemd[1]: Reached target System Initialization. worker1 # [ 1.883644] systemd[1]: Reached target Basic System. worker2 # [ 1.912892] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 1.913823] systemd[1]: Reached target System Initialization. worker2 # [ 1.914613] systemd[1]: Reached target Basic System. client # [ 2.187007] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.197634] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.200555] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.207048] serio: i8042 KBD port at 0x60,0x64 irq 1 client # [ 2.207738] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.183816] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.192954] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.221001] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.221854] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 client # [ 2.223527] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.229624] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 lb # [ 2.201687] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.230488] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.232151] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.234488] usb usb1: Product: EHCI Host Controller client # [ 2.235344] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.214962] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.210313] serio: i8042 KBD port at 0x60,0x64 irq 1 client # [ 2.238007] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.211013] serio: i8042 AUX port at 0x60,0x64 irq 12 client # [ 2.239941] hub 1-0:1.0: USB hub found worker1 # [ 2.219092] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.241469] hub 1-0:1.0: 6 ports detected worker2 # [ 2.238567] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 worker2 # [ 2.240574] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.216722] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.244994] uhci_hcd 0000:00:1d.0: UHCI Host Controller lb # [ 2.217634] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 client # [ 2.245737] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 lb # [ 2.219329] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 worker1 # [ 2.225696] serio: i8042 KBD port at 0x60,0x64 irq 1 worker1 # [ 2.226371] serio: i8042 AUX port at 0x60,0x64 irq 12 worker2 # [ 2.247768] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 2.230308] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 2.225192] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 lb # [ 2.226063] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 worker2 # [ 2.253277] ehci-pci 0000:00:1d.7: EHCI Host Controller lb # [ 2.229161] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.254046] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 lb # [ 2.230118] usb usb1: Product: EHCI Host Controller lb # [ 2.230948] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker2 # [ 2.255965] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.259827] uhci_hcd 0000:00:1d.0: detected 2 ports lb # [ 2.234203] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.234968] hub 1-0:1.0: USB hub found worker2 # [ 2.260370] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.236013] hub 1-0:1.0: 6 ports detected client # [ 2.265239] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker1 # [ 2.245054] ehci-pci 0000:00:1d.7: EHCI Host Controller worker2 # [ 2.263693] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker1 # [ 2.245940] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker2 # [ 2.264628] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 lb # [ 2.240818] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker1 # [ 2.247773] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.241695] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker2 # [ 2.267602] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.269625] usb usb1: Product: EHCI Host Controller worker2 # [ 2.270440] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd client # [ 2.273722] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.271270] serio: i8042 AUX port at 0x60,0x64 irq 12 client # [ 2.274867] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.272103] usb usb1: SerialNumber: 0000:00:1d.7 worker2 # [ 2.273451] hub 1-0:1.0: USB hub found worker1 # [ 2.255489] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker2 # [ 2.274877] hub 1-0:1.0: 6 ports detected worker1 # [ 2.258629] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 worker1 # [ 2.259776] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.262965] usb usb1: Product: EHCI Host Controller lb # [ 2.258080] uhci_hcd 0000:00:1d.0: detected 2 ports worker1 # [ 2.265472] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.266208] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.259891] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker2 # [ 2.284516] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker2 # [ 2.285239] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker1 # [ 2.267628] hub 1-0:1.0: USB hub found worker1 # [ 2.269236] hub 1-0:1.0: 6 ports detected client # [ 2.293637] usb usb2: Product: UHCI Host Controller client # [ 2.294315] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.273925] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker1 # [ 2.274783] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 lb # [ 2.269401] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.270518] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.299349] SCSI subsystem initialized worker2 # [ 2.298854] uhci_hcd 0000:00:1d.0: detected 2 ports worker1 # [ 2.286359] uhci_hcd 0000:00:1d.0: detected 2 ports worker1 # [ 2.288035] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 client # [ 2.175210] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.307831] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.287413] SCSI subsystem initialized client # [ 2.316628] usb usb2: SerialNumber: 0000:00:1d.0 lb # [ 2.289977] usb usb2: Product: UHCI Host Controller lb # [ 2.290782] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.315143] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.183144] (udev-worker)[86]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.298370] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.187208] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.189345] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.299498] usb usb2: SerialNumber: 0000:00:1d.0 lb # [ 2.167390] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.326072] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.309312] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.171178] (udev-worker)[86]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.313092] usb usb2: Product: UHCI Host Controller worker1 # [ 2.316733] SCSI subsystem initialized client # [ 2.339682] hub 2-0:1.0: USB hub found worker2 # [ 2.337742] usb usb2: Product: UHCI Host Controller client # [ 2.341643] hub 2-0:1.0: 2 ports detected worker2 # [ 2.338560] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.320318] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.321301] usb usb2: SerialNumber: 0000:00:1d.0 lb # [ 2.180433] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.182410] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.206387] systemd[1]: Starting Virtual Console Setup... lb # [ 2.319204] hub 2-0:1.0: USB hub found lb # [ 2.321490] hub 2-0:1.0: 2 ports detected client # [ 2.348732] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker1 # [ 2.194091] (udev-worker)[88]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.213041] (udev-worker)[86]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.332965] hub 2-0:1.0: USB hub found client # [ 2.219356] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.328008] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker1 # [ 2.199569] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.354412] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.222740] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.220148] (udev-worker)[88]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.338504] hub 2-0:1.0: 2 ports detected worker2 # [ 2.357545] SCSI subsystem initialized worker2 # [ 2.228212] (udev-worker)[86]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.214328] (udev-worker)[86]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.373724] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.374434] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.217673] (udev-worker)[86]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.375824] hub 2-0:1.0: USB hub found worker2 # [ 2.379097] hub 2-0:1.0: 2 ports detected client # [ 2.382403] uhci_hcd 0000:00:1d.1: detected 2 ports lb # [ 2.220350] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.383285] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 lb # [ 2.223571] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.246356] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.359636] uhci_hcd 0000:00:1d.1: UHCI Host Controller lb # [ 2.360371] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker2 # [ 2.249670] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.369232] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.255993] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 2.370021] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 client # [ 2.257587] systemd[1]: Reached target Initrd Root Device. client # [ 2.395015] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.262079] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.239910] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.237785] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 2.378956] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 lb # [ 2.239312] systemd[1]: Reached target Initrd Root Device. lb # [ 2.374281] uhci_hcd 0000:00:1d.1: detected 2 ports lb # [ 2.375034] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 client # [ 2.402621] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.245544] systemd[1]: Finished Virtual Console Setup. client # [ 2.403857] usb usb3: Product: UHCI Host Controller worker2 # [ 2.400848] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.404563] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.242952] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 2.406610] usb usb3: SerialNumber: 0000:00:1d.1 lb # [ 2.379433] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.407467] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker2 # [ 2.408193] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 lb # [ 2.384344] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.385507] usb usb3: Product: UHCI Host Controller client # [ 2.413779] hub 3-0:1.0: USB hub found client # [ 2.415849] hub 3-0:1.0: 2 ports detected lb # [ 2.391186] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.391926] usb usb3: SerialNumber: 0000:00:1d.1 lb # [ 2.393468] hub 3-0:1.0: USB hub found worker2 # [ 2.282909] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 2.401493] uhci_hcd 0000:00:1d.1: detected 2 ports client # [ 2.423285] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker2 # [ 2.285069] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.402365] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker2 # [ 2.421694] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.422480] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 client # [ 2.425651] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.399896] hub 3-0:1.0: 2 ports detected worker2 # [ 2.289065] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 2.428719] uhci_hcd 0000:00:1d.2: detected 2 ports client # [ 2.429522] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.408629] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.402801] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.405183] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker1 # [ 2.275160] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 2.429725] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.432892] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.276052] systemd[1]: Reached target Initrd Root Device. worker2 # [ 2.430842] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.407207] uhci_hcd 0000:00:1d.2: detected 2 ports lb # [ 2.407998] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 client # [ 2.436843] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.281068] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.436705] usb usb3: Product: UHCI Host Controller worker1 # [ 2.418586] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.412234] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.437394] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.414333] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.421399] usb usb3: Product: UHCI Host Controller client # [ 2.307509] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 2.422151] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.446636] usb usb4: Product: UHCI Host Controller worker2 # [ 2.443704] usb usb3: SerialNumber: 0000:00:1d.1 client # [ 2.447438] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.445304] hub 3-0:1.0: USB hub found lb # [ 2.286799] systemd-fsck[108]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.446373] hub 3-0:1.0: 2 ports detected client # [ 2.315227] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.430479] usb usb3: SerialNumber: 0000:00:1d.1 lb # [ 2.291395] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.432530] hub 3-0:1.0: USB hub found client # [ 2.454654] usb usb4: SerialNumber: 0000:00:1d.2 client # [ 2.455543] hub 4-0:1.0: USB hub found worker1 # [ 2.434602] hub 3-0:1.0: 2 ports detected worker2 # [ 2.452969] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.428900] usb usb4: Product: UHCI Host Controller lb # [ 2.429754] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.457766] hub 4-0:1.0: 2 ports detected lb # [ 2.430660] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.454802] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.432632] hub 4-0:1.0: USB hub found worker2 # [ 2.458763] uhci_hcd 0000:00:1d.2: detected 2 ports lb # [ 2.434489] hub 4-0:1.0: 2 ports detected worker2 # [ 2.459586] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.441422] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker1 # [ 2.442492] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker2 # [ 2.462885] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.445514] uhci_hcd 0000:00:1d.2: detected 2 ports worker2 # [ 2.328762] systemd-fsck[105]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 2.446336] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker2 # [ 2.466707] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.450868] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.335964] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.453627] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.473234] usb usb4: Product: UHCI Host Controller worker1 # [ 2.322566] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.477850] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.478849] usb usb4: SerialNumber: 0000:00:1d.2 lb # [ 2.454677] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode client # [ 2.483039] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.481065] hub 4-0:1.0: USB hub found client # [ 2.484633] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 2.327176] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 2.482734] hub 4-0:1.0: 2 ports detected worker1 # [ 2.467910] usb usb4: Product: UHCI Host Controller worker1 # [ 2.468928] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.470516] usb usb4: SerialNumber: 0000:00:1d.2 lb # [ 2.465316] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) client # [ 2.493686] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.466273] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.472653] hub 4-0:1.0: USB hub found client # [ 2.494544] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.474898] hub 4-0:1.0: 2 ports detected lb # [ 2.471179] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 2.504339] scsi host0: ahci lb # [ 2.477226] scsi host0: ahci worker2 # [ 2.501603] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode lb # [ 2.478872] scsi host1: ahci client # [ 2.508341] scsi host1: ahci lb # [ 2.480854] scsi host2: ahci client # [ 2.510115] scsi host2: ahci lb # [ 2.482567] scsi host3: ahci client # [ 2.511652] scsi host3: ahci lb # [ 2.484176] scsi host4: ahci client # [ 2.513298] scsi host4: ahci lb # [ 2.485785] scsi host5: ahci client # [ 2.514902] scsi host5: ahci lb # [ 2.486868] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 client # [ 2.515422] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 lb # [ 2.488534] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker2 # [ 2.514133] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.489674] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 client # [ 2.517534] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker1 # [ 2.496886] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode lb # [ 2.490800] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 client # [ 2.518698] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker2 # [ 2.515695] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 2.498046] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.491936] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 client # [ 2.519828] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 lb # [ 2.493075] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.520948] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker2 # [ 2.518696] ahci 0000:00:1f.2: flags: 64bit ncq only client # [ 2.522168] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker1 # [ 2.505484] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.525740] scsi host0: ahci worker2 # [ 2.527635] scsi host1: ahci worker1 # [ 2.509999] ahci 0000:00:1f.2: flags: 64bit ncq only worker2 # [ 2.529225] scsi host2: ahci worker2 # [ 2.530564] scsi host3: ahci worker2 # [ 2.532268] scsi host4: ahci worker2 # [ 2.533740] scsi host5: ahci worker2 # [ 2.534300] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.518165] scsi host0: ahci worker2 # [ 2.536412] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker1 # [ 2.519845] scsi host1: ahci worker2 # [ 2.537733] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker2 # [ 2.538833] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker1 # [ 2.521664] scsi host2: ahci worker2 # [ 2.540042] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.523247] scsi host3: ahci worker2 # [ 2.541177] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker1 # [ 2.524945] scsi host4: ahci worker1 # [ 2.526594] scsi host5: ahci worker1 # [ 2.527199] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.529324] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker1 # [ 2.530541] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.531676] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker1 # [ 2.533007] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.534209] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.614789] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 client # [ 2.617687] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 client # [ 2.620869] usb 1-1: Product: QEMU USB Tablet client # [ 2.622681] usb 1-1: Manufacturer: QEMU client # [ 2.624264] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 lb # [ 2.600691] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 lb # [ 2.603532] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 lb # [ 2.606684] usb 1-1: Product: QEMU USB Tablet lb # [ 2.608448] usb 1-1: Manufacturer: QEMU lb # [ 2.610012] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker2 # [ 2.645117] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 client # [ 2.653048] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.648039] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker2 # [ 2.651270] usb 1-1: Product: QEMU USB Tablet worker2 # [ 2.653135] usb 1-1: Manufacturer: QEMU worker2 # [ 2.654833] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker1 # [ 2.636949] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.639992] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker1 # [ 2.643066] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.644856] usb 1-1: Manufacturer: QEMU worker1 # [ 2.646686] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 lb # [ 2.641685] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.674051] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 2.678761] hid: raw HID events driver (C) Jiri Kosina client # [ 2.587905] systemd[1]: Mounting /sysroot... worker1 # [ 2.586221] systemd[1]: Mounting /sysroot... lb # [ 2.597073] systemd[1]: Mounting /sysroot... worker2 # [ 2.629353] systemd[1]: Mounting /sysroot... lb # [ 2.808360] ata4: SATA link down (SStatus 0 SControl 300) client # [ 2.837119] ata2: SATA link down (SStatus 0 SControl 300) client # [ 2.838044] ata4: SATA link down (SStatus 0 SControl 300) lb # [ 2.810478] ata1: SATA link down (SStatus 0 SControl 300) client # [ 2.839267] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) client # [ 2.840305] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 client # [ 2.841163] ata3.00: applying bridge limits lb # [ 2.812864] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) client # [ 2.841937] ata1: SATA link down (SStatus 0 SControl 300) client # [ 2.842860] ata5: SATA link down (SStatus 0 SControl 300) client # [ 2.843733] ata6: SATA link down (SStatus 0 SControl 300) lb # [ 2.815276] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 client # [ 2.844534] ata3.00: configured for UDMA/100 lb # [ 2.817316] ata3.00: applying bridge limits client # [ 2.845678] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.819345] ata2: SATA link down (SStatus 0 SControl 300) lb # [ 2.821469] ata6: SATA link down (SStatus 0 SControl 300) lb # [ 2.823724] ata5: SATA link down (SStatus 0 SControl 300) lb # [ 2.825866] ata3.00: configured for UDMA/100 worker2 # [ 2.850831] ata2: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.853082] ata4: SATA link down (SStatus 0 SControl 300) lb # [ 2.828191] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker2 # [ 2.855703] ata5: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.858058] ata6: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.860200] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker2 # [ 2.862888] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker2 # [ 2.864858] ata3.00: applying bridge limits worker2 # [ 2.866935] ata1: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.869043] ata3.00: configured for UDMA/100 worker1 # [ 2.851080] ata6: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.853372] ata4: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.871463] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker1 # [ 2.855967] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker1 # [ 2.858406] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker1 # [ 2.860543] ata3.00: applying bridge limits worker1 # [ 2.862502] ata2: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.864763] ata5: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.866882] ata1: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.869258] ata3.00: configured for UDMA/100 worker1 # [ 2.871752] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 client # [ 2.900847] usbcore: registered new interface driver usbhid client # [ 2.904397] usbhid: USB HID core driver client # [ 2.924817] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 2.903382] usbcore: registered new interface driver usbhid lb # [ 2.904160] usbhid: USB HID core driver client # [ 2.937835] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 client # [ 2.939625] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 2.941629] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 lb # [ 2.920453] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker2 # [ 2.945772] usbcore: registered new interface driver usbhid worker2 # [ 2.946598] usbhid: USB HID core driver lb # [ 2.922232] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 2.925251] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 client # [ 2.956332] EXT4-fs (vda): mounted filesystem 76799aa6-4ca2-4066-b0be-376a60231b64 r/w with ordered data mode. Quota mode: none. lb # [ 2.934245] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 2.826886] systemd[1]: Mounted /sysroot. client # [ 2.828773] systemd[1]: Reached target Initrd Root File System. worker1 # [ 2.945669] usbcore: registered new interface driver usbhid worker1 # [ 2.946420] usbhid: USB HID core driver client # [ 2.832085] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 2.976104] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 lb # [ 2.953215] EXT4-fs (vda): mounted filesystem e136e456-ff5d-48ae-b492-c371e2be8835 r/w with ordered data mode. Quota mode: none. worker2 # [ 2.977892] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 lb # [ 2.823579] systemd[1]: Mounted /sysroot. worker2 # [ 2.982362] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray client # [ 2.849877] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 2.826139] systemd[1]: Reached target Initrd Root File System. worker1 # [ 2.966596] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 2.828250] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 2.855768] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 2.857239] systemd[1]: Reloading... worker2 # [ 2.992836] cdrom: Uniform CD-ROM driver Revision: 3.20 worker2 # [ 2.997199] EXT4-fs (vda): mounted filesystem ed1e66d5-357a-4014-b002-15e4458b76d3 r/w with ordered data mode. Quota mode: none. worker1 # [ 2.980651] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 2.982400] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker1 # [ 2.984664] cdrom: Uniform CD-ROM driver Revision: 3.20 worker2 # [ 2.867702] systemd[1]: Mounted /sysroot. lb # [ 2.844191] systemd-sysroot-fstab-check[139]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 2.869906] systemd[1]: Reached target Initrd Root File System. lb # [ 2.848066] systemd[1]: Reload requested from client PID 139 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 2.849494] systemd[1]: Reloading... worker2 # [ 2.873322] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 2.998987] EXT4-fs (vda): mounted filesystem 8851ecf9-41c2-47fe-beec-3e4751d76772 r/w with ordered data mode. Quota mode: none. worker2 # [ 2.885792] systemd-sysroot-fstab-check[137]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 2.868837] systemd[1]: Mounted /sysroot. worker1 # [ 2.871903] systemd[1]: Reached target Initrd Root File System. worker2 # [ 2.891068] systemd[1]: Reload requested from client PID 137 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 2.892505] systemd[1]: Reloading... worker1 # [ 2.873767] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 2.888767] systemd-sysroot-fstab-check[139]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 2.891538] systemd[1]: Reload requested from client PID 139 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 2.892972] systemd[1]: Reloading... client # [ 2.938440] systemd[1]: Reloading finished in 83 ms. client # [ 2.947268] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... client # [ 2.949511] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... client # [ 2.955139] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 2.956556] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 2.957565] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 2.934894] systemd[1]: Reloading finished in 87 ms. lb # [ 2.944544] systemd-sysroot-fstab-check[139]: Requesting initrd-fs.target/start/replace... lb # [ 2.947146] systemd-sysroot-fstab-check[139]: Requesting swap.target/start/replace... worker2 # [ 2.972416] systemd[1]: Reloading finished in 81 ms. lb # [ 2.952765] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 2.954280] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 2.955290] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 2.981723] systemd-sysroot-fstab-check[137]: Requesting initrd-fs.target/start/replace... worker2 # [ 2.985156] systemd-sysroot-fstab-check[137]: Requesting swap.target/start/replace... worker2 # [ 2.989546] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 2.990913] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 2.992056] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 2.973494] systemd[1]: Reloading finished in 79 ms. worker1 # [ 2.981281] systemd-sysroot-fstab-check[139]: Requesting initrd-fs.target/start/replace... worker1 # [ 2.984152] systemd-sysroot-fstab-check[139]: Requesting swap.target/start/replace... worker1 # [ 2.988569] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 2.989895] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 2.990909] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.589309] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.596164] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.603178] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.588167] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.614167] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.594152] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.623334] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.602158] systemd[1]: Mounting /sysroot/run... lb # [ 3.599187] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.605156] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.630716] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.612231] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.633728] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.613202] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.638124] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.619969] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.642143] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.646727] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.623304] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.661409] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.636549] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.666513] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.670083] systemd[1]: Mounted /sysroot/run. client # [ 3.671236] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.668174] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.672637] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.676986] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.674930] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.676948] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.677610] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.659218] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.680809] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.662729] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.683468] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.687343] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.665421] systemd[1]: Mounted /sysroot/run. client # [ 3.688743] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.690964] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 3.668908] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.669877] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.668450] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.669289] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.673609] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.693043] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.694430] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.696549] systemd[1]: Mounting /sysroot/nix/store... lb # [ 3.674257] systemd[1]: Mounted /sysroot/run. lb # [ 3.675889] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.677913] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.685055] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 3.681670] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.686052] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.687815] systemd[1]: Mounting /sysroot/nix/store... client # [ 3.712773] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.714088] systemd[1]: Reached target Initrd File Systems. client # [ 3.715625] systemd[1]: Starting Find NixOS closure... lb # [ 3.692324] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.719149] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 3.716329] systemd[1]: Mounted /sysroot/nix/store. lb # [ 3.693749] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.717420] systemd[1]: Reached target Initrd File Systems. worker2 # [ 3.719052] systemd[1]: Starting Find NixOS closure... worker2 # [ 3.721631] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 3.708344] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 3.709299] systemd[1]: Reached target Initrd File Systems. worker1 # [ 3.710851] systemd[1]: Starting Find NixOS closure... worker1 # [ 3.713322] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 3.735968] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 3.739214] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 3.743085] systemd[1]: Finished Find NixOS closure. client # [ 3.744479] systemd[1]: Reached target Initrd Default Target. client # [ 3.746289] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 3.746841] systemd[1]: Finished Find NixOS closure. worker2 # [ 3.748173] systemd[1]: Reached target Initrd Default Target. worker2 # [ 3.749624] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 3.731988] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 3.759559] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 3.738330] systemd[1]: Finished Find NixOS closure. client # [ 3.760896] systemd[1]: Stopped target Basic System. client # [ 3.761807] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 3.739480] systemd[1]: Reached target Initrd Default Target. client # [ 3.762760] systemd[1]: Stopped target Path Units. worker1 # [ 3.741440] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 3.763644] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.765113] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 3.762099] systemd[1]: Stopped target Initrd Default Target. client # [ 3.766305] systemd[1]: Stopped target Slice Units. worker2 # [ 3.763471] systemd[1]: Stopped target Basic System. client # [ 3.767264] systemd[1]: Stopped target Socket Units. worker2 # [ 3.764397] systemd[1]: Stopped target Initrd Root Device. client # [ 3.769123] systemd[1]: Stopped target System Initialization. worker2 # [ 3.766107] systemd[1]: Stopped target Path Units. client # [ 3.769953] systemd[1]: Stopped target Swaps. worker2 # [ 3.766963] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.770643] systemd[1]: Stopped target Timer Units. client # [ 3.771338] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 3.767988] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 3.772217] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 3.769103] systemd[1]: Stopped target Slice Units. worker2 # [ 3.769808] systemd[1]: Stopped target Socket Units. client # [ 3.773053] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 3.770623] systemd[1]: Stopped target System Initialization. client # [ 3.774068] systemd[1]: Stopped Find NixOS closure. worker2 # [ 3.771771] systemd[1]: Stopped target Swaps. worker2 # [ 3.772617] systemd[1]: Stopped target Timer Units. client # [ 3.775724] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.754188] systemd[1]: Stopped target Initrd Default Target. client # [ 3.777156] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.774134] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 3.755444] systemd[1]: Stopped target Basic System. client # [ 3.778201] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.775084] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 3.756354] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 3.775888] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 3.779200] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 3.757289] systemd[1]: Stopped target Path Units. worker2 # [ 3.776884] systemd[1]: Stopped Find NixOS closure. client # [ 3.780366] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 3.778061] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.759110] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.781388] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 3.779117] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 3.760108] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 3.761135] systemd[1]: Stopped target Slice Units. worker2 # [ 3.780229] systemd[1]: Stopped Apply Kernel Variables. client # [ 3.783295] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 3.761806] systemd[1]: Stopped target Socket Units. client # [ 3.784527] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 3.762589] systemd[1]: Stopped target System Initialization. client # [ 3.785733] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 3.763658] systemd[1]: Stopped target Swaps. worker2 # [ 3.782886] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 3.764331] systemd[1]: Stopped target Timer Units. client # [ 3.786888] systemd[1]: Stopped target Local File Systems. worker2 # [ 3.783882] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 3.765025] systemd[1]: dbus.socket: Deactivated successfully. client # [ 3.787912] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 3.784595] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 3.766132] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 3.785793] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 3.789214] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.766923] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 3.768122] systemd[1]: Stopped Find NixOS closure. client # [ 3.790393] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 3.787325] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 3.769552] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.788647] systemd[1]: Stopped Create System Files and Directories. client # [ 3.792137] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 3.789524] systemd[1]: Stopped target Local File Systems. client # [ 3.793131] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 3.771173] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 3.794324] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 3.772236] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.791335] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 3.773252] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 3.792294] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.774332] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 3.793576] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 3.775275] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 3.794909] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 3.795923] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 3.777118] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 3.778304] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 3.798129] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 3.779473] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 3.780653] systemd[1]: Stopped target Local File Systems. client # [ 3.803974] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 3.781739] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 3.783125] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.784251] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 3.803870] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 3.808142] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 3.786548] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 3.787575] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 3.788888] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 3.808108] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 3.813196] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 3.810942] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 3.815906] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 3.814204] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 3.817775] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.819301] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.797386] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 3.820510] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 3.817580] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 3.818774] systemd[1]: Closed udev Control Socket. client # [ 3.822135] systemd[1]: Closed udev Control Socket. worker2 # [ 3.819831] systemd[1]: Starting Cleanup udev Database... client # [ 3.823426] systemd[1]: Starting Cleanup udev Database... worker2 # [ 3.820629] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 3.801668] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 3.824332] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 3.822146] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 3.825709] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 3.823060] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 3.826584] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 3.824124] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 3.828330] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 3.806378] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 3.829320] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 3.826115] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 3.827052] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 3.807936] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 3.831129] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 3.828096] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.809479] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.829474] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.810889] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.812375] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 3.814086] systemd[1]: Closed udev Control Socket. worker1 # [ 3.815432] systemd[1]: Starting Cleanup udev Database... worker1 # [ 3.816712] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 3.817764] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 3.818648] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 3.820133] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 3.821091] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 3.822106] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 3.845814] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 3.847167] systemd[1]: Finished Cleanup udev Database. worker2 # [ 3.844333] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 3.848363] systemd[1]: Reached target Switch Root. worker2 # [ 3.845900] systemd[1]: Finished Cleanup udev Database. client # [ 3.849878] systemd[1]: Starting NixOS Activation... worker2 # [ 3.847955] systemd[1]: Reached target Switch Root. worker2 # [ 3.848927] systemd[1]: Starting NixOS Activation... worker1 # [ 3.836482] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 3.838171] systemd[1]: Finished Cleanup udev Database. worker1 # [ 3.840116] systemd[1]: Reached target Switch Root. worker1 # [ 3.842132] systemd[1]: Starting NixOS Activation... client # [ 3.911423] initrd-nixos-activation-start[190]: booting system configuration /nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test worker2 # [ 3.920742] initrd-nixos-activation-start[189]: booting system configuration /nix/store/wdxq8bvyk71y06pp5xincwaxz514413f-nixos-system-worker2-test worker1 # [ 3.910184] initrd-nixos-activation-start[191]: booting system configuration /nix/store/mjibqls45h3zals0927jhbwnzswp7x1m-nixos-system-worker1-test client # [ 3.937256] initrd-nixos-activation-start[190]: running activation script... worker2 # [ 3.948074] initrd-nixos-activation-start[189]: running activation script... worker1 # [ 3.936268] initrd-nixos-activation-start[191]: running activation script... client # [ 4.133862] initrd-nixos-activation-start[213]: setting up /etc... worker2 # [ 4.149249] initrd-nixos-activation-start[212]: setting up /etc... worker1 # [ 4.130809] initrd-nixos-activation-start[214]: setting up /etc... client # [ 4.227085] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 4.229164] systemd[1]: Finished NixOS Activation. client # [ 4.230407] systemd[1]: Starting Switch Root... client # [ 4.244348] systemd[1]: Switching root. worker1 # [ 4.235171] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 4.254867] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 4.237097] systemd[1]: Finished NixOS Activation. worker2 # [ 4.257078] systemd[1]: Finished NixOS Activation. worker1 # [ 4.238597] systemd[1]: Starting Switch Root... worker2 # [ 4.258330] systemd[1]: Starting Switch Root... worker2 # [ 4.269574] systemd[1]: Switching root. worker1 # [ 4.251583] systemd[1]: Switching root. client # [ 4.506461] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker1 # [ 4.512511] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker2 # [ 4.535727] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 4.594166] NET: Registered PF_VSOCK protocol family worker2 # [ 4.604443] NET: Registered PF_VSOCK protocol family worker1 # [ 4.599636] NET: Registered PF_VSOCK protocol family lb # [ 4.598290] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.646097] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.648647] systemd[1]: Reached target Initrd File Systems. lb # [ 4.651139] systemd[1]: Starting Find NixOS closure... lb # [ 4.652628] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.671206] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.674451] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.680892] systemd[1]: Finished Find NixOS closure. lb # [ 4.682168] systemd[1]: Reached target Initrd Default Target. lb # [ 4.683648] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.695804] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.697133] systemd[1]: Stopped target Basic System. lb # [ 4.698066] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.698975] systemd[1]: Stopped target Path Units. lb # [ 4.699875] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.702109] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.703110] systemd[1]: Stopped target Slice Units. lb # [ 4.703779] systemd[1]: Stopped target Socket Units. lb # [ 4.704673] systemd[1]: Stopped target System Initialization. lb # [ 4.705467] systemd[1]: Stopped target Swaps. lb # [ 4.706121] systemd[1]: Stopped target Timer Units. lb # [ 4.706775] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.707618] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.709065] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.710092] systemd[1]: Stopped Find NixOS closure. lb # [ 4.711141] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.712083] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.714164] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.715128] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.716297] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.717279] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.718599] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.720117] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.721164] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.722055] systemd[1]: Stopped target Local File Systems. lb # [ 4.723094] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.723984] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.726107] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.728131] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.730860] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.731798] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.737118] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.741494] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 4.745917] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.749117] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.750214] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.751265] systemd[1]: Closed udev Control Socket. lb # [ 4.752461] systemd[1]: Starting Cleanup udev Database... lb # [ 4.753302] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.755203] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.756034] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.757044] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.757925] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.758855] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 4.759692] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.762137] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.776661] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 4.779072] systemd[1]: Finished Cleanup udev Database. lb # [ 4.779770] systemd[1]: Reached target Switch Root. lb # [ 4.780934] systemd[1]: Starting NixOS Activation... client # [ 4.954694] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 4.966305] systemd[1]: Detected virtualization kvm. client # [ 4.968667] systemd[1]: Detected architecture x86-64. client # [ 4.970931] systemd[1]: Detected first boot. worker2 # [ 4.964610] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 4.975531] systemd[1]: Initializing machine ID from random generator. worker2 # [ 4.974293] systemd[1]: Detected virtualization kvm. worker2 # [ 4.976122] systemd[1]: Detected architecture x86-64. worker2 # [ 4.978053] systemd[1]: Detected first boot. worker2 # [ 4.981946] systemd[1]: Initializing machine ID from random generator. worker1 # [ 4.958197] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 4.968007] systemd[1]: Detected virtualization kvm. worker1 # [ 4.969986] systemd[1]: Detected architecture x86-64. worker1 # [ 4.971940] systemd[1]: Detected first boot. worker1 # [ 4.975965] systemd[1]: Initializing machine ID from random generator. lb # [ 4.852820] initrd-nixos-activation-start[191]: booting system configuration /nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test lb # [ 4.880144] initrd-nixos-activation-start[191]: running activation script... client # [ 5.208028] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 5.220557] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 5.077627] initrd-nixos-activation-start[214]: setting up /etc... worker1 # [ 5.217972] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 5.327840] systemd[1]: Applying preset policy. worker2 # [ 5.331616] systemd[1]: Applying preset policy. lb # [ 5.182492] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.184146] systemd[1]: Finished NixOS Activation. lb # [ 5.185770] systemd[1]: Starting Switch Root... worker1 # [ 5.333876] systemd[1]: Applying preset policy. lb # [ 5.198639] systemd[1]: Switching root. lb # [ 5.459543] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 5.507033] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 5.509225] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 5.510885] systemd[1]: Populated /etc with preset unit settings. lb # [ 5.553617] NET: Registered PF_VSOCK protocol family client # [ 5.714846] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 5.693549] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 5.711982] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 5.695077] systemd[1]: Stopped initrd-switch-root.service. client # [ 5.716452] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 5.713528] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 5.697473] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 5.719003] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 5.716041] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 5.721045] systemd[1]: Created slice Slice /system/getty. worker1 # [ 5.699761] systemd[1]: Created slice Slice /system/getty. worker2 # [ 5.718031] systemd[1]: Created slice Slice /system/getty. worker1 # [ 5.701043] systemd[1]: Created slice User and Session Slice. client # [ 5.722451] systemd[1]: Created slice User and Session Slice. worker2 # [ 5.719440] systemd[1]: Created slice User and Session Slice. worker1 # [ 5.701994] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 5.723402] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.720326] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.703219] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 5.724729] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 5.721554] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 5.704301] systemd[1]: Expecting device /dev/hvc0... client # [ 5.725821] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 5.722694] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 5.705074] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 5.723402] systemd[1]: Expecting device /dev/ttyS0... client # [ 5.726522] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 5.724152] systemd[1]: Reached target Local Encrypted Volumes. client # [ 5.727328] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 5.705945] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 5.724970] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 5.706789] systemd[1]: Stopped target initrd-fs.target. client # [ 5.728214] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 5.725742] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 5.707551] systemd[1]: Stopped target initrd-root-fs.target. client # [ 5.729032] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 5.726534] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 5.708306] systemd[1]: Stopped target initrd-switch-root.target. client # [ 5.729842] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 5.727402] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 5.709204] systemd[1]: Reached target Virtual Machines and Containers. client # [ 5.730749] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 5.728281] systemd[1]: Reached target Path Units. worker1 # [ 5.710151] systemd[1]: Reached target Path Units. client # [ 5.731676] systemd[1]: Reached target Path Units. worker2 # [ 5.729014] systemd[1]: Reached target Remote File Systems. worker1 # [ 5.710863] systemd[1]: Reached target Remote File Systems. client # [ 5.732397] systemd[1]: Reached target Remote File Systems. worker2 # [ 5.729817] systemd[1]: Reached target Slice Units. worker1 # [ 5.711626] systemd[1]: Reached target Slice Units. client # [ 5.733239] systemd[1]: Reached target Slice Units. worker2 # [ 5.730536] systemd[1]: Reached target Swaps. worker1 # [ 5.712309] systemd[1]: Reached target Swaps. client # [ 5.734052] systemd[1]: Reached target Swaps. worker2 # [ 5.732461] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 5.714174] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 5.736075] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 5.716607] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 5.734946] systemd[1]: Listening on Process Core Dump Socket. client # [ 5.738750] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 5.718308] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 5.740533] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 5.736798] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 5.720159] systemd[1]: Listening on Factory Reset Management. worker2 # [ 5.750398] systemd[1]: Listening on Factory Reset Management. worker1 # [ 5.732607] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 5.751305] systemd[1]: Listening on Hostname Service Socket. client # [ 5.742630] systemd[1]: Listening on Factory Reset Management. client # [ 5.755463] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 5.735378] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 5.754115] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 5.736726] systemd[1]: Listening on Journal Audit Socket. client # [ 5.758419] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 5.755437] systemd[1]: Listening on Journal Audit Socket. client # [ 5.759908] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 5.739352] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 5.757995] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 5.740509] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 5.762784] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 5.759608] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 5.741589] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.763913] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 5.760762] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.742894] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 5.765122] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.762050] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 5.766447] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 5.746778] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 5.747811] systemd[1]: Listening on udev Control Socket. worker2 # [ 5.766149] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 5.748676] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 5.767161] systemd[1]: Listening on udev Control Socket. client # [ 5.770287] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 5.768006] systemd[1]: Listening on udev Varlink Socket. client # [ 5.771354] systemd[1]: Listening on udev Control Socket. worker1 # [ 5.750761] systemd[1]: Mounting Huge Pages File System... client # [ 5.772332] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 5.770161] systemd[1]: Mounting Huge Pages File System... client # [ 5.774465] systemd[1]: Mounting Huge Pages File System... worker1 # [ 5.753908] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 5.772829] systemd[1]: Mounting POSIX Message Queue File System... client # [ 5.778783] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 5.778811] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 5.760850] systemd[1]: Mounting Kernel Debug File System... client # [ 5.786019] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 5.767823] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 5.787528] systemd[1]: Mounting Kernel Trace File System... client # [ 5.792107] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 5.794325] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 5.778474] systemd[1]: Starting Create List of Static Device Nodes... client # [ 5.801780] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 5.784939] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 5.804530] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 5.787816] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 5.808115] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 5.813068] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 5.791803] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 5.810236] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 5.813900] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 5.795827] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 5.820000] systemd[1]: Mounting FUSE Control File System... client # [ 5.823193] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 5.809704] systemd[1]: Mounting FUSE Control File System... client # [ 5.831558] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 5.829062] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 5.815653] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 5.840010] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 5.852413] systemd[1]: Mounting FUSE Control File System... worker1 # [ 5.835403] systemd[1]: Starting Journal Service... client # [ 5.855530] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 5.855907] systemd[1]: Starting Journal Service... worker1 # [ 5.840354] systemd[1]: Starting Load Kernel Modules... client # [ 5.862458] systemd[1]: Starting Journal Service... worker2 # [ 5.864959] systemd[1]: Starting Load Kernel Modules... worker1 # [ 5.849861] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 5.872889] systemd[1]: Starting Load Kernel Modules... worker2 # [ 5.873584] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 5.856511] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 5.857404] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.878348] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 5.880719] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.863274] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 5.867775] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 5.886430] systemd[1]: Starting Coldplug All udev Devices... client # [ 5.889941] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 5.868947] systemd[1]: Mounted Huge Pages File System. worker2 # [ 5.889939] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 5.872188] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 5.892226] systemd[1]: Mounted Huge Pages File System. worker1 # [ 5.874126] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 5.894837] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.876663] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 5.896839] systemd[1]: Mounted Kernel Debug File System. client # [ 5.900046] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 5.879402] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 5.898242] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 5.881778] systemd[1]: Mounted FUSE Control File System. client # [ 5.903016] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.901228] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 5.903060] systemd[1]: Mounted FUSE Control File System. worker1 # [ 5.886952] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 5.910772] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 5.909241] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 5.917346] systemd-journald[283]: Collecting audit messages is enabled. worker1 # [ 5.902969] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 5.921490] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 5.925300] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 5.922917] systemd[1]: Finished Load Kernel Module configfs. worker1 # [ 5.905546] systemd[1]: Finished Load Kernel Module configfs. client # [ 5.929791] systemd[1]: Mounted Huge Pages File System. worker2 # [ 5.927822] systemd[1]: Mounting Kernel Configuration File System... client # [ 5.932911] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.912490] systemd[1]: Mounting Kernel Configuration File System... client # [ 5.801752] systemd[1]: Queued start job for default target Multi-User System. client # [ 5.804376] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 5.941760] systemd[1]: Started Journal Service. client # [ 5.812144] systemd[1]: Mounted Kernel Debug File System. client # [ 5.812921] systemd[1]: Mounted Kernel Trace File System. lb # [ 5.913086] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.922438] systemd[1]: Detected virtualization kvm. lb # [ 5.924187] systemd[1]: Detected architecture x86-64. worker2 # [ 5.949770] systemd[1]: Mounted Kernel Configuration File System. lb # [ 5.925978] systemd[1]: Detected first boot. client # [ 5.819198] systemd[1]: Finished Create List of Static Device Nodes. client # [ 5.820527] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 5.934740] systemd[1]: Mounted Kernel Configuration File System. lb # [ 5.929878] systemd[1]: Initializing machine ID from random generator. client # [ 5.826302] systemd[1]: Finished Load Kernel Module configfs. client # [ 5.979042] loop: module loaded worker2 # [ 5.978247] EXT4-fs (vda): re-mounted ed1e66d5-357a-4014-b002-15e4458b76d3. worker1 # [ 5.961498] EXT4-fs (vda): re-mounted 8851ecf9-41c2-47fe-beec-3e4751d76772. client # [ 5.847608] systemd-modules-load[284]: Inserted module 'loop' worker2 # [ 5.980901] systemd-journald[282]: Collecting audit messages is enabled. worker1 # [ 5.968512] systemd-journald[284]: Collecting audit messages is enabled. worker1 # [ 5.970537] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 5.989055] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 5.971698] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 5.991002] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 5.860137] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 5.973530] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 5.976160] loop: module loaded worker2 # [ 5.993744] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 5.864109] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 5.998165] loop: module loaded client # [ 5.865590] systemd[1]: Mounted FUSE Control File System. worker1 # [ 5.981848] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.000326] systemd[1]: Starting Load/Save OS Random Seed... client # [ 5.869084] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.002703] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.984483] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.008404] EXT4-fs (vda): re-mounted 76799aa6-4ca2-4066-b0be-376a60231b64. worker1 # [ 5.987604] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.009526] systemd[1]: Finished Load Kernel Modules. worker1 # [ 5.991785] systemd[1]: Starting Firewall... client # [ 5.880519] systemd-oomd[286]: No swap; memory pressure usage will be degraded worker2 # [ 6.015996] systemd[1]: Starting Firewall... worker1 # [ 5.997732] systemd[1]: Starting Apply Kernel Variables... client # [ 5.887241] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 6.022403] systemd[1]: Starting Apply Kernel Variables... client # [ 5.891353] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 5.892373] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 5.897226] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 5.906389] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 5.907258] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 6.022282] systemd[1]: Started Journal Service. worker2 # [ 6.043085] systemd[1]: Started Journal Service. worker1 # [ 5.892354] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 5.911506] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 5.893481] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 5.913632] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 5.920104] systemd[1]: Starting Load/Save OS Random Seed... client # [ 5.920895] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.900876] systemd-modules-load[285]: Inserted module 'loop' worker2 # [ 5.921691] systemd-modules-load[283]: Inserted module 'loop' worker1 # [ 5.905061] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 5.924506] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 5.911366] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 5.943581] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 5.925128] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 5.945880] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 5.967361] systemd[1]: Mounted Kernel Configuration File System. client # [ 6.104534] systemd-journald[283]: Received client request to flush runtime journal. worker1 # [ 5.953058] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 5.953927] systemd[1]: Reached target First Boot Complete. worker2 # [ 5.973994] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 5.974885] systemd[1]: Reached target First Boot Complete. worker2 # [ 6.134894] systemd-journald[282]: Received client request to flush runtime journal. worker1 # [ 6.121906] systemd-journald[284]: Received client request to flush runtime journal. client # [ 6.054289] systemd[1]: Finished Load/Save OS Random Seed. client # [ 6.056246] systemd[1]: Reached target First Boot Complete. client # [ 6.057727] systemd[1]: Finished Apply Kernel Variables. lb # [ 6.168230] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.062349] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 6.088695] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 6.093551] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 6.096220] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 6.099306] systemd-oomd[284]: No swap; memory pressure usage will be degraded worker2 # [ 6.104266] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 6.107118] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 6.118951] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 6.119933] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 6.122711] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 6.111166] systemd-oomd[286]: No swap; memory pressure usage will be degraded worker1 # [ 6.113246] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 6.115215] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 6.116925] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 6.139462] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 6.118152] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 6.120145] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 6.122106] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 6.152620] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 6.154187] systemd[1]: Reached target Preparation for Local File Systems. client # [ 6.156848] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 6.272180] systemd[1]: Applying preset policy. client # [ 6.186843] systemd-udevd[315]: Using default interface naming scheme 'v261'. worker1 # [ 6.177404] systemd-udevd[317]: Using default interface naming scheme 'v261'. worker2 # [ 6.213336] systemd-udevd[317]: Using default interface naming scheme 'v261'. client # [ 6.219221] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 6.248582] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 6.288575] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 6.300727] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 6.334193] systemd[1]: Finished Coldplug All udev Devices. lb # [ 6.446699] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.324364] systemd[1]: Finished Coldplug All udev Devices. client # [ 6.349047] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 6.360056] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. client # [ 6.381901] (udev-worker)[321]: Network interface NamePolicy= disabled on kernel command line. client # [ 6.387241] (udev-worker)[320]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 6.389555] (udev-worker)[320]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.432058] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 6.415744] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.459101] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 6.460251] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.462080] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.464181] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.466117] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.467369] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.492475] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 6.513434] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 6.660658] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 client # [ 6.669103] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console lb # [ 6.641762] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.643631] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.646601] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.648706] systemd[1]: Created slice Slice /system/getty. lb # [ 6.650036] systemd[1]: Created slice User and Session Slice. lb # [ 6.650919] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 6.520974] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. lb # [ 6.652108] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 6.679764] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 6.653195] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.653864] systemd[1]: Expecting device /dev/ttyS0... lb # [ 6.654633] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.655451] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.656195] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.656949] systemd[1]: Stopped target initrd-switch-root.target. client # [ 6.684854] ACPI: button: Power Button [PWRF] lb # [ 6.657773] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.658653] systemd[1]: Reached target Path Units. lb # [ 6.659349] systemd[1]: Reached target Remote File Systems. lb # [ 6.660091] systemd[1]: Reached target Slice Units. lb # [ 6.660781] systemd[1]: Reached target Swaps. worker2 # [ 6.550487] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. lb # [ 6.662843] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.665367] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.667163] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 6.549459] (udev-worker)[361]: Network interface NamePolicy= disabled on kernel command line. lb # [ 6.669022] systemd[1]: Listening on Factory Reset Management. lb # [ 6.681099] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.683858] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.685218] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 6.553947] (udev-worker)[359]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 6.556386] (udev-worker)[359]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.575445] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. lb # [ 6.687742] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.689193] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.582744] systemd[1]: Mounting /run/wrappers... lb # [ 6.690404] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.691661] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 6.562126] systemd[1]: Mounting /run/wrappers... worker2 # [ 6.581230] systemd[1]: Mounting /run/wrappers... lb # [ 6.695946] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.696938] systemd[1]: Listening on udev Control Socket. lb # [ 6.697776] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.699897] systemd[1]: Mounting Huge Pages File System... lb # [ 6.702421] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 6.710207] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 6.598225] (udev-worker)[349]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 6.600797] (udev-worker)[349]: Network interface NamePolicy= disabled on kernel command line. lb # [ 6.716341] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.727961] systemd[1]: Starting Create List of Static Device Nodes... client # [ 6.620549] systemd[1]: Mounted /run/wrappers. client # [ 6.621276] systemd[1]: Reached target Local File Systems. worker2 # [ 6.620957] systemd[1]: Mounted /run/wrappers. client # [ 6.624535] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 6.621733] systemd[1]: Reached target Local File Systems. client # [ 6.627129] systemd[1]: Starting register-nix-paths.service... worker2 # [ 6.624277] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.630081] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 6.626993] systemd[1]: Starting register-nix-paths.service... client # [ 6.630909] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 6.629247] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 6.741794] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 6.630728] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 6.614481] systemd[1]: Mounted /run/wrappers. worker1 # [ 6.615653] systemd[1]: Reached target Local File Systems. worker2 # [ 6.635210] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 6.618131] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 6.748531] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 6.639295] systemd[1]: Starting Create System Files and Directories... worker1 # [ 6.621063] systemd[1]: Starting register-nix-paths.service... worker1 # [ 6.623349] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 6.625473] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 6.757007] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 6.629353] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 6.631140] systemd[1]: Starting Create System Files and Directories... lb # [ 6.764269] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.774493] systemd[1]: Mounting FUSE Control File System... lb # [ 6.779227] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.791466] systemd[1]: Starting Journal Service... lb # [ 6.799213] systemd[1]: Starting Load Kernel Modules... client # [ 6.693989] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 6.700290] systemd[1]: Starting Create System Files and Directories... lb # [ 6.808069] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.819782] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 6.824287] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.832693] systemd[1]: Starting Coldplug All udev Devices... client # [ 6.693527] mousedev: PS/2 mouse device common for all mice worker2 # [ 6.734738] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 6.716104] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 6.849307] systemd-journald[284]: Collecting audit messages is enabled. worker2 # [ 6.738380] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 6.742121] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 6.720469] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 6.854213] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.746750] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 6.732122] rtc_cmos PNP0B00:00: RTC can wake from S4 client # [ 6.769417] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 6.769486] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T16:12:15 UTC (1790179935) lb # [ 6.857300] systemd[1]: Mounted Huge Pages File System. lb # [ 6.861251] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 6.864405] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.868496] systemd[1]: Mounted Kernel Trace File System. lb # [ 6.875935] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.882038] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 6.751345] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.752509] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 6.781746] systemd[1]: Finished Create System Files and Directories. lb # [ 6.891451] systemd[1]: Finished Load Kernel Module configfs. lb # [ 6.894842] loop: module loaded lb # [ 6.896383] systemd[1]: Started Journal Service. client # [ 6.789238] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 6.768074] systemd[1]: Finished Create System Files and Directories. lb # [ 6.764385] systemd-modules-load[285]: Inserted module 'loop' worker2 # [ 6.788128] systemd[1]: Finished Create System Files and Directories. client # [ 6.794489] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 6.769511] systemd[1]: Mounted FUSE Control File System. worker1 # [ 6.773534] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 6.793550] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 6.775987] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 6.769822] systemd-journald[283]: Time jumped backwards, rotating. worker2 # [ 6.796959] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 6.772610] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs client # [ 6.789349] parport_pc 00:02: reported by Plug and Play ACPI client # [ 6.789439] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] lb # [ 6.776195] systemd[1]: Finished Load Kernel Modules. lb # [ 6.922231] EXT4-fs (vda): re-mounted e136e456-ff5d-48ae-b492-c371e2be8835. lb # [ 6.794072] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.799065] systemd[1]: Starting Firewall... lb # [ 6.806156] systemd[1]: Starting Apply Kernel Variables... lb # [ 6.816190] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.819989] systemd-oomd[287]: No swap; memory pressure usage will be degraded lb # [ 6.825142] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 6.825993] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 6.858460] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 6.833268] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 6.848224] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 6.871743] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 6.853634] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 6.850127] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.881321] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 6.856517] systemd[1]: Starting Load/Save OS Random Seed... client # [ 6.850979] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 6.872011] Console: switching to colour dummy device 80x25 lb # [ 6.857690] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.020527] [drm] Found bochs VGA, ID 0xb0c5. client # [ 7.020530] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. client # [ 6.886891] systemd[1]: Starting Update is Completed... worker1 # [ 6.898736] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 6.899906] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.924288] systemd[1]: Finished Update is Completed. worker1 # [ 6.901917] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 6.904594] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.065815] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client # [ 7.066531] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 6.908278] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.928426] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 6.911238] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.931697] systemd[1]: Starting Update is Completed... worker1 # [ 6.912936] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.917572] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 6.919489] systemd[1]: Starting Update is Completed... lb # [ 7.051138] systemd-journald[284]: Received client request to flush runtime journal. worker1 # [ 6.984350] systemd[1]: Finished Update is Completed. worker2 # [ 7.015230] systemd[1]: Finished Update is Completed. lb # [ 7.007368] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.031861] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 7.009779] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.011562] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.034394] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.012490] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.036133] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.015058] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.038452] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.042556] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.044194] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.045809] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.041696] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.048515] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 7.091960] systemd[1]: Starting Virtual Console Setup... client # [ 7.151605] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker1 # [ 7.104480] systemd[1]: Finished Firewall. lb # [ 7.103939] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 7.264267] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 client # [ 7.151847] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 client # [ 7.159253] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt client # [ 7.159353] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD lb # [ 7.128140] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.128989] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.131470] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 7.241782] Console: switching to colour frame buffer device 160x50 client # [ 7.298288] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker1 # [ 7.301017] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker2 # [ 7.330557] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console lb # [ 7.175373] systemd-udevd[326]: Using default interface naming scheme 'v261'. worker2 # [ 7.202408] systemd[1]: Finished Firewall. client # [ 7.205401] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.208150] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 7.322485] ACPI: button: Power Button [PWRF] client # [ 7.212878] systemd[1]: Starting Virtual Console Setup... worker1 # [ 7.336323] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 7.356353] rtc_cmos PNP0B00:00: RTC can wake from S4 worker1 # [ 7.364890] mousedev: PS/2 mouse device common for all mice lb # [ 7.225300] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 7.393172] ppdev: user-space parallel port driver worker1 # [ 7.244577] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 7.247198] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 7.271113] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 7.274083] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 7.288937] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 7.291897] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 7.433264] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 7.454319] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) client # [ 7.327260] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.331220] systemd[1]: Stopped Virtual Console Setup. client # [ 7.334931] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.347460] ACPI: button: Power Button [PWRF] worker2 # [ 7.429472] mousedev: PS/2 mouse device common for all mice lb # [ 7.339588] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.449446] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 7.461778] rtc_cmos PNP0B00:00: registered as rtc0 worker2 # [ 7.461843] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T16:12:16 UTC (1790179936) worker2 # [ 7.461915] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker2 # [ 7.477095] Console: switching to colour dummy device 80x25 client # [ 7.566962] kvm_amd: TSC scaling supported client # [ 7.567385] kvm_amd: Nested Virtualization enabled client # [ 7.568130] kvm_amd: Nested Paging enabled client # [ 7.568508] kvm_amd: LBR virtualization supported worker1 # [ 7.388846] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 7.569232] kvm_amd: Virtual VMLOAD VMSAVE supported client # [ 7.569891] kvm_amd: Virtual GIF supported worker1 # [ 7.388913] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T16:12:15 UTC (1790179935) client # [ 7.570276] kvm_amd: Virtual NMI enabled lb # [ 7.419083] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 7.485117] parport_pc 00:02: reported by Plug and Play ACPI worker2 # [ 7.485209] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker1 # [ 7.388981] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker1 # [ 7.389707] systemd-journald[284]: Time jumped backwards, rotating. worker1 # [ 7.421997] parport_pc 00:02: reported by Plug and Play ACPI lb # [ 7.432223] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker2 # [ 7.497330] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized worker2 # [ 7.501285] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 client # [ 7.610467] EDAC MC: Ver: 3.0.0 worker1 # [ 7.422080] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker1 # [ 7.461368] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 worker2 # [ 7.624078] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker2 # [ 7.624189] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker2 # [ 7.629416] [drm] Found bochs VGA, ID 0xb0c5. worker2 # [ 7.629417] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. lb # [ 7.472907] (udev-worker)[354]: Network interface NamePolicy= disabled on kernel command line. lb # [ 7.476697] (udev-worker)[358]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 7.489716] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized lb # [ 7.480320] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 7.547483] Console: switching to colour dummy device 80x25 worker1 # [ 7.597475] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker2 # [ 7.648761] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker2 # [ 7.649458] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 lb # [ 7.513726] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.597612] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker1 # [ 7.668144] [drm] Found bochs VGA, ID 0xb0c5. worker1 # [ 7.668147] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 7.670263] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker1 # [ 7.671041] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 7.675600] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker1 # [ 7.676665] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker2 # [ 7.704180] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker2 # [ 7.705354] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 7.552570] systemd[1]: Mounted /run/wrappers. lb # [ 7.553373] systemd[1]: Reached target Local File Systems. lb # [ 7.556314] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 7.560139] systemd[1]: Starting register-nix-paths.service... worker1 # [ 7.565062] systemd[1]: Finished register-nix-paths.service. worker1 # [ 7.566302] systemd[1]: Reached target System Initialization. lb # [ 7.563476] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.565176] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.570460] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 7.573033] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 7.574161] systemd[1]: Reached target Timer Units. worker1 # [ 7.576122] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 7.575443] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 7.581441] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 7.582304] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 7.602586] systemd[1]: Finished register-nix-paths.service. worker2 # [ 7.605176] systemd[1]: Reached target System Initialization. worker1 # [ 7.586099] systemd[1]: Reached target Socket Units. worker2 # [ 7.606656] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 7.584859] systemd[1]: Starting Create System Files and Directories... worker1 # [ 7.590212] systemd[1]: Reached target Basic System. worker2 # [ 7.609198] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 7.611208] systemd[1]: Reached target Timer Units. worker2 # [ 7.614152] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 7.616039] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 7.597750] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 7.616816] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 7.598646] systemd[1]: Started backdoor.service. worker1 # [ 7.600256] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 7.619631] systemd[1]: Reached target Socket Units. worker2 # [ 7.621139] systemd[1]: Reached target Basic System. worker2 # [ 7.624732] systemd[1]: Started backdoor.service. worker1 # [ 7.606170] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 7.626283] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 7.634410] systemd[1]: Finished register-nix-paths.service. worker2 # [ 7.633107] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 7.614397] systemd[1]: Starting Post-Boot Actions... worker2 # [ 7.634053] systemd[1]: Starting Post-Boot Actions... worker2 # [ 7.634755] systemd[1]: Started Reset console on configuration changes. worker1 # [ 7.616242] systemd[1]: Started Reset console on configuration changes. worker1 # [ 7.620305] systemd[1]: Starting resolvconf update... client # [ 7.643716] systemd-vconsole-setup[420]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 7.642600] systemd[1]: Starting resolvconf update... client # [ 7.646322] systemd[1]: Finished Virtual Console Setup. client # [ 7.648116] systemd[1]: Reached target System Initialization. worker2 # [ 7.646068] systemd[1]: Starting D-Bus System Message Bus... client # [ 7.649319] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 7.627931] systemd[1]: Starting D-Bus System Message Bus... client # [ 7.650348] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 7.651520] systemd[1]: Reached target Timer Units. client # [ 7.652474] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 7.654215] systemd[1]: Listening on Nix Daemon Socket. client # [ 7.654963] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 7.656226] systemd[1]: Reached target Socket Units. client # [ 7.656933] systemd[1]: Reached target Basic System. client # [ 7.660040] systemd[1]: Started backdoor.service. client # [ 7.662102] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 7.666121] systemd[1]: Starting nftables firewall... client # [ 7.671343] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 7.678140] systemd[1]: Starting Post-Boot Actions... client # [ 7.686242] systemd[1]: Started Reset console on configuration changes. worker2 # [ 7.684057] systemd[1]: Starting Virtual Console Setup... lb # [ 7.661342] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 7.671232] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 7.697794] systemd[1]: Starting resolvconf update... worker1 # [ 7.702068] systemd[1]: Starting Virtual Console Setup... client # [ 7.726225] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 7.731789] systemd[1]: Finished Post-Boot Actions. client # connecting to host... lb # [ 7.726800] systemd[1]: Finished Create System Files and Directories. lb # [ 7.731406] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 7.732928] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.735146] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.736944] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.763302] nsncd[428]: Sep 23 16:12:16.628 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 7.767524] systemd[1]: Finished Post-Boot Actions. lb # [ 7.744138] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 7.768421] nsncd[512]: Sep 23 16:12:16.936 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 7.746077] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.770344] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 7.747530] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.748682] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.775168] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 7.775162] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 7.776548] systemd[1]: Reached target User and Group Name Lookups. lb # [ 7.754290] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # connecting to host... client # [ 7.786975] systemd[1]: Reached target Host and Network Name Lookups. client: Guest shell says: b'Spawning backdoor root shell...\n' client # [ 7.790469] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 7.773131] systemd[1]: Finished Post-Boot Actions. client # [ 7.805815] systemd[1]: Starting User Login Management... worker1 # [ 7.781388] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" client # [ 7.809103] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 7.810547] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 7.818332] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 7.821360] nix-grpc-daemon[512]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR client # [ 7.867817] dbus-broker-launch[436]: Looking up NSS user entry for 'systemd-timesync'... client: connected to guest root shell worker1 # [ 7.823963] nix-grpc-daemon[512]: E0000 00:00:1790179936.068043 512 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client: (connecting took 8.52 seconds) worker1 # connecting to host... client: (finished: waiting for the VM to finish booting, in 8.52 seconds) worker1 # [ 7.834519] nix-grpc-daemon[512]: E0000 00:00:1790179936.080900 512 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 7.839957] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 7.838930] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 7.841771] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 7.852028] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Could not connect to server" client # [ 7.878717] dbus-broker-launch[436]: NSS returned no entry for 'systemd-timesync' client # [ 7.879735] dbus-broker-launch[436]: Invalid user-name in /nix/store/c4mprcfibsx637q0hk2g869f3wvd2k8k-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 7.859615] nix-grpc-daemon[512]: E0000 00:00:1790179936.105779 570 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 7.871654] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 7.871273] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 7.876026] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 7.876898] systemd[1]: Reached target User and Group Name Lookups. lb # [ 7.878952] systemd[1]: Starting Update is Completed... worker2 # [ 8.009531] Console: switching to colour frame buffer device 160x50 worker2 # [ 8.038552] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker2 # [ 7.779928] systemd[1]: Starting User Login Management... worker2 # [ 7.906892] dbus-broker-launch[516]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 7.888154] systemd[1]: Starting User Login Management... worker1 # [ 7.889395] nsncd[515]: Sep 23 16:12:16.132 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 7.913702] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 7.913860] dbus-broker-launch[516]: NSS returned no entry for 'systemd-timesync' worker1 # [ 7.895170] nix-grpc-daemon[512]: E0000 00:00:1790179936.138899 570 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 7.916093] dbus-broker-launch[516]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 7.897204] nix-grpc-daemon[512]: E0000 00:00:1790179936.139122 570 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 7.918696] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 7.924357] systemd-logind[458]: New seat seat0. client # [ 7.930280] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 7.930120] systemd[1]: Started D-Bus System Message Bus. client # [ 7.933358] systemd[1]: Stopping Host and Network Name Lookups... client # [ 7.934188] systemd[1]: Stopped target User and Group Name Lookups. client # [ 7.935041] systemd[1]: Stopping User and Group Name Lookups... client # [ 7.935843] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 7.934982] dbus-broker-launch[516]: Ready worker1 # [ 7.915741] nix-grpc-daemon[512]: E0000 00:00:1790179936.162804 570 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. client # [ 7.940482] systemd-logind[458]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 7.939483] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 7.940550] systemd[1]: Stopped Virtual Console Setup. client # [ 7.945068] systemd-logind[458]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 7.946244] systemd-logind[458]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 7.945209] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.946072] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 7.925148] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" client # [ 7.949960] systemd[1]: Started User Login Management. worker2 # [ 7.946940] systemd[1]: Stopping Host and Network Name Lookups... client # [ 7.950742] systemd[1]: nscd.service: Deactivated successfully. client # [ 7.956733] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 7.948839] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 7.955765] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 7.938374] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 7.957708] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 7.962820] dbus-broker-launch[436]: Ready worker2 # [ 7.960198] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 7.963474] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 7.966388] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 8.078050] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker2 # [ 7.969091] systemd-logind[536]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 7.970837] systemd-logind[536]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 7.974245] systemd-logind[536]: New seat seat0. worker1 # [ 7.957487] dbus-broker-launch[519]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 7.977075] systemd[1]: Started User Login Management. worker2 # [ 7.977818] systemd[1]: Starting linger-users.service... client # [ 7.984298] systemd[1]: Starting linger-users.service... client # [ 7.987235] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 7.961985] systemd[1]: Finished Update is Completed. worker1 # [ 7.977668] dbus-broker-launch[519]: NSS returned no entry for 'systemd-timesync' worker1 # [ 7.979224] dbus-broker-launch[519]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 7.989844] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 8.024169] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 8.025476] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 8.028083] systemd[1]: Reached target User and Group Name Lookups. lb # [ 8.142160] mousedev: PS/2 mouse device common for all mice worker2 # [ 8.030727] nsncd[585]: Sep 23 16:12:17.199 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 8.150828] ACPI: button: Power Button [PWRF] worker2 # [ 8.041310] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 8.043183] systemd[1]: Finished linger-users.service. client # [ 8.053825] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 8.033401] dbus-broker-launch[519]: Ready client # [ 8.056491] systemd[1]: Finished linger-users.service. client # [ 8.062708] systemd[1]: Finished resolvconf update. worker2 # [ 8.061258] systemd[1]: Finished resolvconf update. worker2 # [ 8.062066] systemd[1]: Reached target Preparation for Network. worker2 # [ 8.067143] systemd[1]: Starting DHCP Client... client # [ 8.069262] nsncd[516]: Sep 23 16:12:16.934 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 8.071882] systemd[1]: Starting DHCP Client... client # [ 8.073427] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 8.073082] systemd[1]: Starting Address configuration of eth1... client # [ 8.076860] systemd[1]: Reached target Host and Network Name Lookups. client # [ 8.077764] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 8.077253] systemd[1]: Starting Extra networking commands.... lb # [ 8.192876] rtc_cmos PNP0B00:00: RTC can wake from S4 lb # [ 8.197512] parport_pc 00:02: reported by Plug and Play ACPI lb # [ 8.208773] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker2 # [ 8.249217] ppdev: user-space parallel port driver client # [ 8.171555] dhcpcd[553]: dhcpcd-10.3.2 starting client # [ 8.176751] dhcpcd[559]: dev: loaded udev client # [ 8.320998] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 8.279622] ppdev: user-space parallel port driver worker1 # [ 8.297260] Console: switching to colour frame buffer device 160x50 worker2 # [ 8.195775] network-addresses-eth1-start[616]: adding address 192.168.1.4/24... done worker2 # [ 8.210801] network-addresses-eth1-start[616]: adding address 2001:db8:1::4/64... done client # [ 8.229207] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 8.232317] systemd[1]: Finished Address configuration of eth1. worker2 # [ 8.385607] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 8.396659] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 8.378537] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker1 # [ 8.093540] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 8.245479] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 8.265592] systemd-vconsole-setup[581]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 8.409790] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 8.271168] systemd[1]: Finished Virtual Console Setup. client # [ 8.410473] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 8.412489] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 8.413379] cfg80211: failed to load regulatory.db worker1 # [ 8.252330] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" worker1 # [ 8.259149] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 8.264095] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 8.267555] nsncd[638]: Sep 23 16:12:16.401 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 8.290872] systemd-logind[536]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 8.277584] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 8.278569] systemd[1]: nscd.service: Deactivated successfully. lb # [ 8.278808] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 8.283276] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 8.284322] systemd-logind[591]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 8.439308] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker1 # [ 8.285414] systemd-logind[591]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker1 # [ 8.286671] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 8.282482] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 8.291980] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 8.292828] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 8.293677] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 8.294528] systemd[1]: Finished resolvconf update. lb # [ 8.234317] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 8.454838] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 8.234391] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T16:12:16 UTC (1790179936) client # [ 8.320474] dhcpcd[559]: eth0: waiting for carrier worker1 # [ 8.296917] systemd[1]: Reached target Preparation for Network. client # [ 8.321285] dhcpcd[559]: eth0: carrier acquired worker1 # [ 8.299251] systemd[1]: Starting DHCP Client... worker1 # [ 8.299865] systemd[1]: Starting Address configuration of eth1... worker1 # [ 8.300702] systemd[1]: Starting Extra networking commands.... worker1 # [ 8.301511] systemd-logind[591]: New seat seat0. worker1 # [ 8.302164] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 8.327964] dhcpcd[559]: DUID 00:01:00:01:32:46:b6:e1:52:54:00:12:34:56 worker1 # [ 8.306376] systemd[1]: Stopped Virtual Console Setup. client # [ 8.328911] dhcpcd[559]: eth0: IAID 00:12:34:56 client # [ 8.330075] dhcpcd[559]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 8.444959] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker1 # [ 8.309725] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.310502] systemd[1]: Started User Login Management. worker1 # [ 8.311221] systemd[1]: Starting linger-users.service... client # [ 8.334530] systemd[1]: Finished nftables firewall. worker2 # [ 8.331563] dhcpcd[626]: dhcpcd-10.3.2 starting client # [ 8.335684] systemd[1]: Reached target Preparation for Network. worker1 # [ 8.315131] network-addresses-eth1-start[665]: adding address 192.168.1.3/24... done client # [ 8.339134] systemd[1]: Starting Address configuration of eth1... worker2 # [ 8.338287] systemd[1]: Finished Extra networking commands.. client # [ 8.341669] systemd[1]: Starting Extra networking commands.... worker2 # [ 8.341199] systemd[1]: Reached target Network. lb # [ 8.234462] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 8.235424] systemd-journald[284]: Time jumped backwards, rotating. worker2 # [ 8.344164] systemd[1]: Starting Permit User Sessions... worker2 # [ 8.345775] dhcpcd[683]: dev: loaded udev worker2 # [ 8.352148] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 8.353164] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.334356] network-addresses-eth1-start[665]: adding address 2001:db8:1::3/64... done worker2 # [ 8.353866] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 8.358526] systemd[1]: Starting Virtual Console Setup... lb # [ 8.247441] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 8.251514] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] lb # [ 8.340980] systemd[1]: Finished Firewall. worker2 # [ 8.507308] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 8.507767] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 8.355378] systemd-logind[591]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 8.517106] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 8.343078] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized lb # [ 8.427203] Console: switching to colour dummy device 80x25 lb # [ 8.449455] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt lb # [ 8.358787] systemd[1]: Finished register-nix-paths.service. lb # [ 8.359675] systemd[1]: Reached target System Initialization. lb # [ 8.362118] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 8.363114] systemd[1]: Started niks3 garbage collection timer. worker1 # [ 8.367142] systemd[1]: Finished Address configuration of eth1. lb # [ 8.363880] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 8.364715] systemd[1]: Reached target Timer Units. lb # [ 8.365415] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 8.391969] network-addresses-eth1-start[585]: adding address 192.168.1.1/24... done lb # [ 8.370614] systemd[1]: Listening on niks3 server socket. lb # [ 8.371371] systemd[1]: Listening on Nix Daemon Socket. lb # [ 8.372084] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 8.373130] systemd[1]: Reached target Socket Units. lb # [ 8.373846] systemd[1]: Reached target Basic System. lb # [ 8.376069] systemd[1]: Started backdoor.service. client # [ 8.403608] network-addresses-eth1-start[585]: adding address 2001:db8:1::1/64... done lb # [ 8.378073] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 8.380166] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 8.384563] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 8.385489] systemd[1]: Finished linger-users.service. client # [ 8.414778] systemd[1]: Finished Address configuration of eth1. lb # [ 8.390257] systemd[1]: Starting Post-Boot Actions... worker2 # [ 8.419193] systemd[1]: Finished Permit User Sessions. worker1 # [ 8.545000] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) lb # [ 8.405271] systemd[1]: Started Reset console on configuration changes. worker2 # [ 8.429703] systemd[1]: Started Getty on tty1. client # [ 8.436340] systemd[1]: Finished Extra networking commands.. client # [ 8.438330] systemd[1]: Reached target Network. worker2 # [ 8.435353] systemd[1]: Reached target Login Prompts. client # [ 8.440334] systemd[1]: Starting Permit User Sessions... lb # [ 8.449553] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD lb # [ 8.537244] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 lb # [ 8.537521] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 8.421388] systemd[1]: Starting resolvconf update... worker1 # [ 8.426289] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.427390] systemd[1]: Stopped Virtual Console Setup. client # [ 8.455356] systemd[1]: Finished Permit User Sessions. worker1 # [ 8.433813] systemd[1]: Starting Virtual Console Setup... lb # [ 8.432094] systemd[1]: Started rustfs.service. client # [ 8.458324] systemd[1]: Started Getty on tty1. client # [ 8.459667] systemd[1]: Reached target Login Prompts. worker2 # [ 8.462204] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 8.456582] systemd[1]: Starting rustfs-bucket.service... worker1 # [ 8.491448] systemd[1]: Finished Extra networking commands.. worker1 # [ 8.492473] systemd[1]: Reached target Network. worker1 # [ 8.495222] systemd[1]: Starting Permit User Sessions... lb # connecting to host... worker1 # [ 8.500435] dhcpcd[676]: dhcpcd-10.3.2 starting lb # [ 8.501461] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 8.660912] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.638044] [drm] Found bochs VGA, ID 0xb0c5. lb # [ 8.638046] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 8.509333] dhcpcd[742]: dev: loaded udev lb # [ 8.515246] nsncd[523]: Sep 23 16:12:16.912 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 8.651077] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker1 # [ 8.657808] 8021q: 802.1Q VLAN Support v1.8 lb # [ 8.651749] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 8.658230] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 8.518441] systemd[1]: Finished Post-Boot Actions. lb # [ 8.519388] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 8.529302] nix-grpc-daemon[512]: ts=2026-09-23T16:12:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" worker2 # [ 8.687575] kvm_amd: TSC scaling supported worker2 # [ 8.688009] kvm_amd: Nested Virtualization enabled worker2 # [ 8.688493] kvm_amd: Nested Paging enabled worker2 # [ 8.689281] kvm_amd: LBR virtualization supported worker2 # [ 8.689959] kvm_amd: Virtual VMLOAD VMSAVE supported worker2 # [ 8.690456] kvm_amd: Virtual GIF supported worker2 # [ 8.691116] kvm_amd: Virtual NMI enabled worker2 # [ 8.706773] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 8.707469] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 8.709433] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 8.710260] cfg80211: failed to load regulatory.db worker1 # [ 8.562500] systemd[1]: Finished Permit User Sessions. worker1 # [ 8.567191] systemd[1]: Started Getty on tty1. worker1 # [ 8.567891] systemd[1]: Reached target Login Prompts. lb # [ 8.564556] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 8.568340] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 8.734957] EDAC MC: Ver: 3.0.0 lb # [ 8.589881] systemd[1]: Starting User Login Management... worker2 # [ 8.750852] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 8.616542] dhcpcd[683]: eth0: waiting for carrier worker2 # [ 8.617757] dhcpcd[683]: eth0: carrier acquired worker2 # [ 8.622775] dhcpcd[683]: DUID 00:01:00:01:32:46:b6:e1:52:54:00:12:34:56 worker2 # [ 8.623968] dhcpcd[683]: eth0: IAID 00:12:34:56 worker1 # [ 8.605184] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 8.624969] dhcpcd[683]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 8.613115] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 8.631399] systemd[1]: Starting Virtual Console Setup... worker2 # [ 8.672660] systemd-vconsole-setup[687]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 8.676216] systemd[1]: Finished Virtual Console Setup. worker1 # [ 8.802529] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 8.829896] kvm_amd: TSC scaling supported worker1 # [ 8.830306] kvm_amd: Nested Virtualization enabled worker1 # [ 8.831217] kvm_amd: Nested Paging enabled worker1 # [ 8.832011] kvm_amd: LBR virtualization supported worker1 # [ 8.832621] kvm_amd: Virtual VMLOAD VMSAVE supported worker1 # [ 8.833072] kvm_amd: Virtual GIF supported worker1 # [ 8.833750] kvm_amd: Virtual NMI enabled worker1 # [ 8.844025] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 8.844696] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 8.847837] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 8.848681] cfg80211: failed to load regulatory.db worker1 # [ 8.887091] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 8.889288] EDAC MC: Ver: 3.0.0 worker1 # [ 8.755149] dhcpcd[742]: eth0: waiting for carrier worker1 # [ 8.756340] dhcpcd[742]: libudev: received NULL device worker1 # [ 8.757795] dhcpcd[742]: libudev: received NULL device worker1 # [ 8.759955] dhcpcd[742]: eth0: carrier acquired lb # [ 8.757108] dbus-broker-launch[536]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 8.766578] dhcpcd[742]: DUID 00:01:00:01:32:46:b6:e1:52:54:00:12:34:56 worker1 # [ 8.767564] dhcpcd[742]: eth0: IAID 00:12:34:56 worker1 # [ 8.768679] dhcpcd[742]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 8.774139] dbus-broker-launch[536]: NSS returned no entry for 'systemd-timesync' lb # [ 8.775140] dbus-broker-launch[536]: Invalid user-name in /nix/store/xhp17752g31xx87ahxdgrix2v9kism59-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 8.796810] systemd-vconsole-setup[726]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.801505] systemd[1]: Finished Virtual Console Setup. lb # [ 8.811499] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 8.813230] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 8.814973] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 8.817181] systemd[1]: Stopping User and Group Name Lookups... lb # [ 8.817947] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 8.825690] systemd[1]: nscd.service: Deactivated successfully. lb # [ 8.828064] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 8.833611] systemd-logind[556]: Watching system buttons on /dev/input/event2 (Power Button) client # [ 8.913679] dhcpcd[559]: eth0: soliciting a DHCP lease client # [ 9.066954] NET: Registered PF_PACKET protocol family client # [ 8.938864] dhcpcd[559]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 8.941368] dhcpcd[559]: eth0: probing address 10.0.2.15/24 worker2 # [ 8.973886] dhcpcd[683]: eth0: soliciting a DHCP lease lb # [ 9.081068] ppdev: user-space parallel port driver worker2 # [ 9.126898] NET: Registered PF_PACKET protocol family worker2 # [ 8.997870] dhcpcd[683]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 9.000359] dhcpcd[683]: eth0: probing address 10.0.2.15/24 lb # [ 9.087545] Console: switching to colour frame buffer device 160x50 lb # [ 9.133004] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 8.836232] systemd-logind[556]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) lb # [ 9.002272] systemd[1]: Started D-Bus System Message Bus. lb # [ 9.004766] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.010122] nsncd[615]: Sep 23 16:12:17.327 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.013296] dbus-broker-launch[536]: Ready lb # [ 9.013892] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.014682] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.016401] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.017254] systemd[1]: Finished resolvconf update. lb # [ 9.017881] systemd[1]: Reached target Preparation for Network. lb # [ 9.018677] systemd[1]: Starting DHCP Client... lb # [ 9.019885] systemd[1]: Starting Address configuration of eth1... lb # [ 9.020997] systemd[1]: Starting Extra networking commands.... lb # [ 9.021777] systemd-logind[556]: New seat seat0. lb # [ 9.022449] systemd[1]: Started User Login Management. lb # [ 9.023837] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.026617] systemd-logind[556]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 9.029245] systemd[1]: Stopped Virtual Console Setup. lb # [ 9.044439] systemd[1]: Starting linger-users.service... lb # [ 9.051704] systemd[1]: Starting Virtual Console Setup... lb # [ 9.197184] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client: (finished: waiting for unit network-addresses-eth1.service, in 9.79 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 9.133073] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.136941] systemd[1]: Stopped Virtual Console Setup. client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.03 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 9.152152] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 9.155777] systemd[1]: Finished linger-users.service. lb # [ 9.297727] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) lb # [ 9.182936] network-addresses-eth1-start[642]: adding address 192.168.1.2/24... done lb # [ 9.189968] systemd[1]: Starting Virtual Console Setup... lb # [ 9.218721] network-addresses-eth1-start[642]: adding address 2001:db8:1::2/64... done lb # [ 9.272852] systemd[1]: Finished Address configuration of eth1. worker1 # [ 9.329769] nix-grpc-daemon[512]: ts=2026-09-23T16:12:17Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" lb # [ 9.412405] systemd[1]: Finished Extra networking commands.. lb # [ 9.416191] systemd[1]: Reached target Network. lb # [ 9.423306] dhcpcd[666]: dhcpcd-10.3.2 starting lb # [ 9.427286] systemd[1]: Starting PostgreSQL Server... lb # [ 9.432087] systemd[1]: Starting Permit User Sessions... lb # [ 9.446392] dhcpcd[727]: dev: loaded udev lb # [ 9.588687] kvm_amd: TSC scaling supported lb # [ 9.589088] kvm_amd: Nested Virtualization enabled lb # [ 9.592756] kvm_amd: Nested Paging enabled lb # [ 9.594886] kvm_amd: LBR virtualization supported lb # [ 9.598508] kvm_amd: Virtual VMLOAD VMSAVE supported lb # [ 9.598977] kvm_amd: Virtual GIF supported lb # [ 9.601434] kvm_amd: Virtual NMI enabled lb # [ 9.618972] 8021q: 802.1Q VLAN Support v1.8 lb # [ 9.619436] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 9.512323] systemd[1]: Finished Permit User Sessions. lb # [ 9.519264] systemd[1]: Started Getty on tty1. lb # [ 9.523796] systemd[1]: Reached target Login Prompts. lb # [ 9.566698] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.704746] EDAC MC: Ver: 3.0.0 lb # [ 9.776832] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 9.806050] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 9.806809] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 9.810503] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 9.811318] cfg80211: failed to load regulatory.db lb # [ 9.754344] postgresql-pre-start[740]: The files belonging to this database system will be owned by user "postgres". lb # [ 9.889966] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 9.756792] postgresql-pre-start[740]: This user must also own the server process. lb # [ 9.758626] dhcpcd[727]: eth0: waiting for carrier lb # [ 9.759941] dhcpcd[727]: eth0: carrier acquired lb # [ 9.765098] postgresql-pre-start[740]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 9.766287] postgresql-pre-start[740]: The default database encoding has accordingly been set to "UTF8". lb # [ 9.767393] postgresql-pre-start[740]: The default text search configuration will be set to "english". lb # [ 9.768468] postgresql-pre-start[740]: Data page checksums are enabled. lb # [ 9.769674] postgresql-pre-start[740]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 9.770794] postgresql-pre-start[740]: creating subdirectories ... ok lb # [ 9.771584] postgresql-pre-start[740]: selecting dynamic shared memory implementation ... posix lb # [ 9.776749] dhcpcd[727]: DUID 00:01:00:01:32:46:b6:e2:52:54:00:12:34:56 lb # [ 9.778129] dhcpcd[727]: eth0: IAID 00:12:34:56 lb # [ 9.778786] dhcpcd[727]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 9.846054] postgresql-pre-start[740]: selecting default "max_connections" ... 100 worker2 # [ 9.883248] dhcpcd[683]: eth0: soliciting an IPv6 router worker2 # [ 9.885417] dhcpcd[683]: eth0: Router Advertisement from fe80::2 worker2 # [ 9.887498] dhcpcd[683]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 9.866400] systemd-vconsole-setup[670]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 9.889659] dhcpcd[683]: eth0: adding route to fec0::/64 worker2 # [ 9.891506] dhcpcd[683]: eth0: adding default route via fe80::2 lb # [ 9.872376] systemd[1]: Finished Virtual Console Setup. lb # [ 9.904250] postgresql-pre-start[740]: selecting default "shared_buffers" ... 128MB client # [ 9.936307] dhcpcd[559]: eth0: soliciting an IPv6 router client # [ 9.939191] dhcpcd[559]: eth0: Router Advertisement from fe80::2 client # [ 9.941359] dhcpcd[559]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 9.943541] dhcpcd[559]: eth0: adding route to fec0::/64 client # [ 9.945436] dhcpcd[559]: eth0: adding default route via fe80::2 worker1 # [ 10.207645] dhcpcd[742]: eth0: soliciting a DHCP lease worker1 # [ 10.361285] NET: Registered PF_PACKET protocol family worker1 # [ 10.230677] dhcpcd[742]: eth0: soliciting an IPv6 router worker1 # [ 10.233812] dhcpcd[742]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 10.236285] dhcpcd[742]: eth0: Router Advertisement from fe80::2 worker1 # [ 10.239273] dhcpcd[742]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 10.242309] dhcpcd[742]: eth0: adding route to fec0::/64 worker1 # [ 10.244527] dhcpcd[742]: eth0: adding default route via fe80::2 worker1 # [ 10.246513] dhcpcd[742]: eth0: probing address 10.0.2.15/24 lb: (finished: waiting for unit network-addresses-eth1.service, in 1.16 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.03 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.04 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.02 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service lb # [ 10.481840] postgresql-pre-start[740]: selecting default time zone ... UTC lb # [ 10.486425] postgresql-pre-start[740]: creating configuration files ... ok lb # [ 10.671791] postgresql-pre-start[740]: running bootstrap script ... ok worker1 # [ 10.930504] nix-grpc-daemon[512]: ts=2026-09-23T16:12:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 11.039468] postgresql-pre-start[740]: performing post-bootstrap initialization ... ok lb # [ 11.057661] dhcpcd[727]: eth0: soliciting a DHCP lease lb # [ 11.210435] NET: Registered PF_PACKET protocol family lb # [ 11.081528] dhcpcd[727]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.084202] dhcpcd[727]: eth0: probing address 10.0.2.15/24 lb # [ 11.196340] postgresql-pre-start[740]: syncing data to disk ... ok lb # [ 11.197360] postgresql-pre-start[740]: initdb: warning: enabling "trust" authentication for local connections lb # [ 11.199245] postgresql-pre-start[740]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 11.201097] postgresql-pre-start[740]: Success. You can now start the database server using: lb # [ 11.202066] postgresql-pre-start[740]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 11.273482] postgres[785]: [785] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 11.275918] postgres[785]: [785] LOG: listening on IPv6 address "::1", port 5432 lb # [ 11.276982] postgres[785]: [785] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 11.281884] postgres[785]: [785] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 11.292237] postgres[794]: [794] LOG: database system was shut down at 2026-09-23 16:12:19 GMT lb # [ 11.299088] postgres[785]: [785] LOG: database system is ready to accept connections lb # [ 11.303248] systemd[1]: Started PostgreSQL Server. lb # [ 11.309079] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 11.427330] postgresql-setup-start[805]: CREATE DATABASE lb # [ 11.452857] postgresql-setup-start[810]: CREATE ROLE lb # [ 11.464834] postgresql-setup-start[812]: ALTER DATABASE lb # [ 11.469097] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 11.470841] systemd[1]: Reached target PostgreSQL. lb # [ 11.943700] dhcpcd[727]: eth0: soliciting an IPv6 router lb # [ 11.946164] dhcpcd[727]: eth0: Router Advertisement from fe80::2 lb # [ 11.948507] dhcpcd[727]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 11.950649] dhcpcd[727]: eth0: adding route to fec0::/64 lb # [ 11.952451] dhcpcd[727]: eth0: adding default route via fe80::2 worker2 # [ 13.470674] dhcpcd[683]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 13.473541] dhcpcd[683]: eth0: adding route to 10.0.2.0/24 worker2 # [ 13.476633] dhcpcd[683]: eth0: adding default route via 10.0.2.2 worker2 # [ 13.555573] systemd[1]: Started DHCP Client. worker2 # [ 13.558212] systemd[1]: Reached target Network is Online. worker2 # [ 13.559987] systemd[1]: Starting scheduler-token.service... client # [ 13.563888] dhcpcd[559]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 13.566897] dhcpcd[559]: eth0: adding route to 10.0.2.0/24 client # [ 13.569559] dhcpcd[559]: eth0: adding default route via 10.0.2.2 worker2 # [ 13.593177] systemd[1]: scheduler-token.service: Main process exited, code=exited, status=7/NOTRUNNING worker2 # [ 13.594733] systemd[1]: scheduler-token.service: Failed with result 'exit-code'. worker2 # [ 13.596705] systemd[1]: Failed to start scheduler-token.service. worker2 # [ 13.598060] systemd[1]: Dependency failed for Nix worker-protocol over gRPC. worker2 # [ 13.599499] systemd[1]: nix-grpc-daemon.service: Job nix-grpc-daemon.service/start failed with result 'dependency'. worker2 # [ 13.601861] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 36ms wall clock time, 3.1M memory peak, 40B incoming IP traffic, 60B outgoing IP traffic. worker2 # [ 13.603703] systemd[1]: Reached target Multi-User System. worker2 # [ 13.604486] systemd[1]: Startup finished in 861ms (kernel) + 3.584s (initrd) + 9.155s (userspace) = 13.601s. client # [ 13.646331] systemd[1]: Started DHCP Client. client # [ 13.648598] systemd[1]: Reached target Multi-User System. client # [ 13.649684] systemd[1]: Startup finished in 855ms (kernel) + 3.580s (initrd) + 9.213s (userspace) = 13.649s. worker1 # [ 14.131221] nix-grpc-daemon[512]: ts=2026-09-23T16:12:22Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 14.845454] systemd[1]: scheduler-token.service: Scheduled restart job, restart counter is at 1. worker2 # [ 14.854111] systemd[1]: Starting scheduler-token.service... worker2 # [ 14.905245] systemd[1]: scheduler-token.service: Main process exited, code=exited, status=7/NOTRUNNING worker2 # [ 14.906887] systemd[1]: scheduler-token.service: Failed with result 'exit-code'. worker2 # [ 14.909309] systemd[1]: Failed to start scheduler-token.service. worker2 # [ 14.911749] systemd[1]: scheduler-token.service: Consumed 33ms CPU time over 1.310s wall clock time, 2.1M memory peak, 40B incoming IP traffic, 60B outgoing IP traffic. worker1 # [ 15.497627] dhcpcd[742]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 15.500587] dhcpcd[742]: eth0: adding route to 10.0.2.0/24 worker1 # [ 15.503271] dhcpcd[742]: eth0: adding default route via 10.0.2.2 worker1 # [ 15.586458] systemd[1]: Started DHCP Client. worker1 # [ 15.588224] systemd[1]: Reached target Multi-User System. worker1 # [ 15.589165] systemd[1]: Startup finished in 842ms (kernel) + 3.598s (initrd) + 11.147s (userspace) = 15.589s. worker2 # [ 16.157120] systemd[1]: scheduler-token.service: Scheduled restart job, restart counter is at 2. worker2 # [ 16.165479] systemd[1]: Starting scheduler-token.service... lb # [ 16.143110] dhcpcd[727]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 16.146325] dhcpcd[727]: eth0: adding route to 10.0.2.0/24 lb # [ 16.148923] dhcpcd[727]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.221214] systemd[1]: scheduler-token.service: Main process exited, code=exited, status=7/NOTRUNNING worker2 # [ 16.224158] systemd[1]: scheduler-token.service: Failed with result 'exit-code'. worker2 # [ 16.227860] systemd[1]: Failed to start scheduler-token.service. worker2 # [ 16.230422] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 1.315s wall clock time, 2.1M memory peak, 40B incoming IP traffic, 60B outgoing IP traffic. lb # [ 16.230103] systemd[1]: Started DHCP Client. lb # [ 16.232284] systemd[1]: Reached target Network is Online. lb # [ 16.235043] systemd[1]: Started Envoy reverse proxy. lb # [ 16.240302] systemd[1]: Started mock-oidc.service. lb # [ 16.375352] mock-oidc-server[922]: Mock OIDC Server running lb # [ 16.377084] mock-oidc-server[922]: OIDC Address: 192.168.1.2:8080 lb # [ 16.378127] mock-oidc-server[922]: Issue Address: 0.0.0.0:8081 lb # [ 16.379083] mock-oidc-server[922]: Issuer: http://192.168.1.2:8080/oidc lb # [ 16.379918] mock-oidc-server[922]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 16.381109] mock-oidc-server[922]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 16.382693] mock-oidc-server[922]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 16.509731] envoy[921]: [2026-09-23 16:12:24.909][921][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104) lb # [ 16.512369] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 16.513751] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 16.515673] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 16.517627] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 16.527194] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 16.528906] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 16.531108] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 16.534746] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 16.538785] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 16.540369] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 16.543041] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 16.545116] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 16.546656] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 16.548345] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 16.550750] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 16.552943] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 16.555850] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 16.557470] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 16.561125] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 16.562752] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 16.564229] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 16.567999] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 16.571509] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 16.573551] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 16.577419] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 16.579131] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 16.582193] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 16.584341] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 16.586427] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 16.587837] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 16.589392] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 16.593805] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 16.595684] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 16.599950] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 16.601719] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 16.603725] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 16.606285] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 16.611505] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 16.612976] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 16.615583] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 16.617917] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 16.621089] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 16.645319] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 16.646920] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 16.648712] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 16.651758] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 16.662217] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 16.664024] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 16.666732] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 16.670468] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 16.672112] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 16.674129] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 16.676439] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 16.678131] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 16.680637] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 16.682611] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 16.684853] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 16.686216] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 16.688821] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 16.691158] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 16.692697] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 16.694186] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.696018] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 16.698792] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 16.700249] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 16.702302] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 16.704628] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 16.707346] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.709584] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 16.715235] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 16.716794] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 16.718180] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 16.720434] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 16.727098] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 16.729056] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 16.730718] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 16.732841] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 16.734424] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 16.739414] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 16.745200] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 16.748142] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 16.749872] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 16.751885] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.753584] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 16.756305] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 16.764902] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 16.767053] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 16.769079] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 16.770530] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 16.772531] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 16.774249] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 16.776481] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 16.780816] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 16.782231] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 16.783796] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 16.787098] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 16.793631] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 16.796218] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 16.797735] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.799874] envoy[921]: [2026-09-23 16:12:24.911][921][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 16.802640] envoy[921]: [2026-09-23 16:12:24.912][921][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 16.804298] envoy[921]: [2026-09-23 16:12:24.912][921][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 16.808085] envoy[921]: [2026-09-23 16:12:24.912][921][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 16.809680] envoy[921]: [2026-09-23 16:12:24.912][921][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 16.811362] envoy[921]: [2026-09-23 16:12:24.912][921][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 16.813146] envoy[921]: [2026-09-23 16:12:24.960][921][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 16.814354] envoy[921]: [2026-09-23 16:12:25.003][921][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 16.826569] envoy[921]: [2026-09-23 16:12:25.003][921][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 16.827936] envoy[921]: [2026-09-23 16:12:25.003][921][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 16.834991] envoy[921]: [2026-09-23 16:12:25.003][921][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 16.841712] envoy[921]: [2026-09-23 16:12:25.241][921][info][main] [source/server/server.cc:968] runtime: {} lb # [ 16.844060] envoy[921]: [2026-09-23 16:12:25.243][921][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 16.845754] envoy[921]: [2026-09-23 16:12:25.245][921][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 16.847292] envoy[921]: [2026-09-23 16:12:25.246][921][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 16.849216] envoy[921]: [2026-09-23 16:12:25.248][921][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 16.850955] envoy[921]: [2026-09-23 16:12:25.250][921][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 16.852327] envoy[921]: [2026-09-23 16:12:25.250][921][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 16.865777] envoy[921]: [2026-09-23 16:12:25.265][921][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 16.869750] envoy[921]: [2026-09-23 16:12:25.269][921][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 16.871752] envoy[921]: [2026-09-23 16:12:25.271][921][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 16.875105] envoy[921]: [2026-09-23 16:12:25.271][921][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 16.876413] envoy[921]: [2026-09-23 16:12:25.274][921][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 16.877832] envoy[921]: [2026-09-23 16:12:25.274][921][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker2 # [ 17.473141] systemd[1]: scheduler-token.service: Scheduled restart job, restart counter is at 3. worker2 # [ 17.481612] systemd[1]: Starting scheduler-token.service... worker2 # [ 17.540579] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 17.544117] systemd[1]: Finished scheduler-token.service. worker2 # [ 17.546542] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 1.308s wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 17.673908] rustfs-bucket-start[945]: mb s3://farm lb # [ 17.685754] systemd[1]: Finished rustfs-bucket.service. lb # [ 17.696512] systemd[1]: Starting niks3 server... lb # [ 17.809561] postgres[957]: [957] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 17.810710] postgres[957]: [957] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 17.832550] niks3-server[952]: 2026/09/23 16:12:26 OK 20241026095416_initial_model.sql (12.92ms) lb # [ 17.835986] niks3-server[952]: 2026/09/23 16:12:26 OK 20251210153512_drop_unused_gin_index.sql (3.6ms) lb # [ 17.840795] niks3-server[952]: 2026/09/23 16:12:26 OK 20251218171726_add_pins.sql (4.8ms) lb # [ 17.845026] niks3-server[952]: 2026/09/23 16:12:26 OK 20260628120000_add_object_size_and_stats.sql (4.25ms) lb # [ 17.849658] niks3-server[952]: 2026/09/23 16:12:26 OK 20260905000000_add_claims.sql (3.29ms) lb # [ 17.853371] niks3-server[952]: 2026/09/23 16:12:26 OK 20260920000000_drop_claims.sql (3.77ms) lb # [ 17.856643] niks3-server[952]: 2026/09/23 16:12:26 OK 20260923120000_add_pushes.sql (1.89ms) lb # [ 17.857683] niks3-server[952]: 2026/09/23 16:12:26 goose: successfully migrated database to version: 20260923120000 lb # [ 17.861666] niks3-server[952]: 2026/09/23 16:12:26 OK 1_commit_pending_closure.sql (4.99ms) lb # [ 17.864664] niks3-server[952]: 2026/09/23 16:12:26 OK 2_object_stats_trigger.sql (2.95ms) lb # [ 17.867429] niks3-server[952]: 2026/09/23 16:12:26 OK 3_commit_push.sql (2.7ms) lb # [ 17.868393] niks3-server[952]: 2026/09/23 16:12:26 goose: up to current file version: 3 lb # [ 17.871893] niks3-server[952]: 2026/09/23 16:12:26 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 17.900250] niks3-server[952]: 2026/09/23 16:12:26 INFO Read proxy enabled — serving cache objects from S3 lb # [ 17.902274] niks3-server[952]: 2026/09/23 16:12:26 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 17.905198] systemd[1]: Started niks3 server. lb # [ 17.906174] niks3-server[952]: 2026/09/23 16:12:26 INFO systemd watchdog enabled interval=15s lb # [ 17.907529] niks3-server[952]: 2026/09/23 16:12:26 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 17.909729] systemd[1]: Reached target Multi-User System. lb # [ 17.911111] systemd[1]: Startup finished in 845ms (kernel) + 4.551s (initrd) + 12.513s (userspace) = 17.910s. lb: (finished: waiting for unit niks3.service, in 7.51 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.02 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.02 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 19.131844] nix-grpc-daemon[512]: ts=2026-09-23T16:12:27Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 24.132458] nix-grpc-daemon[512]: ts=2026-09-23T16:12:32Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 27.919744] nix-grpc-daemon[512]: ts=2026-09-23T16:12:36Z level=info event=scheduler_take_over lb # [ 27.915658] niks3-server[952]: 2026/09/23 16:12:36 INFO lead: acquired remote=192.168.1.3:41386 worker1 # [ 29.133193] nix-grpc-daemon[512]: ts=2026-09-23T16:12:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 31.873178] envoy[921]: [2026-09-23 16:12:40.271][921][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 31.879568] envoy[921]: [2026-09-23 16:12:40.271][921][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 31.883707] envoy[921]: [2026-09-23 16:12:40.271][921][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 31.889432] envoy[921]: [2026-09-23 16:12:40.289][921][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 32.538302] envoy[921]: conn peer=[::1]:44334 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=5 lb: (finished: waiting for TCP port 50051 on localhost, in 14.51 seconds) worker2 # [ 32.660606] systemd[1]: Starting scheduler-token.service... worker2 # [ 32.708617] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 32.711386] systemd[1]: Finished scheduler-token.service. worker2 # [ 32.712379] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 50ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 32.716074] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 32.810424] nix-grpc-daemon[802]: ts=2026-09-23T16:12:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 32.820292] nix-grpc-daemon[802]: ts=2026-09-23T16:12:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 32.822420] nix-grpc-daemon[802]: ts=2026-09-23T16:12:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 32.836056] nix-grpc-daemon[802]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 32.837631] nix-grpc-daemon[802]: E0000 00:00:1790179962.009454 802 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 32.843649] nix-grpc-daemon[802]: E0000 00:00:1790179962.017212 802 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 32.847077] nix-grpc-daemon[802]: ts=2026-09-23T16:12:42Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 32.848797] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 32.860054] nix-grpc-daemon[802]: E0000 00:00:1790179962.033306 844 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 32.873062] nix-grpc-daemon[802]: E0000 00:00:1790179962.046252 844 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 32.875167] nix-grpc-daemon[802]: E0000 00:00:1790179962.046343 844 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 32.886603] nix-grpc-daemon[802]: E0000 00:00:1790179962.060293 844 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 32.903678] nix-grpc-daemon[802]: ts=2026-09-23T16:12:42Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 32.903584] nix-grpc-daemon[512]: ts=2026-09-23T16:12:41Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 32.907231] nix-grpc-daemon[512]: ts=2026-09-23T16:12:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:41838 worker1 # [ 32.911188] nix-grpc-daemon[512]: ts=2026-09-23T16:12:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 34.134665] nix-grpc-daemon[512]: ts=2026-09-23T16:12:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.139255] nix-grpc-daemon[512]: ts=2026-09-23T16:12:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:41838 worker1 # [ 34.143240] nix-grpc-daemon[512]: ts=2026-09-23T16:12:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 37.863789] nix-grpc-daemon[512]: E0000 00:00:1790179966.109327 568 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 37.892133] nix-grpc-daemon[512]: E0000 00:00:1790179966.139356 568 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 37.902134] nix-grpc-daemon[512]: E0000 00:00:1790179966.144887 568 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 37.924061] nix-grpc-daemon[512]: E0000 00:00:1790179966.171261 568 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 39.551766] nix-grpc-daemon[512]: ts=2026-09-23T16:12:47Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:41850 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 7.40 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.49 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 40.939765] nix-grpc-daemon[802]: ts=2026-09-23T16:12:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 40.953401] systemd[1]: Started Nix Daemon. worker2 # [ 41.068371] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 41.083701] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5) worker2 # [ 41.127261] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5) worker2 # [ 41.164214] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 3/5) worker2 # [ 41.273455] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 398 ms (attempt 4/5) worker2 # [ 41.674416] nix-grpc-daemon[802]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 41.683542] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 41.708750] nix-grpc-daemon[802]: ts=2026-09-23T16:12:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 41.718623] nix-grpc-daemon[802]: ts=2026-09-23T16:12:50Z level=debug event=path_info_miss path=nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test error="error: path '/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 41.716853] nix-grpc-daemon[512]: ts=2026-09-23T16:12:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 41.728465] systemd[1]: Started Nix Daemon. worker1 # [ 41.836130] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 41.849694] nix-grpc-daemon[512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5) worker1 # [ 41.949100] nix-grpc-daemon[512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5) worker1 # [ 42.013166] nix-grpc-daemon[512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 322 ms (attempt 3/5) worker1 # [ 42.337123] nix-grpc-daemon[512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 4/5) worker1 # [ 42.530319] nix-grpc-daemon[512]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 42.533545] nix-grpc-daemon[512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 42.545746] nix-grpc-daemon[512]: ts=2026-09-23T16:12:50Z level=debug event=path_info_miss path=nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test error="error: path '/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.06 seconds) client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds) worker1 # [ 42.783885] nix-grpc-daemon[512]: ts=2026-09-23T16:12:51Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:41870 worker2 # [ 42.826803] nix-grpc-daemon[802]: ts=2026-09-23T16:12:51Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 42.832079] nix-grpc-daemon[802]: ts=2026-09-23T16:12:51Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:42756 worker2 # [ 42.840603] nix-grpc-daemon[802]: ts=2026-09-23T16:12:52Z level=debug event=path_info_miss path=nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test error="error: path '/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 42.826976] nix-grpc-daemon[512]: ts=2026-09-23T16:12:51Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:41870 worker1 # [ 42.836737] nix-grpc-daemon[512]: ts=2026-09-23T16:12:51Z level=debug event=path_info_miss path=nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test error="error: path '/nix/store/nf1wlrlyhicscmwy500v7v7lpchfyxh3-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 45.031426] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=info event=denied method=QueryMissing cn=- role=none worker1 # [ 45.148677] nix-grpc-daemon[512]: ts=2026-09-23T16:12:53Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.44 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker1 # [ 45.285661] nix-grpc-daemon[512]: ts=2026-09-23T16:12:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker1 # [ 45.295364] nix-grpc-daemon[512]: ts=2026-09-23T16:12:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 45.324248] nix-grpc-daemon[512]: ts=2026-09-23T16:12:53Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=0 worker1 # [ 45.328757] nix-grpc-daemon[512]: ts=2026-09-23T16:12:53Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 45.358193] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 45.370347] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying 1 paths to 'lb:50051'... worker2 # [ 45.382206] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'... worker2 # [ 45.401344] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 45.430843] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=info event=niks3_push_started pid=872 worker2 # [ 45.583183] nix-grpc-daemon[872]: time=2026-09-23T16:12:54.756Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 45.673315] nix-daemon[860]: accepted connection from pid 884, user nix-grpc-daemon (trusted) worker2 # [ 45.686436] nix-daemon[860]: reaped child process 887, status = succeeded lb # [ 45.673297] niks3-server[952]: 2026/09/23 16:12:54 INFO Received push request method=POST path=/api/pushes worker2 # [ 45.709325] nix-grpc-daemon[872]: time=2026-09-23T16:12:54.883Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 45.710841] nix-grpc-daemon[872]: time=2026-09-23T16:12:54.884Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)" lb # [ 45.720166] niks3-server[952]: 2026/09/23 16:12:54 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign lb # [ 45.732490] niks3-server[952]: 2026/09/23 16:12:54 INFO Signed narinfos id=1 count=1 worker2 # [ 45.758025] nix-grpc-daemon[872]: time=2026-09-23T16:12:54.931Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.745830] niks3-server[952]: 2026/09/23 16:12:54 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst lb # [ 45.749886] niks3-server[952]: 2026/09/23 16:12:54 INFO Received complete push request method=POST path=/api/pushes/1/complete lb # [ 45.762833] niks3-server[952]: 2026/09/23 16:12:54 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo lb # [ 45.766730] niks3-server[952]: 2026/09/23 16:12:54 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls worker2 # [ 45.799791] nix-grpc-daemon[872]: time=2026-09-23T16:12:54.973Z level=INFO msg="Upload complete. (212ms)" worker2 # [ 45.804917] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 45.811353] nix-grpc-daemon[802]: ts=2026-09-23T16:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 45.822251] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 45.832647] nix-daemon[860]: reaped child process 870, status = succeeded client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051... worker2 # [ 45.840952] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 46.677582] nix-daemon[860]: reaped child process 891, status = succeeded worker2 # [ 46.760884] nix-daemon[860]: accepted connection from pid 908, user nix-grpc-daemon (trusted) worker2 # [ 46.773044] nix-daemon[860]: reaped child process 911, status = succeeded lb # [ 46.756217] niks3-server[952]: 2026/09/23 16:12:55 INFO Received push request method=POST path=/api/pushes worker2 # [ 46.789679] nix-grpc-daemon[872]: time=2026-09-23T16:12:55.963Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 46.791277] nix-grpc-daemon[872]: time=2026-09-23T16:12:55.964Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)" lb # [ 46.802943] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv lb # [ 46.810512] niks3-server[952]: 2026/09/23 16:12:55 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign lb # [ 46.820421] niks3-server[952]: 2026/09/23 16:12:55 INFO Signed narinfos id=2 count=1 worker2 # [ 46.843867] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.017Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.824269] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 46.828894] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls lb # [ 46.838727] niks3-server[952]: 2026/09/23 16:12:55 INFO Received complete push request method=POST path=/api/pushes/2/complete lb # [ 46.846568] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo worker2 # [ 46.868219] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.041Z level=INFO msg="Upload complete. (219ms)" worker2 # [ 46.874884] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=1 worker1 # [ 46.864908] nix-grpc-daemon[512]: ts=2026-09-23T16:12:55Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=0 worker2 # [ 46.889513] nix-daemon[860]: reaped child process 889, status = succeeded worker1 # [ 46.869908] nix-grpc-daemon[512]: ts=2026-09-23T16:12:55Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=1 lb # [ 46.869238] envoy[921]: conn peer=[2001:db8:1::1]:53022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=1664 lb # [ 46.870697] envoy[921]: conn peer=[2001:db8:1::1]:42370 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=63 worker2 # [ 46.892621] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 lb # [ 46.872054] envoy[921]: conn peer=[2001:db8:1::1]:42384 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=8 lb # [ 46.873423] envoy[921]: conn peer=[2001:db8:1::1]:42398 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=9 worker2 # [ 46.902630] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 46.911198] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.084Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 46.918421] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'... worker2 # [ 46.940080] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 47.031579] nix-daemon[860]: accepted connection from pid 915, user nix-grpc-daemon (trusted) worker2 # [ 47.046906] nix-daemon[860]: reaped child process 918, status = succeeded lb # [ 47.027758] niks3-server[952]: 2026/09/23 16:12:55 INFO Received push request method=POST path=/api/pushes worker2 # [ 47.059189] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.232Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 47.060704] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.234Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)" lb # [ 47.060639] niks3-server[952]: 2026/09/23 16:12:55 INFO Received sign narinfos request method=POST path=/api/pushes/3/sign worker2 # [ 47.090148] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.263Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.066571] niks3-server[952]: 2026/09/23 16:12:55 INFO Signed narinfos id=3 count=1 lb # [ 47.074134] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls lb # [ 47.083656] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst lb # [ 47.091460] niks3-server[952]: 2026/09/23 16:12:55 INFO Received complete push request method=POST path=/api/pushes/3/complete lb # [ 47.101086] niks3-server[952]: 2026/09/23 16:12:55 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo worker2 # [ 47.123214] nix-grpc-daemon[872]: time=2026-09-23T16:12:56.296Z level=INFO msg="Upload complete. (173ms)" worker2 # [ 47.127296] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 47.133201] nix-grpc-daemon[802]: ts=2026-09-23T16:12:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 47.141738] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 47.149105] nix-daemon[860]: reaped child process 913, status = succeeded worker2 # [ 47.153247] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051... client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 48.019920] nix-daemon[860]: reaped child process 922, status = succeeded worker2 # [ 48.065505] nix-daemon[860]: accepted connection from pid 938, user nix-grpc-daemon (trusted) worker2 # [ 48.076415] nix-daemon[860]: reaped child process 941, status = succeeded lb # [ 48.061937] niks3-server[952]: 2026/09/23 16:12:56 INFO Received push request method=POST path=/api/pushes worker2 # [ 48.093610] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.267Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 48.095127] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.268Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)" lb # [ 48.103146] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls lb # [ 48.110483] niks3-server[952]: 2026/09/23 16:12:56 INFO Received sign narinfos request method=POST path=/api/pushes/4/sign worker2 # [ 48.139776] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.313Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.116539] niks3-server[952]: 2026/09/23 16:12:56 INFO Signed narinfos id=4 count=1 lb # [ 48.124143] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst lb # [ 48.131275] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv lb # [ 48.140274] niks3-server[952]: 2026/09/23 16:12:56 INFO Received complete push request method=POST path=/api/pushes/4/complete worker2 # [ 48.169455] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.343Z level=INFO msg="Upload complete. (209ms)" lb # [ 48.147879] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo worker2 # [ 48.175703] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=1 worker1 # [ 48.164672] nix-grpc-daemon[512]: ts=2026-09-23T16:12:56Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=0 worker1 # [ 48.168759] nix-grpc-daemon[512]: ts=2026-09-23T16:12:56Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 48.191692] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 48.195359] nix-daemon[860]: reaped child process 920, status = succeeded worker2 # [ 48.202514] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 48.212590] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.386Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 48.220740] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'... worker2 # [ 48.239209] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 48.333271] nix-daemon[860]: accepted connection from pid 945, user nix-grpc-daemon (trusted) worker2 # [ 48.344502] nix-daemon[860]: reaped child process 948, status = succeeded lb # [ 48.329317] niks3-server[952]: 2026/09/23 16:12:56 INFO Received push request method=POST path=/api/pushes worker2 # [ 48.360289] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.534Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 48.361762] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.535Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)" lb # [ 48.364658] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst lb # [ 48.373874] niks3-server[952]: 2026/09/23 16:12:56 INFO Received sign narinfos request method=POST path=/api/pushes/5/sign lb # [ 48.380533] niks3-server[952]: 2026/09/23 16:12:56 INFO Signed narinfos id=5 count=1 worker2 # [ 48.406839] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.580Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.388915] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls lb # [ 48.399949] niks3-server[952]: 2026/09/23 16:12:56 INFO Received complete push request method=POST path=/api/pushes/5/complete lb # [ 48.406904] niks3-server[952]: 2026/09/23 16:12:56 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo worker2 # [ 48.432828] nix-grpc-daemon[872]: time=2026-09-23T16:12:57.606Z level=INFO msg="Upload complete. (182ms)" worker2 # [ 48.437791] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 48.443718] nix-grpc-daemon[802]: ts=2026-09-23T16:12:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 48.456111] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 48.463313] nix-daemon[860]: reaped child process 943, status = succeeded worker2 # [ 48.470808] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051... client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 49.103469] nix-daemon[860]: reaped child process 952, status = succeeded worker2 # [ 49.187569] nix-daemon[860]: accepted connection from pid 968, user nix-grpc-daemon (trusted) worker2 # [ 49.199299] nix-daemon[860]: reaped child process 971, status = succeeded lb # [ 49.185379] niks3-server[952]: 2026/09/23 16:12:57 INFO Received push request method=POST path=/api/pushes worker2 # [ 49.216142] nix-grpc-daemon[872]: time=2026-09-23T16:12:58.389Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 49.219147] nix-grpc-daemon[872]: time=2026-09-23T16:12:58.391Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)" lb # [ 49.221898] niks3-server[952]: 2026/09/23 16:12:57 INFO Received sign narinfos request method=POST path=/api/pushes/6/sign lb # [ 49.230225] niks3-server[952]: 2026/09/23 16:12:57 INFO Signed narinfos id=6 count=1 worker2 # [ 49.256302] nix-grpc-daemon[872]: time=2026-09-23T16:12:58.429Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.239192] niks3-server[952]: 2026/09/23 16:12:57 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst lb # [ 49.245249] niks3-server[952]: 2026/09/23 16:12:57 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv lb # [ 49.251539] niks3-server[952]: 2026/09/23 16:12:57 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls lb # [ 49.261516] niks3-server[952]: 2026/09/23 16:12:57 INFO Received complete push request method=POST path=/api/pushes/6/complete lb # [ 49.267453] niks3-server[952]: 2026/09/23 16:12:57 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo worker2 # [ 49.294091] nix-grpc-daemon[872]: time=2026-09-23T16:12:58.467Z level=INFO msg="Upload complete. (214ms)" worker2 # [ 49.299610] nix-grpc-daemon[802]: ts=2026-09-23T16:12:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=1 worker2 # [ 49.314058] nix-daemon[860]: reaped child process 950, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 4.15 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds) client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client # copying 3 paths... client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 4.50 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds) worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker2 # [ 50.155238] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42756 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # [ 50.171093] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 50.180413] nix-grpc-daemon[872]: time=2026-09-23T16:12:59.354Z level=INFO msg="All 2 paths already cached" worker1 # [ 50.177334] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 0.80 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2 worker1 # [ 50.684969] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 client # don't know how to build these paths: client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker1 # [ 50.693593] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 50.714202] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=0 worker2 # [ 50.736792] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 50.718469] nix-grpc-daemon[512]: ts=2026-09-23T16:12:58Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 50.746979] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 50.760580] nix-grpc-daemon[872]: time=2026-09-23T16:12:59.933Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'... worker2 # [ 50.769295] nix-grpc-daemon[802]: ts=2026-09-23T16:12:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 50.781309] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 50.882970] nix-daemon[860]: accepted connection from pid 988, user nix-grpc-daemon (trusted) worker2 # [ 50.897564] nix-daemon[860]: reaped child process 991, status = succeeded lb # [ 50.880885] niks3-server[952]: 2026/09/23 16:12:59 INFO Received push request method=POST path=/api/pushes worker2 # [ 50.914857] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.088Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 50.916299] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.088Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)" lb # [ 50.912202] niks3-server[952]: 2026/09/23 16:12:59 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign lb # [ 50.915620] niks3-server[952]: 2026/09/23 16:12:59 INFO Signed narinfos id=7 count=1 worker2 # [ 50.942424] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.116Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.924136] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst lb # [ 50.928694] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls lb # [ 50.936987] niks3-server[952]: 2026/09/23 16:12:59 INFO Received complete push request method=POST path=/api/pushes/7/complete lb # [ 50.941973] niks3-server[952]: 2026/09/23 16:12:59 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo worker2 # [ 50.969236] nix-grpc-daemon[872]: time=2026-09-23T16:13:00.142Z level=INFO msg="Upload complete. (175ms)" worker2 # [ 50.973453] nix-grpc-daemon[802]: ts=2026-09-23T16:13:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 50.977970] nix-grpc-daemon[802]: ts=2026-09-23T16:13:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 50.988585] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 50.992744] nix-daemon[860]: reaped child process 986, status = succeeded worker2 # [ 50.996483] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051... client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 52.236210] nix-daemon[860]: reaped child process 995, status = succeeded worker2 # [ 52.317588] nix-daemon[860]: accepted connection from pid 1011, user nix-grpc-daemon (trusted) worker2 # [ 52.329496] nix-daemon[860]: reaped child process 1014, status = succeeded lb # [ 52.313127] niks3-server[952]: 2026/09/23 16:13:00 INFO Received push request method=POST path=/api/pushes worker2 # [ 52.344508] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.346331] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.519Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)" lb # [ 52.356557] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls lb # [ 52.361376] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv lb # [ 52.367183] niks3-server[952]: 2026/09/23 16:13:00 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign worker2 # [ 52.396822] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.570Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.373995] niks3-server[952]: 2026/09/23 16:13:00 INFO Signed narinfos id=8 count=1 lb # [ 52.379564] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst lb # [ 52.390185] niks3-server[952]: 2026/09/23 16:13:00 INFO Received complete push request method=POST path=/api/pushes/8/complete worker2 # [ 52.421158] nix-grpc-daemon[872]: time=2026-09-23T16:13:01.594Z level=INFO msg="Upload complete. (216ms)" lb # [ 52.399147] niks3-server[952]: 2026/09/23 16:13:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo worker2 # [ 52.426701] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=1 worker2 # [ 52.440948] nix-daemon[860]: reaped child process 993, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.90 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.20 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2 worker2 # [ 52.814645] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42756 client # don't know how to build these paths: client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker2 # [ 52.823223] nix-grpc-daemon[802]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 52.828216] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=0 worker1 # [ 52.834492] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 52.839231] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 52.849599] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'... worker1 # [ 52.861974] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 52.870813] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 52.893535] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=info event=niks3_push_started pid=918 worker1 # [ 53.018171] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.264Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 53.107521] nix-daemon[868]: accepted connection from pid 931, user nix-grpc-daemon (trusted) worker1 # [ 53.122273] nix-daemon[868]: reaped child process 934, status = succeeded lb # [ 53.123467] niks3-server[952]: 2026/09/23 16:13:01 INFO Received push request method=POST path=/api/pushes worker1 # [ 53.134918] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.382Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.136474] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.383Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)" lb # [ 53.159128] niks3-server[952]: 2026/09/23 16:13:01 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign lb # [ 53.165128] niks3-server[952]: 2026/09/23 16:13:01 INFO Signed narinfos id=9 count=1 worker1 # [ 53.172180] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.419Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.172501] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls lb # [ 53.189119] niks3-server[952]: 2026/09/23 16:13:01 INFO Received complete push request method=POST path=/api/pushes/9/complete lb # [ 53.191818] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst worker1 # [ 53.201825] nix-grpc-daemon[918]: time=2026-09-23T16:13:01.449Z level=INFO msg="Upload complete. (179ms)" lb # [ 53.199943] niks3-server[952]: 2026/09/23 16:13:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo worker1 # [ 53.204654] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 53.210762] nix-grpc-daemon[512]: ts=2026-09-23T16:13:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 53.220559] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 53.223364] nix-daemon[868]: reaped child process 916, status = succeeded worker1 # [ 53.227218] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051... client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker1 # [ 54.242360] nix-daemon[868]: reaped child process 938, status = succeeded worker1 # [ 54.316507] nix-daemon[868]: accepted connection from pid 954, user nix-grpc-daemon (trusted) worker1 # [ 54.328456] nix-daemon[868]: reaped child process 957, status = succeeded lb # [ 54.330941] niks3-server[952]: 2026/09/23 16:13:02 INFO Received push request method=POST path=/api/pushes worker1 # [ 54.343002] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.590Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 54.344967] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.591Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)" lb # [ 54.377296] niks3-server[952]: 2026/09/23 16:13:02 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign lb # [ 54.382764] niks3-server[952]: 2026/09/23 16:13:02 INFO Signed narinfos id=10 count=1 worker1 # [ 54.389482] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.636Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.396478] niks3-server[952]: 2026/09/23 16:13:02 INFO Received complete push request method=POST path=/api/pushes/10/complete lb # [ 54.409576] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 54.415692] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 54.422801] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls lb # [ 54.428080] niks3-server[952]: 2026/09/23 16:13:02 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo worker1 # [ 54.435487] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.682Z level=INFO msg="Upload complete. (201ms)" worker1 # [ 54.441966] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 worker1 # [ 54.453650] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=0 worker1 # [ 54.458623] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 54.462199] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 54.472296] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 54.474672] nix-daemon[868]: reaped child process 936, status = succeeded worker1 # [ 54.479274] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.726Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 54.484584] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'... worker1 # [ 54.496148] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 54.579611] nix-daemon[868]: accepted connection from pid 963, user nix-grpc-daemon (trusted) worker1 # [ 54.592835] nix-daemon[868]: reaped child process 966, status = succeeded lb # [ 54.596865] niks3-server[952]: 2026/09/23 16:13:02 INFO Received push request method=POST path=/api/pushes worker1 # [ 54.608480] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 54.610471] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.857Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)" lb # [ 54.624742] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign lb # [ 54.630348] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=11 count=1 worker1 # [ 54.637736] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.884Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.639835] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls lb # [ 54.649255] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst lb # [ 54.657935] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/11/complete lb # [ 54.665973] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo worker1 # [ 54.673147] nix-grpc-daemon[918]: time=2026-09-23T16:13:02.920Z level=INFO msg="Upload complete. (172ms)" worker1 # [ 54.677922] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 54.684276] nix-grpc-daemon[512]: ts=2026-09-23T16:13:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 54.693290] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 54.700793] nix-daemon[868]: reaped child process 961, status = succeeded worker1 # [ 54.705454] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051... client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker1 # [ 55.138316] nix-daemon[868]: reaped child process 970, status = succeeded worker1 # [ 55.212794] nix-daemon[868]: accepted connection from pid 986, user nix-grpc-daemon (trusted) worker1 # [ 55.224869] nix-daemon[868]: reaped child process 989, status = succeeded lb # [ 55.228447] niks3-server[952]: 2026/09/23 16:13:03 INFO Received push request method=POST path=/api/pushes worker1 # [ 55.240766] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 55.242127] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.487Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)" lb # [ 55.267063] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls lb # [ 55.270761] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign lb # [ 55.279171] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=12 count=1 worker1 # [ 55.287157] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.534Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.284599] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv lb # [ 55.291887] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst lb # [ 55.301325] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/12/complete lb # [ 55.309789] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo worker1 # [ 55.312662] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.560Z level=INFO msg="Upload complete. (201ms)" worker1 # [ 55.318692] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 worker1 # [ 55.330127] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=0 worker1 # [ 55.335366] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 55.339516] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 55.349152] nix-daemon[868]: reaped child process 968, status = succeeded worker1 # [ 55.350780] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 55.359417] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.606Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 55.365322] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'... worker1 # [ 55.371344] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 55.456926] nix-daemon[868]: accepted connection from pid 995, user nix-grpc-daemon (trusted) worker1 # [ 55.470404] nix-daemon[868]: reaped child process 998, status = succeeded lb # [ 55.473274] niks3-server[952]: 2026/09/23 16:13:03 INFO Received push request method=POST path=/api/pushes worker1 # [ 55.491787] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.739Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 55.493443] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.740Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)" lb # [ 55.510160] niks3-server[952]: 2026/09/23 16:13:03 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign lb # [ 55.520201] niks3-server[952]: 2026/09/23 16:13:03 INFO Signed narinfos id=13 count=1 worker1 # [ 55.528664] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.775Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.527120] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls lb # [ 55.531395] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst lb # [ 55.543311] niks3-server[952]: 2026/09/23 16:13:03 INFO Received complete push request method=POST path=/api/pushes/13/complete lb # [ 55.550776] niks3-server[952]: 2026/09/23 16:13:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo worker1 # [ 55.562676] nix-grpc-daemon[918]: time=2026-09-23T16:13:03.810Z level=INFO msg="Upload complete. (186ms)" worker1 # [ 55.568444] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 55.575331] nix-grpc-daemon[512]: ts=2026-09-23T16:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 55.584654] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 55.596386] nix-daemon[868]: reaped child process 993, status = succeeded client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051... worker1 # [ 55.603394] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker1 # [ 56.842670] nix-daemon[868]: reaped child process 1002, status = succeeded lb # [ 56.868779] envoy[921]: conn peer=[2001:db8:1::1]:42406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9434 tx=4339 ms=4015 lb # [ 56.872593] envoy[921]: conn peer=[2001:db8:1::1]:42408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=55 lb # [ 56.876005] envoy[921]: conn peer=[2001:db8:1::1]:42414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1899 ms=1734 worker1 # [ 56.919491] nix-daemon[868]: accepted connection from pid 1018, user nix-grpc-daemon (trusted) worker1 # [ 56.930577] nix-daemon[868]: reaped child process 1021, status = succeeded lb # [ 56.935361] niks3-server[952]: 2026/09/23 16:13:05 INFO Received push request method=POST path=/api/pushes worker1 # [ 56.947760] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.194Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 56.949257] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.196Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)" lb # [ 56.977517] niks3-server[952]: 2026/09/23 16:13:05 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign lb # [ 56.981889] niks3-server[952]: 2026/09/23 16:13:05 INFO Signed narinfos id=14 count=1 worker1 # [ 56.988350] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.235Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.991640] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst lb # [ 56.996711] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls lb # [ 57.000888] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv lb # [ 57.007827] niks3-server[952]: 2026/09/23 16:13:05 INFO Received complete push request method=POST path=/api/pushes/14/complete lb # [ 57.014064] niks3-server[952]: 2026/09/23 16:13:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo worker1 # [ 57.017537] nix-grpc-daemon[918]: time=2026-09-23T16:13:05.264Z level=INFO msg="Upload complete. (207ms)" worker1 # [ 57.025628] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 worker1 # [ 57.038741] nix-daemon[868]: reaped child process 1000, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.40 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 57.283192] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:60930 worker1 # [ 57.288779] nix-grpc-daemon[512]: ts=2026-09-23T16:13:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:60930 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.15 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 57.444305] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:38328 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 57.469264] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:38328 worker2 # [ 57.475695] nix-grpc-daemon[802]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'... worker2 # [ 57.489354] nix-grpc-daemon[802]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'... worker2 # [ 57.501357] nix-grpc-daemon[802]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds) client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/903" client # removing stale temporary roots file "/nix/var/nix/temproots/847" client # removing stale temporary roots file "/nix/var/nix/temproots/880" client # removing stale temporary roots file "/nix/var/nix/temproots/867" client # removing stale temporary roots file "/nix/var/nix/temproots/859" client # removing stale temporary roots file "/nix/var/nix/temproots/826" client # removing stale temporary roots file "/nix/var/nix/temproots/811" client # removing stale temporary roots file "/nix/var/nix/temproots/888" client # deleting specified paths... client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 57.766392] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 57.772115] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 57.776456] nix-grpc-daemon[918]: time=2026-09-23T16:13:06.023Z level=INFO msg="All 1 paths already cached" worker2 # [ 57.803341] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 57.791409] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying 3 paths... worker2 # [ 57.820322] nix-grpc-daemon[802]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'... worker1 # [ 57.817311] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 57.822689] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 57.843561] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 57.830624] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 worker2 # [ 57.849062] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 57.835570] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.35 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 5.21 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 57.913648] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 8c933fa1c6354e2cb434908e2905059e client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 58.043305] nix[956]: don't know how to build these paths: client # [ 58.044663] nix[956]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker2 # [ 58.038944] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 58.031366] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker2 # [ 58.051692] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 58.035517] nix-grpc-daemon[512]: ts=2026-09-23T16:13:06Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 58.063323] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 client # [ 58.076472] nix[956]: copying 1 paths to 'lb:50051'... client # [ 58.077354] nix[956]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'... worker2 # [ 58.074485] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 58.084987] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 58.187950] nix-daemon[860]: accepted connection from pid 1021, user nix-grpc-daemon (trusted) worker2 # [ 58.201772] nix-daemon[860]: reaped child process 1024, status = succeeded lb # [ 58.185316] niks3-server[952]: 2026/09/23 16:13:06 INFO Received push request method=POST path=/api/pushes worker2 # [ 58.216735] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.389Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 58.218138] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.389Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)" lb # [ 58.213958] niks3-server[952]: 2026/09/23 16:13:06 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign worker2 # [ 58.242560] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.416Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.220459] niks3-server[952]: 2026/09/23 16:13:06 INFO Signed narinfos id=15 count=1 lb # [ 58.227611] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst lb # [ 58.234794] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls lb # [ 58.246685] niks3-server[952]: 2026/09/23 16:13:06 INFO Received complete push request method=POST path=/api/pushes/15/complete lb # [ 58.252929] niks3-server[952]: 2026/09/23 16:13:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo worker2 # [ 58.281441] nix-grpc-daemon[872]: time=2026-09-23T16:13:07.455Z level=INFO msg="Upload complete. (185ms)" worker2 # [ 58.285703] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 58.291268] nix-grpc-daemon[802]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 58.300250] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 58.309130] nix-daemon[860]: reaped child process 1019, status = succeeded worker2 # [ 58.312325] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # [ 58.315590] nix[956]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client # [ 58.967623] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: b57ae4626980468aa04207299271cf4a client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds) client: must succeed: sleep 2 worker1 # [ 59.116776] nix-grpc-daemon[512]: ts=2026-09-23T16:13:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 client # [ 59.144791] nix[972]: don't know how to build these paths: client # [ 59.145784] nix[972]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker1 # [ 59.129427] nix-grpc-daemon[512]: ts=2026-09-23T16:13:07Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker2 # [ 59.153377] nix-grpc-daemon[802]: ts=2026-09-23T16:13:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 client # [ 59.159406] nix[972]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 61.343129] nix-daemon[860]: reaped child process 1028, status = succeeded worker2 # [ 61.421415] nix-daemon[860]: accepted connection from pid 1054, user nix-grpc-daemon (trusted) worker2 # [ 61.432846] nix-daemon[860]: reaped child process 1057, status = succeeded lb # [ 61.417343] niks3-server[952]: 2026/09/23 16:13:09 INFO Received push request method=POST path=/api/pushes worker2 # [ 61.449130] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.622Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 61.450695] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.624Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)" lb # [ 61.464797] niks3-server[952]: 2026/09/23 16:13:09 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign lb # [ 61.470108] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv lb # [ 61.475535] niks3-server[952]: 2026/09/23 16:13:09 INFO Signed narinfos id=16 count=1 worker2 # [ 61.498738] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.672Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.479364] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 61.486217] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls lb # [ 61.490449] niks3-server[952]: 2026/09/23 16:13:09 INFO Received complete push request method=POST path=/api/pushes/16/complete worker2 # [ 61.519863] nix-grpc-daemon[872]: time=2026-09-23T16:13:10.693Z level=INFO msg="Upload complete. (198ms)" lb # [ 61.499968] niks3-server[952]: 2026/09/23 16:13:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo worker2 # [ 61.526452] nix-grpc-daemon[802]: ts=2026-09-23T16:13:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 61.533496] nix-grpc-daemon[802]: ts=2026-09-23T16:13:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 61.545821] nix-daemon[860]: reaped child process 1026, status = succeeded client # [ 61.573536] systemd[1]: dedup2.service: Deactivated successfully. client # [ 61.576748] systemd[1]: dedup2.service: Consumed 135ms CPU time over 2.608s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic. client # [ 61.581640] systemd[1]: dedup1.service: Deactivated successfully. client # [ 61.584595] systemd[1]: dedup1.service: Consumed 114ms CPU time over 3.670s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: two clients wanting the same drv build it once, in 4.64 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 62.650846] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 62.663856] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=0 worker1 # [ 62.670576] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 62.675352] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 62.682801] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 62.693877] nix-grpc-daemon[512]: ts=2026-09-23T16:13:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 62.704275] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 62.798182] nix-daemon[868]: accepted connection from pid 1048, user nix-grpc-daemon (trusted) worker1 # [ 62.812260] nix-daemon[868]: reaped child process 1051, status = succeeded lb # [ 62.816481] niks3-server[952]: 2026/09/23 16:13:11 INFO Received push request method=POST path=/api/pushes worker1 # [ 62.827336] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.074Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 62.829237] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.076Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)" lb # [ 62.847588] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls lb # [ 62.852609] niks3-server[952]: 2026/09/23 16:13:11 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign lb # [ 62.861307] niks3-server[952]: 2026/09/23 16:13:11 INFO Signed narinfos id=17 count=1 worker1 # [ 62.868608] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.115Z level=INFO msg="Uploading 1 narinfos" lb # [ 62.869808] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst lb # [ 62.881259] niks3-server[952]: 2026/09/23 16:13:11 INFO Received complete push request method=POST path=/api/pushes/17/complete worker1 # [ 62.892868] nix-grpc-daemon[918]: time=2026-09-23T16:13:11.140Z level=INFO msg="Upload complete. (177ms)" lb # [ 62.889882] niks3-server[952]: 2026/09/23 16:13:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo worker1 # [ 62.896691] nix-grpc-daemon[512]: ts=2026-09-23T16:13:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 62.901501] nix-grpc-daemon[512]: ts=2026-09-23T16:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 62.913105] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 62.916840] nix-daemon[868]: reaped child process 1046, status = succeeded worker1 # [ 62.921762] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) lb # [ 66.869966] envoy[921]: conn peer=[2001:db8:1::1]:36184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=4336 ms=4242 lb # [ 66.873821] envoy[921]: conn peer=[2001:db8:1::1]:36200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2872 ms=94 lb # [ 66.877378] envoy[921]: conn peer=[2001:db8:1::1]:36208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=42 lb # [ 66.880913] envoy[921]: conn peer=[2001:db8:1::1]:36222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=29 lb # [ 66.884478] envoy[921]: conn peer=[2001:db8:1::1]:36220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=35 lb # [ 66.887899] envoy[921]: conn peer=[2001:db8:1::1]:36232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1521 ms=3500 lb # [ 66.891471] envoy[921]: conn peer=[2001:db8:1::1]:36234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=2411 worker1 # [ 69.134211] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 worker1 # [ 69.146137] nix-daemon[868]: reaped child process 1053, status = succeeded worker1 # [ 69.147307] nix-daemon[868]: reaped child process 1055, status = succeeded worker1 # [ 69.299618] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 69.309957] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=0 worker2 # [ 69.333388] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 69.314525] nix-grpc-daemon[512]: ts=2026-09-23T16:13:17Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 69.345695] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 69.356649] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 69.367621] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 69.473368] nix-daemon[860]: accepted connection from pid 1070, user nix-grpc-daemon (trusted) worker2 # [ 69.487744] nix-daemon[860]: reaped child process 1073, status = succeeded lb # [ 69.472948] niks3-server[952]: 2026/09/23 16:13:17 INFO Received push request method=POST path=/api/pushes worker2 # [ 69.502232] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.675Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 69.503648] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.675Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)" lb # [ 69.506730] niks3-server[952]: 2026/09/23 16:13:17 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign lb # [ 69.513289] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst worker2 # [ 69.540855] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.714Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.517630] niks3-server[952]: 2026/09/23 16:13:17 INFO Signed narinfos id=18 count=1 lb # [ 69.531155] niks3-server[952]: 2026/09/23 16:13:17 INFO Received complete push request method=POST path=/api/pushes/18/complete lb # [ 69.534755] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls worker2 # [ 69.562141] nix-grpc-daemon[872]: time=2026-09-23T16:13:18.735Z level=INFO msg="Upload complete. (182ms)" lb # [ 69.540966] niks3-server[952]: 2026/09/23 16:13:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo worker2 # [ 69.566407] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 69.570993] nix-grpc-daemon[802]: ts=2026-09-23T16:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 69.580686] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 69.588145] nix-daemon[860]: reaped child process 1068, status = succeeded worker2 # [ 69.591153] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 75.804484] nix-grpc-daemon[802]: ts=2026-09-23T16:13:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 worker2 # [ 75.815771] nix-daemon[860]: reaped child process 1075, status = succeeded worker2 # [ 75.816829] nix-daemon[860]: reaped child process 1077, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.38 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share client # [ 75.953553] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: c53babbde545413b890b57e1eda7b917 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share worker2 # [ 76.088277] nix-grpc-daemon[802]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42756 client # [ 76.094687] nix[1045]: don't know how to build these paths: client # [ 76.096091] nix[1045]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker1 # [ 76.085298] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker1 # [ 76.089364] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 76.094504] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 76.105661] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 client # [ 76.138089] nix[1045]: copying 1 paths to 'lb:50051'... client # [ 76.138880] nix[1045]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'... worker1 # [ 76.116747] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 76.127337] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 76.225979] nix-daemon[868]: accepted connection from pid 1082, user nix-grpc-daemon (trusted) worker1 # [ 76.238400] nix-daemon[868]: reaped child process 1085, status = succeeded lb # [ 76.241456] niks3-server[952]: 2026/09/23 16:13:24 INFO Received push request method=POST path=/api/pushes worker1 # [ 76.254160] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 76.256123] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.503Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)" lb # [ 76.284775] niks3-server[952]: 2026/09/23 16:13:24 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign worker1 # [ 76.294212] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.541Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.290803] niks3-server[952]: 2026/09/23 16:13:24 INFO Signed narinfos id=19 count=1 lb # [ 76.293555] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst lb # [ 76.303718] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls lb # [ 76.318438] niks3-server[952]: 2026/09/23 16:13:24 INFO Received complete push request method=POST path=/api/pushes/19/complete lb # [ 76.329337] niks3-server[952]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo worker1 # [ 76.333197] nix-grpc-daemon[918]: time=2026-09-23T16:13:24.580Z level=INFO msg="Upload complete. (194ms)" worker1 # [ 76.338910] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 76.344254] nix-grpc-daemon[512]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 76.357180] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 76.364097] nix-daemon[868]: reaped child process 1080, status = succeeded client # [ 76.393706] nix[1045]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... worker1 # [ 76.372697] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) lb # [ 76.870661] envoy[921]: conn peer=[2001:db8:1::1]:36248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=6498 lb # [ 76.874450] envoy[921]: conn peer=[2001:db8:1::1]:60702 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=6498 client # [ 77.019849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: dbfa7d3c04eb441a9c9556aa805707a8 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds) worker1 # [ 77.148077] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 client # [ 77.179287] nix[1061]: this derivation will be built: worker2 # [ 77.178496] nix-grpc-daemon[802]: ts=2026-09-23T16:13:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 77.190749] nix-grpc-daemon[802]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'... client # [ 77.210089] nix[1061]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker1 # [ 77.193198] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker1 # [ 77.196242] nix-grpc-daemon[512]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 client # [ 77.219324] nix[1061]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 80.236655] systemd[1]: share1.service: Sent signal SIGINT to main process 1045 (nix) on client request. client # [ 80.244933] nix[1045]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 80.251899] nix[1045]: error: interrupted by the user client # [ 80.294329] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 80.295873] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 80.297896] systemd[1]: share1.service: Consumed 117ms CPU time over 4.344s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds) client: waiting for success: ! systemctl is-active share2 worker1 # [ 81.595109] nix-daemon[868]: reaped child process 1089, status = succeeded worker1 # [ 81.669578] nix-daemon[868]: accepted connection from pid 1125, user nix-grpc-daemon (trusted) worker1 # [ 81.682082] nix-daemon[868]: reaped child process 1128, status = succeeded lb # [ 81.682897] niks3-server[952]: 2026/09/23 16:13:30 INFO Received push request method=POST path=/api/pushes worker1 # [ 81.697354] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 81.699130] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.946Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)" lb # [ 81.732298] niks3-server[952]: 2026/09/23 16:13:30 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign worker1 # [ 81.740592] nix-grpc-daemon[918]: time=2026-09-23T16:13:29.987Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.737226] niks3-server[952]: 2026/09/23 16:13:30 INFO Signed narinfos id=20 count=1 lb # [ 81.744955] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls lb # [ 81.749885] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv lb # [ 81.754707] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 81.765340] niks3-server[952]: 2026/09/23 16:13:30 INFO Received complete push request method=POST path=/api/pushes/20/complete worker1 # [ 81.774862] nix-grpc-daemon[918]: time=2026-09-23T16:13:30.022Z level=INFO msg="Upload complete. (189ms)" lb # [ 81.773194] niks3-server[952]: 2026/09/23 16:13:30 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo worker1 # [ 81.784168] nix-grpc-daemon[512]: ts=2026-09-23T16:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker1 # [ 81.789860] nix-grpc-daemon[512]: ts=2026-09-23T16:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker1 # [ 81.802659] nix-daemon[868]: reaped child process 1087, status = succeeded client # [ 81.839607] systemd[1]: share2.service: Deactivated successfully. client # [ 81.844245] systemd[1]: share2.service: Consumed 130ms CPU time over 4.823s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds) (finished: subtest: the first client leaving does not fail the second, in 6.71 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 82.716871] systemd[1]: Started Nix Daemon. client # [ 82.809198] nix-daemon[1112]: accepted connection from pid 1109, user root (trusted) worker1 # [ 82.910166] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 82.915237] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 82.924902] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=0 worker2 # [ 82.948343] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 82.929185] nix-grpc-daemon[512]: ts=2026-09-23T16:13:31Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 82.959698] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 82.970557] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 82.981460] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 83.080865] nix-daemon[860]: accepted connection from pid 1117, user nix-grpc-daemon (trusted) worker2 # [ 83.093802] nix-daemon[860]: reaped child process 1120, status = succeeded lb # [ 83.077378] niks3-server[952]: 2026/09/23 16:13:31 INFO Received push request method=POST path=/api/pushes worker2 # [ 83.109148] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 83.111093] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.284Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)" lb # [ 83.112112] niks3-server[952]: 2026/09/23 16:13:31 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign lb # [ 83.116382] niks3-server[952]: 2026/09/23 16:13:31 INFO Signed narinfos id=21 count=1 worker2 # [ 83.142520] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.316Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.124564] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst lb # [ 83.134867] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls lb # [ 83.142711] niks3-server[952]: 2026/09/23 16:13:31 INFO Received complete push request method=POST path=/api/pushes/21/complete worker2 # [ 83.176112] nix-grpc-daemon[872]: time=2026-09-23T16:13:32.349Z level=INFO msg="Upload complete. (183ms)" lb # [ 83.155982] niks3-server[952]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo worker2 # [ 83.181179] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 83.186979] nix-grpc-daemon[802]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 83.198204] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 83.211218] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 83.213406] nix-daemon[860]: reaped child process 1115, status = succeeded worker2 # [ 84.251549] nix-daemon[860]: reaped child process 1124, status = succeeded worker2 # [ 84.326373] nix-daemon[860]: accepted connection from pid 1140, user nix-grpc-daemon (trusted) worker2 # [ 84.337817] nix-daemon[860]: reaped child process 1143, status = succeeded lb # [ 84.321688] niks3-server[952]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes worker2 # [ 84.354144] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 84.355906] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.529Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)" lb # [ 84.364673] niks3-server[952]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign worker2 # [ 84.395736] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.569Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.372935] niks3-server[952]: 2026/09/23 16:13:32 INFO Signed narinfos id=22 count=1 lb # [ 84.380130] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 84.384733] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv lb # [ 84.389468] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls lb # [ 84.398657] niks3-server[952]: 2026/09/23 16:13:32 INFO Received complete push request method=POST path=/api/pushes/22/complete worker2 # [ 84.433236] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.606Z level=INFO msg="Upload complete. (213ms)" lb # [ 84.412112] niks3-server[952]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo worker2 # [ 84.439711] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=1 worker2 # [ 84.447462] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 84.459378] nix-daemon[860]: reaped child process 1122, status = succeeded worker1 # [ 84.441896] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 84.453270] nix-grpc-daemon[512]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'... worker1 # [ 84.467274] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 84.652275] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 84.640726] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 84.652124] nix-grpc-daemon[918]: time=2026-09-23T16:13:32.898Z level=INFO msg="All 1 paths already cached" worker1 # [ 84.666872] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=0 worker2 # [ 84.688600] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 84.671449] nix-grpc-daemon[512]: ts=2026-09-23T16:13:32Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 84.700509] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 84.713092] nix-grpc-daemon[872]: time=2026-09-23T16:13:33.886Z level=INFO msg="All 1 paths already cached" worker2 # [ 84.720703] nix-grpc-daemon[802]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 84.731164] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 84.829553] nix-daemon[860]: accepted connection from pid 1147, user nix-grpc-daemon (trusted) worker2 # [ 84.842690] nix-daemon[860]: reaped child process 1150, status = succeeded lb # [ 84.825409] niks3-server[952]: 2026/09/23 16:13:33 INFO Received push request method=POST path=/api/pushes worker2 # [ 84.856285] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.030Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 84.857820] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.031Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)" lb # [ 84.858917] niks3-server[952]: 2026/09/23 16:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign lb # [ 84.863149] niks3-server[952]: 2026/09/23 16:13:33 INFO Signed narinfos id=23 count=1 worker2 # [ 84.890313] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.064Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.871552] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls lb # [ 84.879669] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst lb # [ 84.890118] niks3-server[952]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/23/complete lb # [ 84.897946] niks3-server[952]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo worker2 # [ 84.926175] nix-grpc-daemon[872]: time=2026-09-23T16:13:34.099Z level=INFO msg="Upload complete. (182ms)" worker2 # [ 84.930465] nix-grpc-daemon[802]: ts=2026-09-23T16:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 84.936338] nix-grpc-daemon[802]: ts=2026-09-23T16:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 84.949113] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 84.959100] nix-daemon[860]: reaped child process 1145, status = succeeded worker2 # [ 84.962034] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 85.776610] nix-daemon[860]: reaped child process 1154, status = succeeded worker2 # [ 85.850035] nix-daemon[860]: accepted connection from pid 1170, user nix-grpc-daemon (trusted) worker2 # [ 85.861318] nix-daemon[860]: reaped child process 1173, status = succeeded lb # [ 85.846365] niks3-server[952]: 2026/09/23 16:13:34 INFO Received push request method=POST path=/api/pushes worker2 # [ 85.879732] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.053Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 85.881255] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.054Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)" lb # [ 85.890365] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv lb # [ 85.900233] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst lb # [ 85.904390] niks3-server[952]: 2026/09/23 16:13:34 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign lb # [ 85.909391] niks3-server[952]: 2026/09/23 16:13:34 INFO Signed narinfos id=24 count=1 worker2 # [ 85.935841] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.109Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.917232] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls lb # [ 85.926502] niks3-server[952]: 2026/09/23 16:13:34 INFO Received complete push request method=POST path=/api/pushes/24/complete worker2 # [ 85.957309] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.130Z level=INFO msg="Upload complete. (189ms)" lb # [ 85.935850] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo worker2 # [ 85.963704] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=1 worker2 # [ 85.971344] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 85.982434] nix-daemon[860]: reaped child process 1152, status = succeeded worker1 # [ 85.970583] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 85.980318] nix-grpc-daemon[512]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'... worker1 # [ 85.993332] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 86.184185] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.190101] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.195821] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.367Z level=INFO msg="All 2 paths already cached" worker1 # [ 86.186642] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=0 worker1 # [ 86.191554] nix-grpc-daemon[512]: ts=2026-09-23T16:13:34Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 86.213568] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.225290] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.237874] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.411Z level=INFO msg="All 2 paths already cached" worker2 # [ 86.245842] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.256139] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 86.355877] nix-daemon[860]: accepted connection from pid 1177, user nix-grpc-daemon (trusted) worker2 # [ 86.370549] nix-daemon[860]: reaped child process 1180, status = succeeded lb # [ 86.352673] niks3-server[952]: 2026/09/23 16:13:34 INFO Received push request method=POST path=/api/pushes worker2 # [ 86.384196] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.557Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 86.385847] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.559Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)" lb # [ 86.384499] niks3-server[952]: 2026/09/23 16:13:34 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign worker2 # [ 86.415448] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.589Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.392933] niks3-server[952]: 2026/09/23 16:13:34 INFO Signed narinfos id=25 count=1 lb # [ 86.399094] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls lb # [ 86.404388] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst lb # [ 86.415859] niks3-server[952]: 2026/09/23 16:13:34 INFO Received complete push request method=POST path=/api/pushes/25/complete lb # [ 86.426426] niks3-server[952]: 2026/09/23 16:13:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo worker2 # [ 86.453140] nix-grpc-daemon[872]: time=2026-09-23T16:13:35.626Z level=INFO msg="Upload complete. (184ms)" worker2 # [ 86.458125] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 86.463702] nix-grpc-daemon[802]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 86.476108] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 86.484275] nix-daemon[860]: reaped child process 1175, status = succeeded worker2 # [ 86.489511] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) lb # [ 86.870552] envoy[921]: conn peer=[2001:db8:1::1]:44796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=770 ms=4171 lb # [ 86.874273] envoy[921]: conn peer=[2001:db8:1::1]:44800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1598 ms=4644 lb # [ 86.877742] envoy[921]: conn peer=[2001:db8:1::1]:50520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3412 tx=1945 ms=1570 lb # [ 86.881363] envoy[921]: conn peer=[2001:db8:1::1]:50524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=334 ms=38 lb # [ 86.884722] envoy[921]: conn peer=[2001:db8:1::1]:50526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3833 tx=2124 ms=1372 lb # [ 86.888373] envoy[921]: conn peer=[2001:db8:1::1]:50538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=379 ms=35 worker2 # [ 87.525553] nix-daemon[860]: reaped child process 1184, status = succeeded worker2 # [ 87.607764] nix-daemon[860]: accepted connection from pid 1200, user nix-grpc-daemon (trusted) worker2 # [ 87.619219] nix-daemon[860]: reaped child process 1203, status = succeeded lb # [ 87.603918] niks3-server[952]: 2026/09/23 16:13:36 INFO Received push request method=POST path=/api/pushes worker2 # [ 87.635528] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.809Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 87.637254] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.810Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)" lb # [ 87.650960] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls lb # [ 87.658423] niks3-server[952]: 2026/09/23 16:13:36 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign lb # [ 87.661805] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst worker2 # [ 87.691072] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.864Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.668891] niks3-server[952]: 2026/09/23 16:13:36 INFO Signed narinfos id=26 count=1 lb # [ 87.671719] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv lb # [ 87.682888] niks3-server[952]: 2026/09/23 16:13:36 INFO Received complete push request method=POST path=/api/pushes/26/complete worker2 # [ 87.715196] nix-grpc-daemon[872]: time=2026-09-23T16:13:36.888Z level=INFO msg="Upload complete. (217ms)" lb # [ 87.693761] niks3-server[952]: 2026/09/23 16:13:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo worker2 # [ 87.721657] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=1 worker2 # [ 87.728952] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 87.741397] nix-daemon[860]: reaped child process 1182, status = succeeded worker2 # [ 87.746639] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 87.750544] nix-grpc-daemon[802]: ts=2026-09-23T16:13:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=232 client # [ 87.824782] nix-daemon[1112]: reaped child process 1114, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.21 seconds) client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.27 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds) client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds) worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 88.515346] nix-daemon[1112]: accepted connection from pid 1195, user root (trusted) client # this derivation will be built: client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker2 # [ 88.636142] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:42756 client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker2 # [ 88.641443] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying 1 paths... client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'... worker2 # [ 88.652856] nix-grpc-daemon[802]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 88.664201] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 88.765954] nix-daemon[860]: accepted connection from pid 1210, user nix-grpc-daemon (trusted) worker2 # [ 88.778394] nix-daemon[860]: reaped child process 1213, status = succeeded lb # [ 88.762652] niks3-server[952]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes worker2 # [ 88.794857] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 88.796757] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.970Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)" lb # [ 88.792420] niks3-server[952]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign lb # [ 88.797967] niks3-server[952]: 2026/09/23 16:13:37 INFO Signed narinfos id=27 count=1 worker2 # [ 88.824206] nix-grpc-daemon[872]: time=2026-09-23T16:13:37.997Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.809176] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 88.815703] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls lb # [ 88.821440] niks3-server[952]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/27/complete lb # [ 88.832275] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo worker2 # [ 88.854414] nix-grpc-daemon[872]: time=2026-09-23T16:13:38.028Z level=INFO msg="Upload complete. (178ms)" worker2 # [ 88.861283] nix-grpc-daemon[802]: ts=2026-09-23T16:13:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=120 worker2 # [ 88.872152] nix-daemon[860]: reaped child process 1208, status = succeeded worker1 # [ 88.855945] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=0 worker1 # [ 88.860880] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 88.867973] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 88.877124] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 88.889740] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 88.901395] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 88.999459] nix-daemon[868]: accepted connection from pid 1147, user nix-grpc-daemon (trusted) worker1 # [ 89.014267] nix-daemon[868]: reaped child process 1150, status = succeeded lb # [ 89.014284] niks3-server[952]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes worker1 # [ 89.027542] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.274Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 89.030216] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.276Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)" worker1 # [ 89.033470] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.276Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)" lb # [ 89.056650] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls lb # [ 89.068466] niks3-server[952]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign lb # [ 89.071775] niks3-server[952]: 2026/09/23 16:13:37 INFO Signed narinfos id=28 count=2 worker1 # [ 89.076373] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.323Z level=INFO msg="Uploading 2 narinfos" lb # [ 89.074492] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls lb # [ 89.077058] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst lb # [ 89.079801] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst lb # [ 89.087440] niks3-server[952]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/28/complete worker1 # [ 89.097270] nix-grpc-daemon[918]: time=2026-09-23T16:13:37.344Z level=INFO msg="Upload complete. (183ms)" lb # [ 89.096714] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo lb # [ 89.098527] niks3-server[952]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo worker1 # [ 89.101637] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=2 nar_bytes_in=976 worker1 # [ 89.107803] nix-grpc-daemon[512]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 89.117498] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 89.121195] nix-daemon[868]: reaped child process 1145, status = succeeded worker1 # [ 89.125846] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 90.542884] nix-daemon[868]: reaped child process 1154, status = succeeded worker1 # [ 90.615956] nix-daemon[868]: accepted connection from pid 1170, user nix-grpc-daemon (trusted) worker1 # [ 90.629334] nix-daemon[868]: reaped child process 1173, status = succeeded lb # [ 90.631706] niks3-server[952]: 2026/09/23 16:13:39 INFO Received push request method=POST path=/api/pushes worker1 # [ 90.644104] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.891Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 90.665554] niks3-server[952]: 2026/09/23 16:13:39 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign worker1 # [ 90.675900] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.923Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.671634] niks3-server[952]: 2026/09/23 16:13:39 INFO Signed narinfos id=29 count=1 lb # [ 90.679421] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv lb # [ 90.684758] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls lb # [ 90.695352] niks3-server[952]: 2026/09/23 16:13:39 INFO Received complete push request method=POST path=/api/pushes/29/complete lb # [ 90.705939] niks3-server[952]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo worker1 # [ 90.714927] nix-grpc-daemon[918]: time=2026-09-23T16:13:38.962Z level=INFO msg="Upload complete. (181ms)" worker1 # [ 90.721962] nix-grpc-daemon[512]: ts=2026-09-23T16:13:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=1 client # copying 1 paths... worker1 # [ 90.737827] nix-daemon[868]: reaped child process 1152, status = succeeded worker2 # [ 90.756115] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 90.765422] nix-grpc-daemon[802]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 90.795480] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 90.800677] nix-grpc-daemon[802]: ts=2026-09-23T16:13:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=120 client # [ 90.875347] nix-daemon[1112]: reaped child process 1198, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.48 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 90.975109] nix-daemon[1112]: accepted connection from pid 1217, user root (trusted) client # this derivation will be built: client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker2 # [ 91.099273] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:42756 worker1 # [ 91.087447] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 91.094117] nix-grpc-daemon[918]: time=2026-09-23T16:13:39.341Z level=INFO msg="All 1 paths already cached" worker1 # [ 91.104187] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=0 worker1 # [ 91.106524] nix-grpc-daemon[512]: ts=2026-09-23T16:13:39Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 91.127122] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 91.138625] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42756 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 91.154842] nix-grpc-daemon[802]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 91.166180] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 91.181993] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5) worker2 # [ 91.267713] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 2/5) worker2 # [ 91.328806] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 250 ms (attempt 3/5) worker2 # [ 91.586769] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 224 ms (attempt 4/5) worker2 # [ 91.813880] nix-grpc-daemon[802]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 91.817861] nix-grpc-daemon[802]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 91.822970] nix-grpc-daemon[802]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 91.938516] nix-daemon[860]: accepted connection from pid 1223, user nix-grpc-daemon (trusted) worker2 # [ 91.955088] nix-daemon[860]: reaped child process 1226, status = succeeded lb # [ 91.938740] niks3-server[952]: 2026/09/23 16:13:40 INFO Received push request method=POST path=/api/pushes worker2 # [ 91.971965] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 91.973485] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.147Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)" lb # [ 91.973611] niks3-server[952]: 2026/09/23 16:13:40 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign lb # [ 91.981655] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls lb # [ 91.989229] niks3-server[952]: 2026/09/23 16:13:40 INFO Signed narinfos id=30 count=1 worker2 # [ 92.016110] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.189Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.994138] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst lb # [ 92.008642] niks3-server[952]: 2026/09/23 16:13:40 INFO Received complete push request method=POST path=/api/pushes/30/complete lb # [ 92.017276] niks3-server[952]: 2026/09/23 16:13:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo worker2 # [ 92.045338] nix-grpc-daemon[872]: time=2026-09-23T16:13:41.218Z level=INFO msg="Upload complete. (204ms)" worker2 # [ 92.050142] nix-grpc-daemon[802]: ts=2026-09-23T16:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_in=560 worker2 # [ 92.055876] nix-grpc-daemon[802]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 92.067202] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) worker2 # [ 92.079264] nix-daemon[860]: reaped child process 1216, status = succeeded worker2 # [ 92.084196] nix-daemon[860]: accepted connection from pid 802, user nix-grpc-daemon (trusted) client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 92.921678] nix-daemon[860]: reaped child process 1230, status = succeeded worker2 # [ 93.002168] nix-daemon[860]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 93.014803] nix-daemon[860]: reaped child process 1249, status = succeeded lb # [ 92.998041] niks3-server[952]: 2026/09/23 16:13:41 INFO Received push request method=POST path=/api/pushes worker2 # [ 93.026187] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.199Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 93.014446] niks3-server[952]: 2026/09/23 16:13:41 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign worker2 # [ 93.039297] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.213Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.016337] niks3-server[952]: 2026/09/23 16:13:41 INFO Signed narinfos id=31 count=1 lb # [ 93.018930] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls lb # [ 93.025593] niks3-server[952]: 2026/09/23 16:13:41 INFO Received complete push request method=POST path=/api/pushes/31/complete lb # [ 93.029195] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker2 # [ 93.058060] nix-grpc-daemon[872]: time=2026-09-23T16:13:42.231Z level=INFO msg="Upload complete. (165ms)" lb # [ 93.036666] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo worker2 # [ 93.060532] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker2 # [ 93.067241] nix-daemon[860]: reaped child process 1228, status = succeeded worker1 # [ 93.049658] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.058058] nix-grpc-daemon[512]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 93.108749] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 93.113552] nix-grpc-daemon[802]: ts=2026-09-23T16:13:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42756 duration_s=0 paths=1 nar_bytes_out=120 client # [ 93.188989] nix-daemon[1112]: reaped child process 1220, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.32 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 93.294121] nix-daemon[1112]: accepted connection from pid 1239, user root (trusted) client # this derivation will be built: client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker1 # [ 93.396229] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:41870 client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 93.401999] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.407465] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.653Z level=INFO msg="All 1 paths already cached" worker1 # [ 93.415742] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=0 worker1 # [ 93.421977] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 93.426980] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.434524] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.444620] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.691Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 93.451531] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.459905] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 93.553356] nix-daemon[868]: accepted connection from pid 1182, user nix-grpc-daemon (trusted) worker1 # [ 93.565506] nix-daemon[868]: reaped child process 1185, status = succeeded lb # [ 93.568455] niks3-server[952]: 2026/09/23 16:13:41 INFO Received push request method=POST path=/api/pushes worker1 # [ 93.577811] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.825Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 93.579476] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.826Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)" lb # [ 93.587326] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls lb # [ 93.590970] niks3-server[952]: 2026/09/23 16:13:41 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign worker1 # [ 93.597547] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.844Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.593580] niks3-server[952]: 2026/09/23 16:13:41 INFO Signed narinfos id=32 count=1 lb # [ 93.599917] niks3-server[952]: 2026/09/23 16:13:41 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst lb # [ 93.604216] niks3-server[952]: 2026/09/23 16:13:42 INFO Received complete push request method=POST path=/api/pushes/32/complete worker1 # [ 93.617031] nix-grpc-daemon[918]: time=2026-09-23T16:13:41.864Z level=INFO msg="Upload complete. (145ms)" lb # [ 93.613821] niks3-server[952]: 2026/09/23 16:13:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo worker1 # [ 93.619221] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_in=560 worker1 # [ 93.623189] nix-grpc-daemon[512]: ts=2026-09-23T16:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 93.628939] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) worker1 # [ 93.631087] nix-daemon[868]: reaped child process 1180, status = succeeded worker1 # [ 93.636214] nix-daemon[868]: accepted connection from pid 512, user nix-grpc-daemon (trusted) client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 94.476428] nix-daemon[868]: reaped child process 1189, status = succeeded worker1 # [ 94.563469] nix-daemon[868]: accepted connection from pid 1205, user nix-grpc-daemon (trusted) worker1 # [ 94.574434] nix-daemon[868]: reaped child process 1208, status = succeeded lb # [ 94.578465] niks3-server[952]: 2026/09/23 16:13:42 INFO Received push request method=POST path=/api/pushes worker1 # [ 94.591309] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.838Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 94.611142] niks3-server[952]: 2026/09/23 16:13:43 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign worker1 # [ 94.620479] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.867Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.617079] niks3-server[952]: 2026/09/23 16:13:43 INFO Signed narinfos id=33 count=1 lb # [ 94.619619] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv lb # [ 94.630673] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls lb # [ 94.641238] niks3-server[952]: 2026/09/23 16:13:43 INFO Received complete push request method=POST path=/api/pushes/33/complete worker1 # [ 94.651967] nix-grpc-daemon[918]: time=2026-09-23T16:13:42.899Z level=INFO msg="Upload complete. (209ms)" lb # [ 94.653347] niks3-server[952]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo worker1 # [ 94.657470] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=1 client # copying 1 paths... worker1 # [ 94.670197] nix-daemon[868]: reaped child process 1187, status = succeeded worker2 # [ 94.687307] nix-grpc-daemon[802]: ts=2026-09-23T16:13:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:42756 worker2 # [ 94.697803] nix-grpc-daemon[802]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 94.708905] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 94.714144] nix-grpc-daemon[512]: ts=2026-09-23T16:13:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41870 duration_s=0 paths=1 nar_bytes_out=120 client # [ 94.807955] nix-daemon[1112]: reaped child process 1242, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.62 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 6.92 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds) client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.26 seconds) client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds) worker2 # [ 95.589812] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 95.874753] nix-grpc-daemon[802]: ts=2026-09-23T16:13:45Z level=info event=signal_exit lb # [ 96.871957] envoy[921]: conn peer=[2001:db8:1::1]:50552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4325 tx=2371 ms=1577 lb # [ 96.876632] envoy[921]: conn peer=[2001:db8:1::1]:50562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=15 lb # [ 96.880679] envoy[921]: conn peer=[2001:db8:1::1]:50574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4225 tx=2051 ms=2176 lb # [ 96.884830] envoy[921]: conn peer=[2001:db8:1::1]:50576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=17 lb # [ 96.888862] envoy[921]: conn peer=[2001:db8:1::1]:50582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2126 ms=2025 lb # [ 96.893246] envoy[921]: conn peer=[2001:db8:1::1]:60056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=17 lb # [ 96.897773] envoy[921]: conn peer=[2001:db8:1::1]:60068 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3717 tx=2083 ms=1329 lb # [ 96.901928] envoy[921]: conn peer=[2001:db8:1::1]:60070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=18 worker1 # [ 97.856435] nix-grpc-daemon[512]: ts=2026-09-23T16:13:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:41838 worker1 # [ 97.860554] nix-grpc-daemon[512]: ts=2026-09-23T16:13:46Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 97.871866] nix-grpc-daemon[802]: ts=2026-09-23T16:13:47Z level=info event=scheduler_restarting worker2 # [ 97.891402] nix-grpc-daemon[802]: ts=2026-09-23T16:13:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 97.911934] nix-daemon[860]: reaped child process 862, status = succeeded worker2 # [ 98.580308] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 98.584321] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 98.589292] systemd[1]: nix-grpc-daemon.service: Consumed 2.331s CPU time over 1min 5.867s wall clock time, 65.8M memory peak, 180K incoming IP traffic, 243.9K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2 worker1 # [ 98.781398] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41870 worker1 # [ 98.783867] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41870 client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds) worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds) worker2 # [ 98.882347] systemd[1]: Starting scheduler-token.service... worker2 # [ 98.918154] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 98.919809] systemd[1]: Finished scheduler-token.service. worker2 # [ 98.922164] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 37ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 98.925605] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 99.000925] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 99.002955] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 99.006132] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 99.010533] nix-grpc-daemon[1274]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 99.011971] nix-grpc-daemon[1274]: E0000 00:00:1790180028.184210 1274 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.016046] nix-grpc-daemon[1274]: E0000 00:00:1790180028.189674 1274 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.020081] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 99.023228] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 99.030048] nix-grpc-daemon[1274]: E0000 00:00:1790180028.201208 1316 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.041064] nix-grpc-daemon[1274]: E0000 00:00:1790180028.214738 1316 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.045145] nix-grpc-daemon[1274]: E0000 00:00:1790180028.217164 1316 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 99.055735] nix-grpc-daemon[1274]: E0000 00:00:1790180028.229437 1316 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 99.067183] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 99.051833] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:41838 worker1 # [ 99.060915] nix-grpc-daemon[512]: ts=2026-09-23T16:13:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1 # [ 99.083630] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 99.877639] nix-grpc-daemon[512]: ts=2026-09-23T16:13:48Z level=info event=signal_exit lb # [ 99.921900] niks3-server[952]: 2026/09/23 16:13:48 INFO lead: released remote=192.168.1.3:41386 worker1 # [ 100.925912] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=scheduler_restarting lb # [ 100.920644] envoy[921]: [2026-09-23 16:13:49.319][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 100.953295] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 100.933756] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 100.941364] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:41838 worker1 # [ 100.945466] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 100.948807] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:41838 worker1 # [ 100.952187] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 100.978602] nix-daemon[868]: reaped child process 870, status = succeeded worker2 # [ 101.153934] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.158707] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 101.150361] nix-grpc-daemon[512]: ts=2026-09-23T16:13:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 101.359800] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.363835] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 101.564952] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.568915] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 101.769951] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.774146] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 101.975645] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.979507] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.180725] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.184679] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.385844] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.389867] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 102.526452] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 102.531878] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 102.534683] systemd[1]: nix-grpc-daemon.service: Consumed 1.972s CPU time over 1min 34.957s wall clock time, 65.1M memory peak, 417.9K incoming IP traffic, 264.5K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 102.591088] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.594989] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2 worker2 # [ 102.772069] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 102.782951] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 102.795487] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 102.799279] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 102.807816] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5) worker2 # [ 102.847284] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5) worker2 # [ 103.001489] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.005334] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.049327] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 366 ms (attempt 3/5) worker2 # [ 103.206399] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.210410] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.411725] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.416437] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.421474] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 515 ms (attempt 4/5) worker2 # [ 103.617158] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.621180] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.822270] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.826237] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.945348] nix-grpc-daemon[1274]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 103.948423] nix-grpc-daemon[1274]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 103.971756] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 103.984301] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 104.007071] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=niks3_push_started pid=1334 worker2 # [ 104.027423] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_connected addr=lb:50051 lb # [ 104.006983] niks3-server[952]: 2026/09/23 16:13:52 INFO lead: acquired remote=192.168.1.4:40598 worker2 # [ 104.032361] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.036268] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_take_over worker2 # [ 104.119736] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.293Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 104.205141] nix-daemon[860]: accepted connection from pid 1346, user nix-grpc-daemon (trusted) worker2 # [ 104.216727] nix-daemon[860]: reaped child process 1349, status = succeeded lb # [ 104.200885] niks3-server[952]: 2026/09/23 16:13:52 INFO Received push request method=POST path=/api/pushes worker2 # [ 104.231578] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.233355] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.407Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 104.235822] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.408Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)" worker2 # [ 104.239103] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.411Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)" worker2 # [ 104.246867] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 104.249394] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:51120 worker2 # [ 104.251684] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb # [ 104.267224] niks3-server[952]: 2026/09/23 16:13:52 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign worker2 # [ 104.296123] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.469Z level=INFO msg="Uploading 2 narinfos" lb # [ 104.274041] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls lb # [ 104.275479] niks3-server[952]: 2026/09/23 16:13:52 INFO Signed narinfos id=34 count=2 lb # [ 104.278331] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 104.287595] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls lb # [ 104.289890] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst lb # [ 104.297693] niks3-server[952]: 2026/09/23 16:13:52 INFO Received complete push request method=POST path=/api/pushes/34/complete lb # [ 104.302076] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo lb # [ 104.307888] niks3-server[952]: 2026/09/23 16:13:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo worker2 # [ 104.334601] nix-grpc-daemon[1334]: time=2026-09-23T16:13:53.508Z level=INFO msg="Upload complete. (211ms)" worker2 # [ 104.339290] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 104.350858] nix-daemon[860]: reaped child process 1332, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.75 seconds) worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds) worker1 # [ 104.421954] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 104.511342] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 104.513493] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 104.516801] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 104.521061] nix-grpc-daemon[1231]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 104.524097] nix-grpc-daemon[1231]: E0000 00:00:1790180032.768312 1231 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 104.526557] nix-grpc-daemon[1231]: E0000 00:00:1790180032.773933 1231 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 104.530279] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 104.534123] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 104.540087] nix-grpc-daemon[1231]: E0000 00:00:1790180032.785389 1274 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 104.552075] nix-grpc-daemon[1231]: E0000 00:00:1790180032.799350 1274 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 104.555169] nix-grpc-daemon[1231]: E0000 00:00:1790180032.801726 1274 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 104.566388] nix-grpc-daemon[1231]: E0000 00:00:1790180032.813840 1274 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 104.579109] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.600158] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:51120 worker2 # [ 104.604656] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.86 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb # [ 105.716936] systemd[1]: Stopping niks3 server... lb # [ 105.718648] niks3-server[952]: 2026/09/23 16:13:54 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 105.722063] niks3-server[952]: 2026/09/23 16:13:54 INFO lead: released remote=192.168.1.4:40598 worker1 # [ 105.728766] nix-grpc-daemon[1231]: ts=2026-09-23T16:13:53Z level=info event=scheduler_lock_lost status=200 worker2 # [ 105.748688] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:54Z level=info event=scheduler_lock_lost status=200 lb # [ 105.737632] systemd[1]: niks3.service: Deactivated successfully. lb # [ 105.740083] systemd[1]: Stopped niks3 server. lb # [ 105.740753] systemd[1]: niks3.service: Consumed 736ms CPU time over 1min 28.042s wall clock time, 28.6M memory peak, 97.5K incoming IP traffic, 139.7K outgoing IP traffic. lb # [ 105.747291] systemd[1]: Starting niks3 server... lb # [ 105.796145] niks3-server[1142]: 2026/09/23 16:13:54 goose: no migrations to run. current version: 20260923120000 lb # [ 105.801282] niks3-server[1142]: 2026/09/23 16:13:54 OK 1_commit_pending_closure.sql (5.89ms) lb # [ 105.804966] niks3-server[1142]: 2026/09/23 16:13:54 OK 2_object_stats_trigger.sql (3.64ms) lb # [ 105.808541] niks3-server[1142]: 2026/09/23 16:13:54 OK 3_commit_push.sql (3.22ms) lb # [ 105.809555] niks3-server[1142]: 2026/09/23 16:13:54 goose: up to current file version: 3 lb # [ 105.810843] niks3-server[1142]: 2026/09/23 16:13:54 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 105.821519] niks3-server[1142]: 2026/09/23 16:13:54 INFO Read proxy enabled — serving cache objects from S3 lb # [ 105.823915] niks3-server[1142]: 2026/09/23 16:13:54 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 105.825457] niks3-server[1142]: 2026/09/23 16:13:54 INFO systemd watchdog enabled interval=15s lb # [ 105.827236] systemd[1]: Started niks3 server. lb # [ 105.828627] niks3-server[1142]: 2026/09/23 16:13:54 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 106.179270] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36232 client # don't know how to build these paths: client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker2 # [ 106.188877] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 106.213723] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=0 worker2 # [ 106.220246] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 106.224741] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 106.235990] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36232 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'... worker2 # [ 106.248280] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 106.259752] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 106.354400] nix-daemon[860]: accepted connection from pid 1366, user nix-grpc-daemon (trusted) worker2 # [ 106.367725] nix-daemon[860]: reaped child process 1369, status = succeeded lb # [ 106.350317] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received push request method=POST path=/api/pushes worker2 # [ 106.386094] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.559Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 106.388246] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.561Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)" lb # [ 106.392689] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign lb # [ 106.404417] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst lb # [ 106.419084] niks3-server[1142]: 2026/09/23 16:13:54 INFO Signed narinfos id=35 count=1 worker2 # [ 106.444283] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.617Z level=INFO msg="Uploading 1 narinfos" lb # [ 106.424640] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls lb # [ 106.429916] niks3-server[1142]: 2026/09/23 16:13:54 INFO Received complete push request method=POST path=/api/pushes/35/complete worker2 # [ 106.460698] nix-grpc-daemon[1334]: time=2026-09-23T16:13:55.634Z level=INFO msg="Upload complete. (191ms)" lb # [ 106.441122] niks3-server[1142]: 2026/09/23 16:13:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo worker2 # [ 106.465888] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=536 worker2 # [ 106.471494] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 106.482486] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 106.491933] nix-daemon[860]: reaped child process 1364, status = succeeded worker2 # [ 106.496990] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051... client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 106.730985] niks3-server[1142]: 2026/09/23 16:13:55 INFO lead: acquired remote=192.168.1.4:56738 lb # [ 106.873174] envoy[921]: conn peer=[2001:db8:1::4]:45820 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3164 tx=1089 ms=64985 lb # [ 106.876919] envoy[921]: conn peer=[2001:db8:1::1]:60082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=13 lb # [ 106.880365] envoy[921]: conn peer=192.168.1.3:39430 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2024 tx=886 ms=67360 lb # [ 106.883898] envoy[921]: conn peer=[2001:db8:1::1]:60090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=305 ms=1582 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 107.933945] nix-daemon[860]: reaped child process 1373, status = succeeded worker2 # [ 108.019086] nix-daemon[860]: accepted connection from pid 1391, user nix-grpc-daemon (trusted) worker2 # [ 108.031571] nix-daemon[860]: reaped child process 1394, status = succeeded lb # [ 108.014527] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received push request method=POST path=/api/pushes worker2 # [ 108.050244] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 108.053252] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.225Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)" lb # [ 108.057209] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls lb # [ 108.066168] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign lb # [ 108.075367] niks3-server[1142]: 2026/09/23 16:13:56 INFO Signed narinfos id=36 count=1 worker2 # [ 108.104856] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.278Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.087432] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 108.094782] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received complete push request method=POST path=/api/pushes/36/complete lb # [ 108.102142] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 108.105505] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo worker2 # [ 108.132204] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.305Z level=INFO msg="Upload complete. (227ms)" worker2 # [ 108.148153] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 worker2 # [ 108.155853] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=0 worker2 # [ 108.159766] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 108.164244] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 108.173517] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 108.177351] nix-daemon[860]: reaped child process 1371, status = succeeded worker2 # [ 108.182450] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.356Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 108.188128] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'... worker2 # [ 108.193100] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 108.276142] nix-daemon[860]: accepted connection from pid 1400, user nix-grpc-daemon (trusted) worker2 # [ 108.288263] nix-daemon[860]: reaped child process 1403, status = succeeded lb # [ 108.272868] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received push request method=POST path=/api/pushes worker2 # [ 108.303861] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 108.305854] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.479Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)" lb # [ 108.302527] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign worker2 # [ 108.331327] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.504Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.308077] niks3-server[1142]: 2026/09/23 16:13:56 INFO Signed narinfos id=37 count=1 lb # [ 108.317255] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst lb # [ 108.329122] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls lb # [ 108.333121] niks3-server[1142]: 2026/09/23 16:13:56 INFO Received complete push request method=POST path=/api/pushes/37/complete worker2 # [ 108.365193] nix-grpc-daemon[1334]: time=2026-09-23T16:13:57.538Z level=INFO msg="Upload complete. (166ms)" lb # [ 108.346658] niks3-server[1142]: 2026/09/23 16:13:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo worker2 # [ 108.369342] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 108.374311] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 108.385682] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 108.388278] nix-daemon[860]: reaped child process 1398, status = succeeded worker2 # [ 108.393665] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 109.411263] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:51124 client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 109.608854] nix-daemon[860]: reaped child process 1407, status = succeeded worker2 # [ 109.682451] nix-daemon[860]: accepted connection from pid 1423, user nix-grpc-daemon (trusted) worker2 # [ 109.694832] nix-daemon[860]: reaped child process 1426, status = succeeded lb # [ 109.679341] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes worker2 # [ 109.711076] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 109.712470] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.886Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)" lb # [ 109.714612] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst lb # [ 109.729269] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv lb # [ 109.736051] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign lb # [ 109.739194] niks3-server[1142]: 2026/09/23 16:13:58 INFO Signed narinfos id=38 count=1 worker2 # [ 109.765367] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.938Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.749885] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls lb # [ 109.753700] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/38/complete lb # [ 109.761914] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo worker2 # [ 109.783771] nix-grpc-daemon[1334]: time=2026-09-23T16:13:58.957Z level=INFO msg="Upload complete. (181ms)" worker2 # [ 109.790146] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 worker2 # [ 109.800717] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=0 worker2 # [ 109.805581] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 109.810505] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 109.820282] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 109.823233] nix-daemon[860]: reaped child process 1405, status = succeeded worker2 # [ 109.829062] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.002Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 109.834642] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'... worker2 # [ 109.839717] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 109.922242] nix-daemon[860]: accepted connection from pid 1432, user nix-grpc-daemon (trusted) worker2 # [ 109.936477] nix-daemon[860]: reaped child process 1435, status = succeeded lb # [ 109.917531] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes worker2 # [ 109.949947] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.123Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 109.951887] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.125Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)" lb # [ 109.946890] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls lb # [ 109.954614] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign lb # [ 109.959984] niks3-server[1142]: 2026/09/23 16:13:58 INFO Signed narinfos id=39 count=1 worker2 # [ 109.985716] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.159Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.966212] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst lb # [ 109.975913] niks3-server[1142]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/39/complete lb # [ 109.981584] niks3-server[1142]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo worker2 # [ 110.008427] nix-grpc-daemon[1334]: time=2026-09-23T16:13:59.182Z level=INFO msg="Upload complete. (163ms)" worker2 # [ 110.012591] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 110.017407] nix-grpc-daemon[1274]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 worker2 # [ 110.028067] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker2 # [ 110.032459] nix-daemon[860]: reaped child process 1430, status = succeeded worker2 # [ 110.036657] nix-daemon[860]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051... client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 110.850656] nix-daemon[860]: reaped child process 1439, status = succeeded worker2 # [ 110.926619] nix-daemon[860]: accepted connection from pid 1455, user nix-grpc-daemon (trusted) worker2 # [ 110.938629] nix-daemon[860]: reaped child process 1458, status = succeeded lb # [ 110.922527] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received push request method=POST path=/api/pushes worker2 # [ 110.953987] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.127Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 110.956359] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.129Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)" lb # [ 110.966346] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign lb # [ 110.970272] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker2 # [ 111.000397] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.173Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.977043] niks3-server[1142]: 2026/09/23 16:13:59 INFO Signed narinfos id=40 count=1 lb # [ 110.983996] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls lb # [ 110.987990] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst lb # [ 110.997568] niks3-server[1142]: 2026/09/23 16:13:59 INFO Received complete push request method=POST path=/api/pushes/40/complete lb # [ 111.002115] niks3-server[1142]: 2026/09/23 16:13:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo worker2 # [ 111.024376] nix-grpc-daemon[1334]: time=2026-09-23T16:14:00.197Z level=INFO msg="Upload complete. (181ms)" worker2 # [ 111.031258] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36232 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 worker2 # [ 111.043871] nix-daemon[860]: reaped child process 1437, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.04 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 5.52 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2 # [ 111.348113] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 112.024623] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:01Z level=info event=signal_exit lb # [ 112.735776] niks3-server[1142]: 2026/09/23 16:14:01 INFO lead: released remote=192.168.1.4:56738 worker2 # [ 113.758827] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=scheduler_restarting lb # [ 113.735687] envoy[921]: [2026-09-23 16:14:02.134][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 113.747792] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:01Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 113.766963] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:51120 worker2 # [ 113.772264] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 113.775447] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 113.780408] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:51120 worker2 # [ 113.784253] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:02Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 113.804397] nix-daemon[860]: reaped child process 1326, status = succeeded worker1 # [ 113.948088] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 113.952353] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 113.981335] nix-grpc-daemon[1274]: ts=2026-09-23T16:14:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 114.153160] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.157197] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.358139] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.362161] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.562952] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.566959] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.768493] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.772273] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 114.973123] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 114.977129] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.177944] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.181985] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.382871] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.386846] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.587660] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.591601] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.793590] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.797186] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.971356] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 115.975364] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 115.979454] systemd[1]: nix-grpc-daemon.service: Consumed 897ms CPU time over 17.048s wall clock time, 38.5M memory peak, 201.3K incoming IP traffic, 128.2K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 115.998436] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.002507] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker1 # [ 116.203233] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.207080] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.408068] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.412086] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.612885] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.616856] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.734829] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:04Z level=info event=scheduler_take_over lb # [ 116.731382] niks3-server[1142]: 2026/09/23 16:14:05 INFO lead: acquired remote=192.168.1.3:42286 worker1 # [ 116.818447] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.822308] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 116.873732] envoy[921]: conn peer=[2001:db8:1::1]:53200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9662 tx=4392 ms=4871 lb # [ 116.877753] envoy[921]: conn peer=[2001:db8:1::4]:59242 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=15028 tx=1063 ms=14909 worker1 # [ 117.023263] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.027190] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 117.227847] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.239602] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45962 worker1 # [ 117.243263] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2 worker1 # [ 118.296596] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 118.307783] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker1 # [ 118.322117] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 118.332264] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5) worker1 # [ 118.412403] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 2/5) worker1 # [ 118.609299] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 281 ms (attempt 3/5) worker1 # [ 118.683569] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:06Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:45986 worker1 # [ 118.892302] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 4/5) worker1 # [ 118.956090] nix-grpc-daemon[1231]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 118.959324] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 118.986839] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=0 worker1 # [ 118.992272] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 118.998375] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 119.010103] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45972 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'... worker1 # [ 119.022427] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 119.030559] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 119.050855] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=info event=niks3_push_started pid=1310 worker1 # [ 119.148259] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.395Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 119.233624] nix-daemon[868]: accepted connection from pid 1323, user nix-grpc-daemon (trusted) worker1 # [ 119.245386] nix-daemon[868]: reaped child process 1326, status = succeeded lb # [ 119.249180] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes worker1 # [ 119.260992] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 119.262490] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.509Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)" lb # [ 119.284158] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls lb # [ 119.290543] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign worker1 # [ 119.300431] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.547Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.295698] niks3-server[1142]: 2026/09/23 16:14:07 INFO Signed narinfos id=41 count=1 lb # [ 119.309576] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst lb # [ 119.317221] niks3-server[1142]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/41/complete worker1 # [ 119.330384] nix-grpc-daemon[1310]: time=2026-09-23T16:14:07.577Z level=INFO msg="Upload complete. (178ms)" lb # [ 119.327871] niks3-server[1142]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo worker1 # [ 119.332928] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 119.336520] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 119.343236] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 119.345539] nix-daemon[868]: reaped child process 1308, status = succeeded worker1 # [ 119.350429] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051... client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 120.987529] nix-daemon[868]: reaped child process 1330, status = succeeded worker1 # [ 121.068964] nix-daemon[868]: accepted connection from pid 1346, user nix-grpc-daemon (trusted) worker1 # [ 121.081343] nix-daemon[868]: reaped child process 1349, status = succeeded lb # [ 121.084528] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes worker1 # [ 121.101927] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 121.103509] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.350Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)" lb # [ 121.128396] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv lb # [ 121.134086] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign lb # [ 121.140099] niks3-server[1142]: 2026/09/23 16:14:09 INFO Signed narinfos id=42 count=1 worker1 # [ 121.146712] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.393Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.147549] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 121.150392] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls lb # [ 121.154278] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/42/complete lb # [ 121.161788] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo worker1 # [ 121.170824] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.418Z level=INFO msg="Upload complete. (212ms)" worker1 # [ 121.176992] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 worker1 # [ 121.189505] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=0 worker1 # [ 121.192988] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 121.197202] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 121.205291] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 121.208098] nix-daemon[868]: reaped child process 1328, status = succeeded worker1 # [ 121.212286] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.459Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 121.217344] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'... worker1 # [ 121.222142] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 121.304803] nix-daemon[868]: accepted connection from pid 1355, user nix-grpc-daemon (trusted) worker1 # [ 121.317607] nix-daemon[868]: reaped child process 1358, status = succeeded lb # [ 121.320059] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes worker1 # [ 121.333388] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 121.335429] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.582Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)" lb # [ 121.352456] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls lb # [ 121.357237] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign lb # [ 121.361418] niks3-server[1142]: 2026/09/23 16:14:09 INFO Signed narinfos id=43 count=1 worker1 # [ 121.369824] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.617Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.370286] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst lb # [ 121.377335] niks3-server[1142]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/43/complete worker1 # [ 121.386610] nix-grpc-daemon[1310]: time=2026-09-23T16:14:09.634Z level=INFO msg="Upload complete. (159ms)" lb # [ 121.383532] niks3-server[1142]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo worker1 # [ 121.388572] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 121.391834] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 121.400074] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 121.401197] nix-daemon[868]: reaped child process 1353, status = succeeded worker1 # [ 121.406168] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051... client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 122.234187] nix-daemon[868]: reaped child process 1362, status = succeeded worker1 # [ 122.308945] nix-daemon[868]: accepted connection from pid 1378, user nix-grpc-daemon (trusted) worker1 # [ 122.321278] nix-daemon[868]: reaped child process 1381, status = succeeded lb # [ 122.324986] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received push request method=POST path=/api/pushes worker1 # [ 122.337188] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.584Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 122.338678] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.586Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)" lb # [ 122.362738] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign worker1 # [ 122.373448] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.620Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.369299] niks3-server[1142]: 2026/09/23 16:14:10 INFO Signed narinfos id=44 count=1 lb # [ 122.381942] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls lb # [ 122.387891] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv lb # [ 122.393995] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst lb # [ 122.402265] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received complete push request method=POST path=/api/pushes/44/complete lb # [ 122.409941] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo worker1 # [ 122.412760] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.660Z level=INFO msg="Upload complete. (200ms)" worker1 # [ 122.423266] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 worker1 # [ 122.431488] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=0 worker1 # [ 122.436323] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 122.439828] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 122.449554] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 122.452767] nix-daemon[868]: reaped child process 1360, status = succeeded worker1 # [ 122.457356] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.704Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 122.462881] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'... worker1 # [ 122.467860] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 122.551852] nix-daemon[868]: accepted connection from pid 1387, user nix-grpc-daemon (trusted) worker1 # [ 122.564204] nix-daemon[868]: reaped child process 1390, status = succeeded lb # [ 122.567437] niks3-server[1142]: 2026/09/23 16:14:10 INFO Received push request method=POST path=/api/pushes worker1 # [ 122.580239] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.827Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 122.582218] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.829Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)" lb # [ 122.602226] niks3-server[1142]: 2026/09/23 16:14:10 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst lb # [ 122.606497] niks3-server[1142]: 2026/09/23 16:14:11 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign lb # [ 122.610436] niks3-server[1142]: 2026/09/23 16:14:11 INFO Signed narinfos id=45 count=1 worker1 # [ 122.619640] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.866Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.619874] niks3-server[1142]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls lb # [ 122.631398] niks3-server[1142]: 2026/09/23 16:14:11 INFO Received complete push request method=POST path=/api/pushes/45/complete lb # [ 122.637574] niks3-server[1142]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo worker1 # [ 122.644943] nix-grpc-daemon[1310]: time=2026-09-23T16:14:10.892Z level=INFO msg="Upload complete. (171ms)" worker1 # [ 122.648143] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 122.652938] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 122.664107] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 122.667764] nix-daemon[868]: reaped child process 1385, status = succeeded worker1 # [ 122.672958] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051... client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 124.113740] nix-daemon[868]: reaped child process 1394, status = succeeded worker1 # [ 124.194869] nix-daemon[868]: accepted connection from pid 1410, user nix-grpc-daemon (trusted) worker1 # [ 124.206145] nix-daemon[868]: reaped child process 1413, status = succeeded lb # [ 124.209752] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received push request method=POST path=/api/pushes worker1 # [ 124.222444] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 124.223900] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.471Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)" lb # [ 124.250517] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign lb # [ 124.256507] niks3-server[1142]: 2026/09/23 16:14:12 INFO Signed narinfos id=46 count=1 worker1 # [ 124.263782] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.510Z level=INFO msg="Uploading 1 narinfos" lb # [ 124.264793] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst lb # [ 124.271677] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls lb # [ 124.277126] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv lb # [ 124.286284] niks3-server[1142]: 2026/09/23 16:14:12 INFO Received complete push request method=POST path=/api/pushes/46/complete lb # [ 124.295288] niks3-server[1142]: 2026/09/23 16:14:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo worker1 # [ 124.303077] nix-grpc-daemon[1310]: time=2026-09-23T16:14:12.550Z level=INFO msg="Upload complete. (221ms)" worker1 # [ 124.310106] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 worker1 # [ 124.325720] nix-daemon[868]: reaped child process 1392, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 6.20 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds) worker2 # [ 124.539429] systemd[1]: Starting scheduler-token.service... worker2 # [ 124.581755] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 124.583359] systemd[1]: Finished scheduler-token.service. worker2 # [ 124.584632] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 46ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 124.589296] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 124.664241] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 124.669063] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 124.672197] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 124.674221] nix-grpc-daemon[1480]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 124.675506] nix-grpc-daemon[1480]: E0000 00:00:1790180053.845039 1480 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.678761] nix-grpc-daemon[1480]: E0000 00:00:1790180053.852426 1480 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.686679] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 124.688574] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 124.693085] nix-grpc-daemon[1480]: E0000 00:00:1790180053.864570 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 124.703909] nix-grpc-daemon[1480]: E0000 00:00:1790180053.877645 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.707141] nix-grpc-daemon[1480]: E0000 00:00:1790180053.879888 1523 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.718050] nix-grpc-daemon[1480]: E0000 00:00:1790180053.891740 1523 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.728484] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 124.717169] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 124.721292] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45962 worker1 # [ 124.728421] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.71 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs client # [ 124.948942] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 3d430631a43040c9923b371fb8fb2625 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds) worker2 # [ 125.090943] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42130 worker2 # [ 125.102531] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted) client # [ 125.122833] nix[1363]: don't know how to build these paths: client # [ 125.124279] nix[1363]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker1 # [ 125.109637] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=0 worker1 # [ 125.115229] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 125.121508] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 125.132641] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45972 client # [ 125.165157] nix[1363]: copying 1 paths to 'lb:50051'... client # [ 125.168180] nix[1363]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'... worker1 # [ 125.146422] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 125.157624] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 125.253826] nix-daemon[868]: accepted connection from pid 1426, user nix-grpc-daemon (trusted) worker1 # [ 125.266236] nix-daemon[868]: reaped child process 1429, status = succeeded lb # [ 125.269595] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes worker1 # [ 125.282184] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 125.283552] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.529Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)" lb # [ 125.299229] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign lb # [ 125.308258] niks3-server[1142]: 2026/09/23 16:14:13 INFO Signed narinfos id=47 count=1 worker1 # [ 125.315626] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.562Z level=INFO msg="Uploading 1 narinfos" lb # [ 125.317676] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst lb # [ 125.329694] niks3-server[1142]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/47/complete worker1 # [ 125.344351] nix-grpc-daemon[1310]: time=2026-09-23T16:14:13.591Z level=INFO msg="Upload complete. (175ms)" lb # [ 125.344916] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo lb # [ 125.347582] niks3-server[1142]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls worker1 # [ 125.349786] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 125.357312] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 worker1 # [ 125.366142] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker1 # [ 125.374830] nix-daemon[868]: reaped child process 1424, status = succeeded worker1 # [ 125.379431] nix-daemon[868]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) client # [ 125.401662] nix[1363]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051... lb # [ 126.874146] envoy[921]: conn peer=[2001:db8:1::1]:49598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4373 ms=6031 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 127.611139] systemd-journald[282]: Received client request to rotate journal, rotating. worker2 # [ 127.632097] systemd-journald[282]: Vacuuming done, freed 0B of archived journals from /var/log/journal/bd1869688d4b467bbe7e20bf3feec49a. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 127.518248] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds) client: must succeed: sleep 5.0 worker1 # [ 127.538163] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:15Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.03 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 132.820313] nix-daemon[868]: reaped child process 1433, status = succeeded worker1 # [ 132.897905] nix-daemon[868]: accepted connection from pid 1474, user nix-grpc-daemon (trusted) worker1 # [ 132.910453] nix-daemon[868]: reaped child process 1477, status = succeeded lb # [ 132.913591] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received push request method=POST path=/api/pushes worker1 # [ 132.929776] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 132.931335] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.178Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)" lb # [ 132.960346] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv lb # [ 132.966718] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign worker1 # [ 132.979880] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.227Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.976114] niks3-server[1142]: 2026/09/23 16:14:21 INFO Signed narinfos id=48 count=1 lb # [ 132.980955] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 132.986726] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls lb # [ 132.996774] niks3-server[1142]: 2026/09/23 16:14:21 INFO Received complete push request method=POST path=/api/pushes/48/complete worker1 # [ 133.006058] nix-grpc-daemon[1310]: time=2026-09-23T16:14:21.253Z level=INFO msg="Upload complete. (212ms)" lb # [ 133.003931] niks3-server[1142]: 2026/09/23 16:14:21 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo worker1 # [ 133.013386] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45972 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 133.028597] nix-daemon[868]: reaped child process 1431, status = succeeded client # [ 133.074558] systemd[1]: rs.service: Deactivated successfully. client # [ 133.079287] systemd[1]: rs.service: Consumed 140ms CPU time over 8.128s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 133.540199] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:21Z level=info event=signal_exit lb # [ 133.734164] niks3-server[1142]: 2026/09/23 16:14:22 INFO lead: released remote=192.168.1.3:42286 lb # [ 134.665775] niks3-server[1142]: 2026/09/23 16:14:23 INFO lead: acquired remote=192.168.1.4:57136 worker2 # [ 134.689179] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:23Z level=info event=scheduler_take_over worker1 # [ 134.740542] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=scheduler_restarting lb # [ 134.735480] envoy[921]: [2026-09-23 16:14:23.133][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 134.746693] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 134.766171] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:23Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 134.753881] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45962 worker1 # [ 134.757873] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 134.761289] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45962 worker1 # [ 134.764987] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:22Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 134.784694] nix-daemon[868]: reaped child process 1300, status = succeeded worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 134.972308] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 134.976319] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 134.959841] nix-grpc-daemon[1231]: ts=2026-09-23T16:14:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 135.177168] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.190155] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 135.193897] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52818 worker2 # [ 135.197713] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 135.560269] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:24Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:52832 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 136.812828] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 136.816466] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 136.818554] systemd[1]: nix-grpc-daemon.service: Consumed 1.104s CPU time over 32.391s wall clock time, 37.7M memory peak, 332.8K incoming IP traffic, 146.9K outgoing IP traffic. worker1 # [ 136.834562] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 136.874387] envoy[921]: conn peer=[2001:db8:1::1]:37894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1498 ms=7960 lb # [ 136.878220] envoy[921]: conn peer=[2001:db8:1::3]:57574 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4814 tx=1191 ms=30378 worker1 # [ 136.922321] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 136.926573] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 136.929866] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 136.933578] nix-grpc-daemon[1485]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 136.934906] nix-grpc-daemon[1485]: E0000 00:00:1790180065.179468 1485 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.937089] nix-grpc-daemon[1485]: E0000 00:00:1790180065.180860 1485 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.941540] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 136.943533] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 136.947057] nix-grpc-daemon[1485]: E0000 00:00:1790180065.194450 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.959138] nix-grpc-daemon[1485]: E0000 00:00:1790180065.206538 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.963130] nix-grpc-daemon[1485]: E0000 00:00:1790180065.208695 1528 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 136.973051] nix-grpc-daemon[1485]: E0000 00:00:1790180065.220466 1528 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 136.982314] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 137.003845] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52818 worker2 # [ 137.008442] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.27 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid client # [ 137.222986] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 4a3ee47dfb484f2da4e28102b09d1870 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.07 seconds) worker1 # [ 137.349846] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58818 worker1 # [ 137.360893] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) client # [ 137.395446] nix[1399]: don't know how to build these paths: client # [ 137.396445] nix[1399]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker2 # [ 137.400322] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker2 # [ 137.406992] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 137.412505] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42130 worker2 # [ 137.422236] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5) worker2 # [ 137.444684] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5) worker2 # [ 137.461315] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 207 ms (attempt 3/5) worker2 # [ 137.670614] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 649 ms (attempt 4/5) worker2 # [ 138.322924] nix-grpc-daemon[1480]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 138.326713] nix-grpc-daemon[1480]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 138.344219] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42130 client # [ 138.358114] nix[1399]: copying 1 paths to 'lb:50051'... client # [ 138.362156] nix[1399]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'... worker2 # [ 138.359302] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42130 worker2 # [ 138.370297] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted) worker2 # [ 138.393392] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=info event=niks3_push_started pid=1604 worker2 # [ 138.507099] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.679Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 138.591648] nix-daemon[860]: accepted connection from pid 1618, user nix-grpc-daemon (trusted) worker2 # [ 138.603414] nix-daemon[860]: reaped child process 1621, status = succeeded lb # [ 138.587848] niks3-server[1142]: 2026/09/23 16:14:26 INFO Received push request method=POST path=/api/pushes worker2 # [ 138.622215] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 138.623748] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.797Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)" lb # [ 138.625234] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls lb # [ 138.631858] niks3-server[1142]: 2026/09/23 16:14:27 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign lb # [ 138.636732] niks3-server[1142]: 2026/09/23 16:14:27 INFO Signed narinfos id=49 count=1 worker2 # [ 138.662727] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.836Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.655121] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst lb # [ 138.662119] niks3-server[1142]: 2026/09/23 16:14:27 INFO Received complete push request method=POST path=/api/pushes/49/complete lb # [ 138.669406] niks3-server[1142]: 2026/09/23 16:14:27 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo worker2 # [ 138.697350] nix-grpc-daemon[1604]: time=2026-09-23T16:14:27.870Z level=INFO msg="Upload complete. (190ms)" worker2 # [ 138.702252] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42130 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 138.707274] nix-grpc-daemon[1480]: ts=2026-09-23T16:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42130 worker2 # [ 138.716706] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted) worker2 # [ 138.722559] nix-daemon[860]: reaped child process 1600, status = succeeded worker2 # [ 138.727428] nix-daemon[860]: accepted connection from pid 1480, user nix-grpc-daemon (trusted) client # [ 138.731393] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 139.768285] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1480 (nix-grpc-daemon) on client request. worker2 # [ 139.770332] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 139.752858] niks3-server[1142]: 2026/09/23 16:14:28 INFO lead: released remote=192.168.1.4:57136 worker2 # [ 139.782756] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL client # [ 139.785437] nix[1399]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 139.786060] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker1 # [ 139.762565] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker2 # [ 139.787269] systemd[1]: nix-grpc-daemon.service: Consumed 382ms CPU time over 15.195s wall clock time, 36M memory peak, 214.2K incoming IP traffic, 53.2K outgoing IP traffic. lb # [ 139.760401] envoy[921]: [2026-09-23 16:14:28.158][921][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} client # [ 139.790669] nix[1399]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 139.797718] systemd[1]: Starting scheduler-token.service... worker2 # [ 139.813312] nix-daemon[860]: reaped child process 1623, status = succeeded worker2 # [ 139.815163] nix-daemon[860]: reaped child process 1539, status = succeeded worker2 # [ 139.824408] nix-daemon[1625]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 139.829922] nix-daemon[860]: reaped child process 1625, status = failed with exit code 1 worker2 # [ 139.848480] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 139.849715] systemd[1]: Finished scheduler-token.service. worker2 # [ 139.851520] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 51ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 139.853695] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 139.856877] systemd[1]: Starting scheduler-token.service... worker2 # [ 139.889420] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 139.890985] systemd[1]: Finished scheduler-token.service. worker2 # [ 139.893164] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 33ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 139.896600] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 139.971894] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 139.973824] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 139.976960] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 139.981326] nix-grpc-daemon[1664]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 139.982951] nix-grpc-daemon[1664]: E0000 00:00:1790180069.154921 1664 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 139.963285] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 139.987088] nix-grpc-daemon[1664]: E0000 00:00:1790180069.160714 1664 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 139.990897] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 139.994382] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 139.976379] niks3-server[1142]: 2026/09/23 16:14:28 INFO lead: acquired remote=192.168.1.4:39378 worker2 # [ 140.000079] nix-grpc-daemon[1664]: E0000 00:00:1790180069.172296 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 140.004242] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=scheduler_take_over worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.27 seconds) client: must succeed: sleep 5.0 worker2 # [ 140.011674] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52872 worker2 # [ 140.014216] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 140.021048] nix-grpc-daemon[1664]: E0000 00:00:1790180069.193709 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 140.025167] nix-grpc-daemon[1664]: E0000 00:00:1790180069.193914 1706 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 140.034397] nix-grpc-daemon[1664]: E0000 00:00:1790180069.208093 1706 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 140.043768] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 140.049247] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 140.050614] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52872 worker2 # [ 140.052247] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 140.082037] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:52888 worker2 # [ 140.292184] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker1 # [ 140.277081] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 worker2 # [ 140.297263] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:29Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 140.289682] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5) worker1 # [ 140.300652] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 2/5) worker1 # [ 140.372590] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 299 ms (attempt 3/5) worker1 # [ 140.673867] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 440 ms (attempt 4/5) worker1 # [ 141.116883] nix-grpc-daemon[1485]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 141.121129] nix-grpc-daemon[1485]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 141.138640] nix-grpc-daemon[1485]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'... worker1 # [ 141.163360] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) worker1 # [ 141.177577] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) client # [ 141.199391] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.04 seconds) worker1 # [ 145.196512] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:33Z level=info event=niks3_push_started pid=1587 client: waiting for success: ! systemctl is-active mid worker1 # [ 145.216853] nix-daemon[868]: reaped child process 1562, status = succeeded worker1 # [ 145.296793] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.543Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 145.381208] nix-daemon[868]: accepted connection from pid 1600, user nix-grpc-daemon (trusted) worker1 # [ 145.394395] nix-daemon[868]: reaped child process 1603, status = succeeded lb # [ 145.397509] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received push request method=POST path=/api/pushes worker1 # [ 145.409657] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.657Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 145.411142] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.658Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)" lb # [ 145.439325] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 145.452230] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign worker1 # [ 145.465635] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.713Z level=INFO msg="Uploading 1 narinfos" lb # [ 145.461418] niks3-server[1142]: 2026/09/23 16:14:33 INFO Signed narinfos id=50 count=1 lb # [ 145.472160] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv lb # [ 145.487178] niks3-server[1142]: 2026/09/23 16:14:33 INFO Received complete push request method=POST path=/api/pushes/50/complete lb # [ 145.490351] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls worker1 # [ 145.496908] nix-grpc-daemon[1587]: time=2026-09-23T16:14:33.744Z level=INFO msg="Upload complete. (196ms)" lb # [ 145.494803] niks3-server[1142]: 2026/09/23 16:14:33 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo worker1 # [ 145.502682] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 145.519895] nix-daemon[868]: reaped child process 1560, status = succeeded client # [ 145.558828] systemd[1]: mid.service: Deactivated successfully. client # [ 145.562114] systemd[1]: mid.service: Consumed 153ms CPU time over 8.335s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.07 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.15 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr client # [ 146.345346] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: eeb9ba1beaef4c0696d5a04d15e5ed05 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds) worker2 # [ 146.480257] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 146.491368] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # [ 146.507249] nix[1435]: don't know how to build these paths: client # [ 146.508294] nix[1435]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker2 # [ 146.513745] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=0 worker2 # [ 146.518650] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 146.525469] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 146.534932] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5) worker2 # [ 146.579341] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 2/5) worker2 # [ 146.716503] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5) worker2 # [ 146.815927] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 762 ms (attempt 4/5) lb # [ 146.874851] envoy[921]: conn peer=[2001:db8:1::4]:43236 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2914 tx=393 ms=15057 lb # [ 146.878763] envoy[921]: conn peer=[2001:db8:1::1]:58866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3878 tx=1949 ms=8169 worker2 # [ 147.594841] nix-grpc-daemon[1664]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 147.598579] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 147.616088] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 client # [ 147.629800] nix[1435]: copying 1 paths to 'lb:50051'... client # [ 147.633694] nix[1435]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'... worker2 # [ 147.630668] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 147.642669] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 147.662694] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:36Z level=info event=niks3_push_started pid=1744 worker2 # [ 147.764461] nix-grpc-daemon[1744]: time=2026-09-23T16:14:36.937Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 147.850124] nix-daemon[860]: accepted connection from pid 1757, user nix-grpc-daemon (trusted) worker2 # [ 147.862303] nix-daemon[860]: reaped child process 1760, status = succeeded lb # [ 147.847749] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received push request method=POST path=/api/pushes worker2 # [ 147.878941] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.052Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 147.880430] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.054Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)" lb # [ 147.883927] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst lb # [ 147.891205] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign lb # [ 147.898599] niks3-server[1142]: 2026/09/23 16:14:36 INFO Signed narinfos id=51 count=1 worker2 # [ 147.924733] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.098Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.908928] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls lb # [ 147.917903] niks3-server[1142]: 2026/09/23 16:14:36 INFO Received complete push request method=POST path=/api/pushes/51/complete lb # [ 147.927557] niks3-server[1142]: 2026/09/23 16:14:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo worker2 # [ 147.954073] nix-grpc-daemon[1744]: time=2026-09-23T16:14:37.127Z level=INFO msg="Upload complete. (186ms)" worker2 # [ 147.958511] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 147.963856] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 147.972353] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 147.982286] nix-daemon[860]: reaped child process 1742, status = succeeded worker2 # [ 147.985621] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # [ 147.988631] nix[1435]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 149.742865] systemd[1]: intr.service: Sent signal SIGINT to main process 1435 (nix) on client request. worker2 # [ 149.745969] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:38Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv client # [ 149.751130] nix[1435]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds) client # [ 149.759609] nix[1435]: error: interrupted by the user client # [ 149.788234] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 149.789958] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 149.792090] systemd[1]: intr.service: Consumed 135ms CPU time over 3.442s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 149.810733] nix-daemon[860]: reaped child process 1762, status = succeeded worker2 # [ 149.812455] nix-daemon[1764]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 149.820792] nix-daemon[860]: reaped child process 1764, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 3.51 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw client # [ 149.854254] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 89227e95768146f79350c4c9c148a5f5 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.03 seconds) worker1 # [ 149.962513] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58818 client # [ 149.986347] nix[1455]: don't know how to build these paths: client # [ 149.987333] nix[1455]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker2 # [ 149.989870] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:39Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=0 worker1 # [ 149.977378] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 worker2 # [ 149.994717] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:39Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 149.985507] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58818 client # [ 150.014229] nix[1455]: copying 1 paths to 'lb:50051'... client # [ 150.015149] nix[1455]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'... worker1 # [ 149.992658] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58818 worker1 # [ 149.998858] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) worker1 # [ 150.084070] nix-daemon[868]: accepted connection from pid 1631, user nix-grpc-daemon (trusted) worker1 # [ 150.097106] nix-daemon[868]: reaped child process 1634, status = succeeded lb # [ 150.099760] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received push request method=POST path=/api/pushes worker1 # [ 150.114182] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 150.116474] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.363Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)" lb # [ 150.129176] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign lb # [ 150.146111] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls lb # [ 150.155407] niks3-server[1142]: 2026/09/23 16:14:38 INFO Signed narinfos id=52 count=1 worker1 # [ 150.162238] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.409Z level=INFO msg="Uploading 1 narinfos" lb # [ 150.158196] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst lb # [ 150.170157] niks3-server[1142]: 2026/09/23 16:14:38 INFO Received complete push request method=POST path=/api/pushes/52/complete worker1 # [ 150.178303] nix-grpc-daemon[1587]: time=2026-09-23T16:14:38.425Z level=INFO msg="Upload complete. (174ms)" lb # [ 150.175582] niks3-server[1142]: 2026/09/23 16:14:38 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo worker1 # [ 150.182213] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 150.187315] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 worker1 # [ 150.198138] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) worker1 # [ 150.204144] nix-daemon[868]: reaped child process 1629, status = succeeded client # [ 150.229567] nix[1455]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051... worker1 # [ 150.207924] nix-daemon[868]: accepted connection from pid 1485, user nix-grpc-daemon (trusted) worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 152.167145] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 152.194703] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker2 # [ 152.383208] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 152.389085] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 152.411629] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=0 worker2 # [ 152.416995] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 152.422391] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 152.434150] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'... worker2 # [ 152.445842] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 152.458181] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 152.551386] nix-daemon[860]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker2 # [ 152.563932] nix-daemon[860]: reaped child process 1818, status = succeeded lb # [ 152.549990] niks3-server[1142]: 2026/09/23 16:14:40 INFO Received push request method=POST path=/api/pushes worker2 # [ 152.582445] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.755Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 152.584028] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.757Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)" lb # [ 152.584906] niks3-server[1142]: 2026/09/23 16:14:40 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign worker2 # [ 152.612064] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.785Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.588841] niks3-server[1142]: 2026/09/23 16:14:40 INFO Signed narinfos id=53 count=1 lb # [ 152.597748] niks3-server[1142]: 2026/09/23 16:14:40 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst lb # [ 152.605928] niks3-server[1142]: 2026/09/23 16:14:41 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls lb # [ 152.609532] niks3-server[1142]: 2026/09/23 16:14:41 INFO Received complete push request method=POST path=/api/pushes/53/complete worker2 # [ 152.638483] nix-grpc-daemon[1744]: time=2026-09-23T16:14:41.812Z level=INFO msg="Upload complete. (169ms)" lb # [ 152.618247] niks3-server[1142]: 2026/09/23 16:14:41 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo worker2 # [ 152.642444] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 152.648122] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 152.656555] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 152.661874] nix-daemon[860]: reaped child process 1813, status = succeeded worker2 # [ 152.665495] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051... client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker1 # [ 152.945632] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:41Z level=info event=draining client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 153.899562] nix-daemon[860]: reaped child process 1822, status = succeeded worker2 # [ 153.986820] nix-daemon[860]: accepted connection from pid 1838, user nix-grpc-daemon (trusted) worker2 # [ 153.999578] nix-daemon[860]: reaped child process 1841, status = succeeded lb # [ 153.981684] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received push request method=POST path=/api/pushes worker2 # [ 154.015700] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 154.017172] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.190Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)" lb # [ 154.023456] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv lb # [ 154.031063] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign lb # [ 154.037087] niks3-server[1142]: 2026/09/23 16:14:42 INFO Signed narinfos id=54 count=1 worker2 # [ 154.063064] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.236Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.044155] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls lb # [ 154.048080] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 154.057731] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received complete push request method=POST path=/api/pushes/54/complete lb # [ 154.063580] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo worker2 # [ 154.089501] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.263Z level=INFO msg="Upload complete. (216ms)" worker2 # [ 154.096367] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=1 worker2 # [ 154.107435] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=0 worker2 # [ 154.112656] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 154.116185] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 154.126300] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 154.129635] nix-daemon[860]: reaped child process 1820, status = succeeded worker2 # [ 154.134840] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.308Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 154.141463] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'... worker2 # [ 154.160845] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 154.246048] nix-daemon[860]: accepted connection from pid 1847, user nix-grpc-daemon (trusted) worker2 # [ 154.258911] nix-daemon[860]: reaped child process 1850, status = succeeded lb # [ 154.241865] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received push request method=POST path=/api/pushes worker2 # [ 154.273736] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.447Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 154.275200] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.448Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)" lb # [ 154.276276] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign worker2 # [ 154.304191] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.477Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.281548] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls lb # [ 154.285169] niks3-server[1142]: 2026/09/23 16:14:42 INFO Signed narinfos id=55 count=1 lb # [ 154.299209] niks3-server[1142]: 2026/09/23 16:14:42 INFO Received complete push request method=POST path=/api/pushes/55/complete worker2 # [ 154.332336] nix-grpc-daemon[1744]: time=2026-09-23T16:14:43.505Z level=INFO msg="Upload complete. (163ms)" lb # [ 154.310783] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst worker2 # [ 154.335026] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 154.315374] niks3-server[1142]: 2026/09/23 16:14:42 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo worker2 # [ 154.338768] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 154.345357] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 154.347646] nix-daemon[860]: reaped child process 1845, status = succeeded worker2 # [ 154.352722] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051... client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 155.977100] nix-daemon[860]: reaped child process 1854, status = succeeded worker2 # [ 156.047212] nix-daemon[860]: accepted connection from pid 1870, user nix-grpc-daemon (trusted) worker2 # [ 156.060570] nix-daemon[860]: reaped child process 1873, status = succeeded lb # [ 156.044657] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received push request method=POST path=/api/pushes worker2 # [ 156.076539] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.250Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 156.078082] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.251Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)" lb # [ 156.083308] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign worker2 # [ 156.112131] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.285Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.090138] niks3-server[1142]: 2026/09/23 16:14:44 INFO Signed narinfos id=56 count=1 lb # [ 156.101520] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls lb # [ 156.106401] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst lb # [ 156.112136] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv lb # [ 156.121258] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received complete push request method=POST path=/api/pushes/56/complete lb # [ 156.127672] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo worker2 # [ 156.154608] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.328Z level=INFO msg="Upload complete. (193ms)" worker2 # [ 156.165238] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=1 worker2 # [ 156.173923] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=0 worker2 # [ 156.180218] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 156.183827] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 156.193281] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 156.196569] nix-daemon[860]: reaped child process 1852, status = succeeded worker2 # [ 156.201932] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.375Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 156.207633] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'... worker2 # [ 156.223144] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 156.307306] nix-daemon[860]: accepted connection from pid 1879, user nix-grpc-daemon (trusted) worker2 # [ 156.320139] nix-daemon[860]: reaped child process 1882, status = succeeded lb # [ 156.303963] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received push request method=POST path=/api/pushes worker2 # [ 156.335645] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.509Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 156.337592] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.510Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)" lb # [ 156.333695] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign worker2 # [ 156.365730] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.539Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.342531] niks3-server[1142]: 2026/09/23 16:14:44 INFO Signed narinfos id=57 count=1 lb # [ 156.346557] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst lb # [ 156.354422] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls lb # [ 156.365128] niks3-server[1142]: 2026/09/23 16:14:44 INFO Received complete push request method=POST path=/api/pushes/57/complete worker2 # [ 156.395992] nix-grpc-daemon[1744]: time=2026-09-23T16:14:45.569Z level=INFO msg="Upload complete. (167ms)" lb # [ 156.373879] niks3-server[1142]: 2026/09/23 16:14:44 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo worker2 # [ 156.399422] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 156.404245] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 156.413673] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 156.419255] nix-daemon[860]: reaped child process 1877, status = succeeded worker2 # [ 156.424156] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051... client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv lb # [ 156.876335] envoy[921]: conn peer=[2001:db8:1::1]:54152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=3291 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 157.867293] nix-daemon[860]: reaped child process 1886, status = succeeded worker2 # [ 157.943479] nix-daemon[860]: accepted connection from pid 1902, user nix-grpc-daemon (trusted) worker2 # [ 157.956037] nix-daemon[860]: reaped child process 1905, status = succeeded lb # [ 157.939420] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received push request method=POST path=/api/pushes worker2 # [ 157.971838] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.145Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 157.973287] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.147Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)" lb # [ 157.981192] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv lb # [ 157.988972] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls lb # [ 157.995358] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign lb # [ 158.002478] niks3-server[1142]: 2026/09/23 16:14:46 INFO Signed narinfos id=58 count=1 worker2 # [ 158.028851] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.202Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.012425] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst lb # [ 158.019175] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received complete push request method=POST path=/api/pushes/58/complete lb # [ 158.028550] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo worker2 # [ 158.050570] nix-grpc-daemon[1744]: time=2026-09-23T16:14:47.224Z level=INFO msg="Upload complete. (217ms)" worker2 # [ 158.059212] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=1 worker2 # [ 158.072699] nix-daemon[860]: reaped child process 1884, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.84 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 158.429124] nix-daemon[868]: reaped child process 1638, status = succeeded worker1 # [ 158.503696] nix-daemon[868]: accepted connection from pid 1679, user nix-grpc-daemon (trusted) worker1 # [ 158.516887] nix-daemon[868]: reaped child process 1682, status = succeeded lb # [ 158.518679] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received push request method=POST path=/api/pushes worker1 # [ 158.531345] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 158.532814] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.780Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)" lb # [ 158.558599] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign lb # [ 158.563790] niks3-server[1142]: 2026/09/23 16:14:46 INFO Signed narinfos id=59 count=1 worker1 # [ 158.570991] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.818Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.575506] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 158.581112] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv lb # [ 158.588956] niks3-server[1142]: 2026/09/23 16:14:46 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls lb # [ 158.597206] niks3-server[1142]: 2026/09/23 16:14:46 INFO Received complete push request method=POST path=/api/pushes/59/complete worker1 # [ 158.605774] nix-grpc-daemon[1587]: time=2026-09-23T16:14:46.853Z level=INFO msg="Upload complete. (183ms)" lb # [ 158.603221] niks3-server[1142]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo worker1 # [ 158.608454] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58818 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 158.615984] nix-daemon[868]: reaped child process 1636, status = succeeded client # [ 158.664227] systemd[1]: sw.service: Deactivated successfully. client # [ 158.667824] systemd[1]: sw.service: Consumed 129ms CPU time over 8.812s wall clock time, 41.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 158.949287] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:47Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker1 # [ 159.951848] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:48Z level=info event=scheduler_restarting worker1 # [ 159.960671] nix-grpc-daemon[1485]: ts=2026-09-23T16:14:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 159.975846] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:52872 worker2 # [ 159.984070] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:49Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 159.971442] nix-daemon[868]: reaped child process 1541, status = succeeded worker1 # [ 160.963988] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 160.965726] systemd[1]: nix-grpc-daemon.service: Consumed 529ms CPU time over 24.130s wall clock time, 34.7M memory peak, 47.7K incoming IP traffic, 54.4K outgoing IP traffic. worker1 # [ 160.970720] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker1 # [ 160.974239] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 161.050625] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 161.054900] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 161.058121] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 161.061592] nix-grpc-daemon[1697]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 161.062943] nix-grpc-daemon[1697]: E0000 00:00:1790180089.305055 1697 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 161.065052] nix-grpc-daemon[1697]: E0000 00:00:1790180089.308779 1697 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 161.072647] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 161.076081] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 161.076922] nix-grpc-daemon[1697]: E0000 00:00:1790180089.322895 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 161.092066] nix-grpc-daemon[1697]: E0000 00:00:1790180089.336971 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 161.094218] nix-grpc-daemon[1697]: E0000 00:00:1790180089.337078 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.78 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 161.104650] nix-grpc-daemon[1697]: E0000 00:00:1790180089.352059 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 161.116867] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 161.140749] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52872 worker2 # [ 161.147594] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.41 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 161.372791] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: a0ccec8a3d2e45c2a42a044c1cdea70e client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 161.492434] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 161.497672] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 161.526386] nix[1511]: don't know how to build these paths: client # [ 161.527359] nix[1511]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv client # [ 161.528286] nix[1511]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv client # [ 161.529454] nix[1511]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv client # [ 161.530683] nix[1511]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker1 # [ 161.513249] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 161.529964] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=0 worker1 # [ 161.518078] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5) worker2 # [ 161.536790] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 161.541957] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=0 worker2 # [ 161.546312] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 161.551703] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=0 worker2 # [ 161.555175] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=0 worker2 # [ 161.558336] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker1 # [ 161.544223] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5) worker2 # [ 161.564939] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 client # [ 161.576451] nix[1511]: copying 1 paths to 'lb:50051'... client # [ 161.578726] nix[1511]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'... worker2 # [ 161.576535] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 worker1 # [ 161.564100] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5) worker2 # [ 161.585884] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 161.670950] nix-daemon[860]: accepted connection from pid 1920, user nix-grpc-daemon (trusted) worker2 # [ 161.683800] nix-daemon[860]: reaped child process 1923, status = succeeded lb # [ 161.667616] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received push request method=POST path=/api/pushes worker2 # [ 161.699473] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.872Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 161.701550] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.874Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)" worker1 # [ 161.682313] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 409 ms (attempt 4/5) lb # [ 161.698220] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls lb # [ 161.707339] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign lb # [ 161.713863] niks3-server[1142]: 2026/09/23 16:14:50 INFO Signed narinfos id=60 count=1 worker2 # [ 161.740088] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.913Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.720873] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst lb # [ 161.732655] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received complete push request method=POST path=/api/pushes/60/complete worker2 # [ 161.762149] nix-grpc-daemon[1744]: time=2026-09-23T16:14:50.935Z level=INFO msg="Upload complete. (168ms)" lb # [ 161.741438] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo worker2 # [ 161.766434] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 161.771515] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 161.781763] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 161.787124] nix-daemon[860]: reaped child process 1918, status = succeeded worker2 # [ 161.791861] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # [ 161.794835] nix[1511]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051... worker1 # [ 162.093212] nix-grpc-daemon[1697]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 162.094697] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 162.105871] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 client # [ 162.135293] nix[1511]: copying 1 paths to 'lb:50051'... worker1 # [ 162.113821] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 client # [ 162.137905] nix[1511]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'... worker1 # [ 162.121676] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 162.134982] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=info event=niks3_push_started pid=1757 worker1 # [ 162.225570] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.472Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 162.310752] nix-daemon[868]: accepted connection from pid 1770, user nix-grpc-daemon (trusted) worker1 # [ 162.321609] nix-daemon[868]: reaped child process 1773, status = succeeded lb # [ 162.326365] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received push request method=POST path=/api/pushes worker1 # [ 162.338733] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.586Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 162.340197] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.587Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)" lb # [ 162.361149] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst lb # [ 162.368496] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign worker1 # [ 162.376955] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.624Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.373192] niks3-server[1142]: 2026/09/23 16:14:50 INFO Signed narinfos id=61 count=1 lb # [ 162.385993] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls lb # [ 162.388503] niks3-server[1142]: 2026/09/23 16:14:50 INFO Received complete push request method=POST path=/api/pushes/61/complete worker1 # [ 162.396943] nix-grpc-daemon[1757]: time=2026-09-23T16:14:50.644Z level=INFO msg="Upload complete. (167ms)" lb # [ 162.394133] niks3-server[1142]: 2026/09/23 16:14:50 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo worker1 # [ 162.399251] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 162.402344] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 162.407833] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 162.412038] nix-daemon[868]: reaped child process 1755, status = succeeded worker1 # [ 162.416541] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 162.438498] nix[1511]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 162.820758] nix-daemon[860]: reaped child process 1927, status = succeeded worker2 # [ 162.899089] nix-daemon[860]: accepted connection from pid 1951, user nix-grpc-daemon (trusted) worker2 # [ 162.910665] nix-daemon[860]: reaped child process 1954, status = succeeded lb # [ 162.894708] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes worker2 # [ 162.927083] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 162.929148] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.102Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)" lb # [ 162.937280] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign lb # [ 162.941323] niks3-server[1142]: 2026/09/23 16:14:51 INFO Signed narinfos id=62 count=1 worker2 # [ 162.966936] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.140Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.948213] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls lb # [ 162.952974] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst lb # [ 162.961650] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/62/complete lb # [ 162.964936] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv lb # [ 162.969357] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo worker2 # [ 162.991696] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.165Z level=INFO msg="Upload complete. (193ms)" worker2 # [ 162.998156] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=1 worker2 # [ 163.005373] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 163.010248] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 163.018292] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 163.022395] nix-daemon[860]: reaped child process 1925, status = succeeded worker2 # [ 163.024335] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 client # [ 163.027868] nix[1511]: copying 1 paths to 'lb:50051'... client # [ 163.030317] nix[1511]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'... worker2 # [ 163.032697] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 163.116154] nix-daemon[860]: accepted connection from pid 1958, user nix-grpc-daemon (trusted) worker2 # [ 163.131297] nix-daemon[860]: reaped child process 1961, status = succeeded lb # [ 163.113221] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes worker2 # [ 163.145094] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.318Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 163.146847] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.320Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)" lb # [ 163.148248] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign lb # [ 163.152599] niks3-server[1142]: 2026/09/23 16:14:51 INFO Signed narinfos id=63 count=1 worker2 # [ 163.178151] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.351Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.160140] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls lb # [ 163.163632] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst lb # [ 163.172654] niks3-server[1142]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/63/complete worker2 # [ 163.199597] nix-grpc-daemon[1744]: time=2026-09-23T16:14:52.373Z level=INFO msg="Upload complete. (161ms)" lb # [ 163.178476] niks3-server[1142]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo worker2 # [ 163.202888] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 163.208235] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 163.215170] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 163.222457] nix-daemon[860]: reaped child process 1956, status = succeeded worker2 # [ 163.225961] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # [ 163.228891] nix[1511]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051... worker1 # [ 163.832405] nix-daemon[868]: reaped child process 1777, status = succeeded worker1 # [ 163.905363] nix-daemon[868]: accepted connection from pid 1802, user nix-grpc-daemon (trusted) worker1 # [ 163.918366] nix-daemon[868]: reaped child process 1805, status = succeeded lb # [ 163.922806] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes worker1 # [ 163.932866] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 163.934581] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.181Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)" lb # [ 163.959206] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv lb # [ 163.973365] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign worker1 # [ 163.984071] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.231Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.980794] niks3-server[1142]: 2026/09/23 16:14:52 INFO Signed narinfos id=64 count=1 lb # [ 163.985069] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls lb # [ 163.990543] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 164.001044] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/64/complete worker1 # [ 164.007784] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.255Z level=INFO msg="Upload complete. (184ms)" lb # [ 164.004923] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo worker1 # [ 164.010540] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=1 worker2 # [ 164.030991] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:53Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 164.014552] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 164.021559] nix-daemon[868]: reaped child process 1775, status = succeeded worker1 # [ 164.023282] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 164.028678] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 client # [ 164.051840] nix[1511]: copying 1 paths to 'lb:50051'... client # [ 164.053740] nix[1511]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'... worker1 # [ 164.037235] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 164.120934] nix-daemon[868]: accepted connection from pid 1809, user nix-grpc-daemon (trusted) worker1 # [ 164.133612] nix-daemon[868]: reaped child process 1812, status = succeeded lb # [ 164.136749] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes worker1 # [ 164.147720] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.149647] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.396Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)" lb # [ 164.163720] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign worker1 # [ 164.173931] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.421Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.170107] niks3-server[1142]: 2026/09/23 16:14:52 INFO Signed narinfos id=65 count=1 lb # [ 164.179976] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls lb # [ 164.187399] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst lb # [ 164.195223] niks3-server[1142]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/65/complete worker1 # [ 164.207919] nix-grpc-daemon[1757]: time=2026-09-23T16:14:52.455Z level=INFO msg="Upload complete. (165ms)" lb # [ 164.205024] niks3-server[1142]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo worker1 # [ 164.210185] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 164.213141] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 164.218436] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 164.223788] nix-daemon[868]: reaped child process 1807, status = succeeded worker1 # [ 164.227804] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 164.249746] nix[1511]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051... worker2 # [ 165.062417] nix-daemon[860]: reaped child process 1965, status = succeeded worker2 # [ 165.143415] nix-daemon[860]: accepted connection from pid 1990, user nix-grpc-daemon (trusted) worker2 # [ 165.156241] nix-daemon[860]: reaped child process 1993, status = succeeded lb # [ 165.140553] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received push request method=POST path=/api/pushes worker2 # [ 165.172125] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.345Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 165.173711] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.347Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)" lb # [ 165.181273] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign worker2 # [ 165.208839] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.382Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.186237] niks3-server[1142]: 2026/09/23 16:14:53 INFO Signed narinfos id=66 count=1 lb # [ 165.195639] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv lb # [ 165.206130] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst lb # [ 165.212121] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls lb # [ 165.216107] niks3-server[1142]: 2026/09/23 16:14:53 INFO Received complete push request method=POST path=/api/pushes/66/complete worker2 # [ 165.244191] nix-grpc-daemon[1744]: time=2026-09-23T16:14:54.417Z level=INFO msg="Upload complete. (208ms)" lb # [ 165.222898] niks3-server[1142]: 2026/09/23 16:14:53 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo worker2 # [ 165.251153] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=1 worker2 # [ 165.264538] nix-daemon[860]: reaped child process 1963, status = succeeded worker1 # [ 166.043204] nix-daemon[868]: reaped child process 1816, status = succeeded worker1 # [ 166.116936] nix-daemon[868]: accepted connection from pid 1841, user nix-grpc-daemon (trusted) worker1 # [ 166.128965] nix-daemon[868]: reaped child process 1844, status = succeeded lb # [ 166.132547] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received push request method=POST path=/api/pushes worker1 # [ 166.145084] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 166.146610] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.394Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)" lb # [ 166.165383] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv lb # [ 166.183621] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign lb # [ 166.186646] niks3-server[1142]: 2026/09/23 16:14:54 INFO Signed narinfos id=67 count=1 worker1 # [ 166.193630] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.440Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.192757] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls lb # [ 166.196713] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst lb # [ 166.208575] niks3-server[1142]: 2026/09/23 16:14:54 INFO Received complete push request method=POST path=/api/pushes/67/complete worker1 # [ 166.215734] nix-grpc-daemon[1757]: time=2026-09-23T16:14:54.463Z level=INFO msg="Upload complete. (181ms)" lb # [ 166.213221] niks3-server[1142]: 2026/09/23 16:14:54 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo worker1 # [ 166.218217] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=1 worker1 # [ 166.225441] nix-daemon[868]: reaped child process 1814, status = succeeded client # [ 166.274689] systemd[1]: burst.service: Deactivated successfully. client # [ 166.278946] systemd[1]: burst.service: Consumed 155ms CPU time over 4.904s wall clock time, 51.9M memory peak, 13.9K incoming IP traffic, 18.7K outgoing IP traffic. lb # [ 166.877501] envoy[921]: conn peer=[2001:db8:1::1]:54156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9594 tx=4361 ms=5687 lb # [ 166.881286] envoy[921]: conn peer=[2001:db8:1::1]:54154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1522 ms=8658 lb # [ 166.884724] envoy[921]: conn peer=[2001:db8:1::3]:39038 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=482 ms=22978 lb # [ 166.888325] envoy[921]: conn peer=[2001:db8:1::1]:38872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=5171 ms=4742 client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.10 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 167.485803] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36734 worker1 # [ 167.475236] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 167.533202] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=0 worker2 # [ 167.537418] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 167.541670] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=0 worker2 # [ 167.545907] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:56Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=0 worker1 # [ 167.529355] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 167.536955] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 167.544290] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 167.549561] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 167.632323] nix-daemon[868]: accepted connection from pid 1852, user nix-grpc-daemon (trusted) worker1 # [ 167.644420] nix-daemon[868]: reaped child process 1855, status = succeeded lb # [ 167.648534] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received push request method=POST path=/api/pushes worker1 # [ 167.660666] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 167.662637] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.909Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)" lb # [ 167.680453] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign lb # [ 167.685610] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst worker1 # [ 167.696034] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.943Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.691804] niks3-server[1142]: 2026/09/23 16:14:56 INFO Signed narinfos id=68 count=1 lb # [ 167.698351] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls lb # [ 167.712518] niks3-server[1142]: 2026/09/23 16:14:56 INFO Received complete push request method=POST path=/api/pushes/68/complete worker1 # [ 167.725488] nix-grpc-daemon[1757]: time=2026-09-23T16:14:55.972Z level=INFO msg="Upload complete. (171ms)" lb # [ 167.723595] niks3-server[1142]: 2026/09/23 16:14:56 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo worker1 # [ 167.727744] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 167.732030] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 167.737886] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 167.740229] nix-daemon[868]: reaped child process 1850, status = succeeded worker1 # [ 167.745216] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 168.959789] nix-daemon[868]: reaped child process 1859, status = succeeded worker1 # [ 169.034796] nix-daemon[868]: accepted connection from pid 1875, user nix-grpc-daemon (trusted) worker1 # [ 169.045564] nix-daemon[868]: reaped child process 1878, status = succeeded lb # [ 169.049941] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received push request method=POST path=/api/pushes worker1 # [ 169.062217] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 169.063730] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.311Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)" lb # [ 169.087319] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls lb # [ 169.097979] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign worker1 # [ 169.110332] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.357Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.105981] niks3-server[1142]: 2026/09/23 16:14:57 INFO Signed narinfos id=69 count=1 lb # [ 169.113630] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 169.118787] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv lb # [ 169.127690] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received complete push request method=POST path=/api/pushes/69/complete worker1 # [ 169.137901] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.385Z level=INFO msg="Upload complete. (186ms)" lb # [ 169.135551] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo worker1 # [ 169.140554] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=1 worker1 # [ 169.148399] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 169.162972] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:58Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 169.151608] nix-daemon[868]: reaped child process 1857, status = succeeded worker1 # [ 169.155805] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 169.173409] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:58Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=0 worker1 # [ 169.161332] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 169.166138] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 169.248848] nix-daemon[868]: accepted connection from pid 1882, user nix-grpc-daemon (trusted) worker1 # [ 169.261965] nix-daemon[868]: reaped child process 1885, status = succeeded lb # [ 169.264132] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received push request method=POST path=/api/pushes worker1 # [ 169.275698] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.523Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 169.277666] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.524Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)" lb # [ 169.293379] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign lb # [ 169.299708] niks3-server[1142]: 2026/09/23 16:14:57 INFO Signed narinfos id=70 count=1 worker1 # [ 169.307202] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.554Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.307947] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst lb # [ 169.317169] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls lb # [ 169.325757] niks3-server[1142]: 2026/09/23 16:14:57 INFO Received complete push request method=POST path=/api/pushes/70/complete worker1 # [ 169.338343] nix-grpc-daemon[1757]: time=2026-09-23T16:14:57.585Z level=INFO msg="Upload complete. (167ms)" lb # [ 169.336048] niks3-server[1142]: 2026/09/23 16:14:57 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo worker1 # [ 169.340403] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 169.344572] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 169.350688] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 169.353065] nix-daemon[868]: reaped child process 1880, status = succeeded worker1 # [ 169.357928] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 170.373907] nix-daemon[868]: reaped child process 1889, status = succeeded worker1 # [ 170.446764] nix-daemon[868]: accepted connection from pid 1905, user nix-grpc-daemon (trusted) worker1 # [ 170.459088] nix-daemon[868]: reaped child process 1908, status = succeeded lb # [ 170.464804] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received push request method=POST path=/api/pushes worker1 # [ 170.477326] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 170.478907] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.726Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)" lb # [ 170.508461] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls lb # [ 170.513445] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 170.522431] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign lb # [ 170.527948] niks3-server[1142]: 2026/09/23 16:14:58 INFO Signed narinfos id=71 count=1 worker1 # [ 170.534762] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.782Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.534884] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv lb # [ 170.544860] niks3-server[1142]: 2026/09/23 16:14:58 INFO Received complete push request method=POST path=/api/pushes/71/complete worker1 # [ 170.552556] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.799Z level=INFO msg="Upload complete. (187ms)" lb # [ 170.550281] niks3-server[1142]: 2026/09/23 16:14:58 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo worker1 # [ 170.555347] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=1 worker1 # [ 170.559558] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 170.579076] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:59Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 170.566328] nix-daemon[868]: reaped child process 1887, status = succeeded worker1 # [ 170.567887] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 170.586737] nix-grpc-daemon[1664]: ts=2026-09-23T16:14:59Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=0 worker1 # [ 170.573322] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 170.577932] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 170.659597] nix-daemon[868]: accepted connection from pid 1912, user nix-grpc-daemon (trusted) worker1 # [ 170.673152] nix-daemon[868]: reaped child process 1915, status = succeeded lb # [ 170.674259] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received push request method=POST path=/api/pushes worker1 # [ 170.686264] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 170.688321] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.935Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)" lb # [ 170.701630] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst lb # [ 170.712074] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign lb # [ 170.718427] niks3-server[1142]: 2026/09/23 16:14:59 INFO Signed narinfos id=72 count=1 worker1 # [ 170.725558] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.972Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.726771] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls lb # [ 170.737106] niks3-server[1142]: 2026/09/23 16:14:59 INFO Received complete push request method=POST path=/api/pushes/72/complete worker1 # [ 170.748495] nix-grpc-daemon[1757]: time=2026-09-23T16:14:58.995Z level=INFO msg="Upload complete. (166ms)" lb # [ 170.745418] niks3-server[1142]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo worker1 # [ 170.750507] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 170.754522] nix-grpc-daemon[1697]: ts=2026-09-23T16:14:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 170.760873] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 170.763212] nix-daemon[868]: reaped child process 1910, status = succeeded worker1 # [ 170.768181] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 172.183260] nix-daemon[868]: reaped child process 1919, status = succeeded worker1 # [ 172.257630] nix-daemon[868]: accepted connection from pid 1935, user nix-grpc-daemon (trusted) worker1 # [ 172.270755] nix-daemon[868]: reaped child process 1938, status = succeeded lb # [ 172.273223] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received push request method=POST path=/api/pushes worker1 # [ 172.285468] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.532Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 172.286932] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.534Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)" lb # [ 172.316173] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign worker1 # [ 172.325418] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.572Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.321724] niks3-server[1142]: 2026/09/23 16:15:00 INFO Signed narinfos id=73 count=1 lb # [ 172.332526] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv lb # [ 172.337667] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls lb # [ 172.342674] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 172.351855] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received complete push request method=POST path=/api/pushes/73/complete worker1 # [ 172.360268] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.607Z level=INFO msg="Upload complete. (185ms)" lb # [ 172.357104] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo worker1 # [ 172.363092] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=1 worker1 # [ 172.366994] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 172.386547] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:01Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.374265] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 172.376291] nix-daemon[868]: reaped child process 1917, status = succeeded worker2 # [ 172.394507] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:01Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=0 worker1 # [ 172.380891] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.628Z level=INFO msg="All 1 paths already cached" worker1 # [ 172.384992] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 172.389697] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 172.471254] nix-daemon[868]: accepted connection from pid 1942, user nix-grpc-daemon (trusted) worker1 # [ 172.483337] nix-daemon[868]: reaped child process 1945, status = succeeded lb # [ 172.487119] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received push request method=POST path=/api/pushes worker1 # [ 172.498662] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.746Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 172.500612] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.747Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)" lb # [ 172.515848] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign worker1 # [ 172.526265] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.773Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.522169] niks3-server[1142]: 2026/09/23 16:15:00 INFO Signed narinfos id=74 count=1 lb # [ 172.532285] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst lb # [ 172.539994] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls lb # [ 172.548362] niks3-server[1142]: 2026/09/23 16:15:00 INFO Received complete push request method=POST path=/api/pushes/74/complete worker1 # [ 172.559218] nix-grpc-daemon[1757]: time=2026-09-23T16:15:00.806Z level=INFO msg="Upload complete. (165ms)" lb # [ 172.556232] niks3-server[1142]: 2026/09/23 16:15:00 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo worker1 # [ 172.561423] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 172.564477] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 172.570840] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 172.574301] nix-daemon[868]: reaped child process 1940, status = succeeded worker1 # [ 172.578857] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 173.594147] nix-daemon[868]: reaped child process 1949, status = succeeded worker1 # [ 173.667849] nix-daemon[868]: accepted connection from pid 1965, user nix-grpc-daemon (trusted) worker1 # [ 173.680304] nix-daemon[868]: reaped child process 1968, status = succeeded lb # [ 173.683883] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes worker1 # [ 173.695954] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.943Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 173.698142] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.944Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)" lb # [ 173.722143] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv lb # [ 173.735234] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign lb # [ 173.738663] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls worker1 # [ 173.747170] nix-grpc-daemon[1757]: time=2026-09-23T16:15:01.994Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.744246] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=75 count=1 lb # [ 173.750323] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst lb # [ 173.762282] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/75/complete lb # [ 173.767043] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo worker1 # [ 173.774331] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.021Z level=INFO msg="Upload complete. (188ms)" worker1 # [ 173.777027] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=1 worker1 # [ 173.781268] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 173.802515] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:02Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 173.788441] nix-daemon[868]: reaped child process 1947, status = succeeded worker1 # [ 173.790315] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 173.807236] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:02Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=0 worker1 # [ 173.795901] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.043Z level=INFO msg="All 1 paths already cached" worker1 # [ 173.799839] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 173.804535] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 173.886371] nix-daemon[868]: accepted connection from pid 1972, user nix-grpc-daemon (trusted) worker1 # [ 173.899792] nix-daemon[868]: reaped child process 1975, status = succeeded lb # [ 173.900742] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes worker1 # [ 173.913024] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 173.915065] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.162Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)" lb # [ 173.930693] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign worker1 # [ 173.940101] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.187Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.936470] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=76 count=1 lb # [ 173.944768] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls lb # [ 173.955089] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst lb # [ 173.961333] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/76/complete worker1 # [ 173.972670] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.220Z level=INFO msg="Upload complete. (163ms)" lb # [ 173.970628] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo worker1 # [ 173.974846] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 173.978601] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 173.985123] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 173.987457] nix-daemon[868]: reaped child process 1970, status = succeeded worker1 # [ 173.992554] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 174.404859] nix-daemon[868]: reaped child process 1979, status = succeeded worker1 # [ 174.479252] nix-daemon[868]: accepted connection from pid 1995, user nix-grpc-daemon (trusted) worker1 # [ 174.492756] nix-daemon[868]: reaped child process 1998, status = succeeded lb # [ 174.493624] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received push request method=POST path=/api/pushes worker1 # [ 174.507069] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 174.508476] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.753Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)" lb # [ 174.534344] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst lb # [ 174.545164] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign lb # [ 174.551188] niks3-server[1142]: 2026/09/23 16:15:02 INFO Signed narinfos id=77 count=1 worker1 # [ 174.558338] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.805Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.559942] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls lb # [ 174.568758] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv lb # [ 174.576866] niks3-server[1142]: 2026/09/23 16:15:02 INFO Received complete push request method=POST path=/api/pushes/77/complete worker1 # [ 174.584910] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.832Z level=INFO msg="Upload complete. (187ms)" lb # [ 174.582624] niks3-server[1142]: 2026/09/23 16:15:02 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo worker1 # [ 174.587250] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=1 worker1 # [ 174.591946] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 174.609142] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:03Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 174.598388] nix-daemon[868]: reaped child process 1977, status = succeeded worker1 # [ 174.600206] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 174.618297] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:03Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=0 worker1 # [ 174.604304] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.851Z level=INFO msg="All 1 paths already cached" worker1 # [ 174.608102] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 174.613068] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 174.695354] nix-daemon[868]: accepted connection from pid 2002, user nix-grpc-daemon (trusted) worker1 # [ 174.709295] nix-daemon[868]: reaped child process 2005, status = succeeded lb # [ 174.710616] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received push request method=POST path=/api/pushes worker1 # [ 174.722478] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.969Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 174.724354] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.971Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)" lb # [ 174.741457] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign worker1 # [ 174.752158] nix-grpc-daemon[1757]: time=2026-09-23T16:15:02.999Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.748463] niks3-server[1142]: 2026/09/23 16:15:03 INFO Signed narinfos id=78 count=1 lb # [ 174.756243] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst lb # [ 174.762215] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls lb # [ 174.772654] niks3-server[1142]: 2026/09/23 16:15:03 INFO Received complete push request method=POST path=/api/pushes/78/complete worker1 # [ 174.785184] nix-grpc-daemon[1757]: time=2026-09-23T16:15:03.032Z level=INFO msg="Upload complete. (167ms)" lb # [ 174.782618] niks3-server[1142]: 2026/09/23 16:15:03 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo worker1 # [ 174.787670] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 174.790768] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 174.798704] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 174.800081] nix-daemon[868]: reaped child process 2000, status = succeeded worker1 # [ 174.805180] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 175.820869] nix-daemon[868]: reaped child process 2009, status = succeeded worker1 # [ 175.894943] nix-daemon[868]: accepted connection from pid 2025, user nix-grpc-daemon (trusted) worker1 # [ 175.907348] nix-daemon[868]: reaped child process 2028, status = succeeded lb # [ 175.910919] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received push request method=POST path=/api/pushes worker1 # [ 175.921186] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.168Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 175.922723] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.170Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)" lb # [ 175.946151] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv lb # [ 175.960546] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign lb # [ 175.967619] niks3-server[1142]: 2026/09/23 16:15:04 INFO Signed narinfos id=79 count=1 worker1 # [ 175.975095] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.222Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.975351] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst lb # [ 175.979282] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls lb # [ 175.990451] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received complete push request method=POST path=/api/pushes/79/complete lb # [ 175.996921] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo worker1 # [ 176.003574] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.251Z level=INFO msg="Upload complete. (192ms)" worker1 # [ 176.006202] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=1 worker1 # [ 176.009931] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 176.027522] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 176.016941] nix-daemon[868]: reaped child process 2007, status = succeeded worker1 # [ 176.018293] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 176.037136] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=0 worker1 # [ 176.023882] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.271Z level=INFO msg="All 2 paths already cached" worker1 # [ 176.028090] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 176.032810] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 176.114964] nix-daemon[868]: accepted connection from pid 2032, user nix-grpc-daemon (trusted) worker1 # [ 176.127716] nix-daemon[868]: reaped child process 2035, status = succeeded lb # [ 176.130253] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received push request method=POST path=/api/pushes worker1 # [ 176.143149] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.390Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.145216] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.392Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)" lb # [ 176.162258] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign lb # [ 176.168060] niks3-server[1142]: 2026/09/23 16:15:04 INFO Signed narinfos id=80 count=1 worker1 # [ 176.175320] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.422Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.177149] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls lb # [ 176.186297] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst lb # [ 176.194838] niks3-server[1142]: 2026/09/23 16:15:04 INFO Received complete push request method=POST path=/api/pushes/80/complete lb # [ 176.199910] niks3-server[1142]: 2026/09/23 16:15:04 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo worker1 # [ 176.208425] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.455Z level=INFO msg="Upload complete. (171ms)" worker1 # [ 176.210438] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 176.213215] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 176.221503] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 176.223361] nix-daemon[868]: reaped child process 2030, status = succeeded worker1 # [ 176.228043] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 176.640477] nix-daemon[868]: reaped child process 2039, status = succeeded worker1 # [ 176.716547] nix-daemon[868]: accepted connection from pid 2055, user nix-grpc-daemon (trusted) worker1 # [ 176.727609] nix-daemon[868]: reaped child process 2058, status = succeeded lb # [ 176.730807] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes worker1 # [ 176.739096] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.740622] nix-grpc-daemon[1757]: time=2026-09-23T16:15:04.988Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)" lb # [ 176.751634] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign lb # [ 176.753133] niks3-server[1142]: 2026/09/23 16:15:05 INFO Signed narinfos id=81 count=1 worker1 # [ 176.758321] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.005Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.757070] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls lb # [ 176.760107] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv lb # [ 176.763131] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst lb # [ 176.767981] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/81/complete worker1 # [ 176.775488] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.022Z level=INFO msg="Upload complete. (140ms)" lb # [ 176.773273] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo worker1 # [ 176.777777] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=1 worker1 # [ 176.782421] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker2 # [ 176.799907] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:05Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 176.788490] nix-daemon[868]: reaped child process 2037, status = succeeded worker1 # [ 176.789940] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 176.795466] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.042Z level=INFO msg="All 2 paths already cached" worker1 # [ 176.799507] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 176.804148] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 176.886126] nix-daemon[868]: accepted connection from pid 2062, user nix-grpc-daemon (trusted) worker1 # [ 176.899299] nix-daemon[868]: reaped child process 2065, status = succeeded lb # [ 176.900926] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes worker1 # [ 176.909696] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.156Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.911073] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.156Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)" lb # [ 176.915251] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign worker1 # [ 176.921524] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.168Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.918460] niks3-server[1142]: 2026/09/23 16:15:05 INFO Signed narinfos id=82 count=1 lb # [ 176.919483] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls lb # [ 176.926083] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst lb # [ 176.927886] niks3-server[1142]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/82/complete worker1 # [ 176.935177] nix-grpc-daemon[1757]: time=2026-09-23T16:15:05.182Z level=INFO msg="Upload complete. (126ms)" lb # [ 176.932876] niks3-server[1142]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo worker1 # [ 176.937256] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 176.940865] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 176.946938] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 176.949102] nix-daemon[868]: reaped child process 2060, status = succeeded worker1 # [ 176.954332] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 177.790378] nix-daemon[868]: reaped child process 2069, status = succeeded worker1 # [ 177.873088] nix-daemon[868]: accepted connection from pid 2085, user nix-grpc-daemon (trusted) worker1 # [ 177.885077] nix-daemon[868]: reaped child process 2088, status = succeeded lb # [ 177.887555] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received push request method=POST path=/api/pushes worker1 # [ 177.899322] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.146Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 177.900837] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.148Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)" lb # [ 177.924651] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst lb # [ 177.934134] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign worker1 # [ 177.943935] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.191Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.939543] niks3-server[1142]: 2026/09/23 16:15:06 INFO Signed narinfos id=83 count=1 lb # [ 177.948059] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv lb # [ 177.953644] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls lb # [ 177.961357] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received complete push request method=POST path=/api/pushes/83/complete lb # [ 177.970728] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo worker1 # [ 177.973957] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.221Z level=INFO msg="Upload complete. (212ms)" worker2 # [ 178.000418] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:07Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 177.980068] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=1 worker1 # [ 177.989159] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 178.000542] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 178.007191] nix-daemon[868]: reaped child process 2067, status = succeeded worker1 # [ 178.013763] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.261Z level=INFO msg="All 2 paths already cached" worker1 # [ 178.020362] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 178.026919] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 178.112765] nix-daemon[868]: accepted connection from pid 2092, user nix-grpc-daemon (trusted) worker1 # [ 178.125629] nix-daemon[868]: reaped child process 2095, status = succeeded lb # [ 178.127978] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received push request method=POST path=/api/pushes worker1 # [ 178.140823] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 178.142329] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.389Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)" lb # [ 178.159966] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign lb # [ 178.168290] niks3-server[1142]: 2026/09/23 16:15:06 INFO Signed narinfos id=84 count=1 worker1 # [ 178.175101] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.422Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.174239] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst lb # [ 178.180184] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls lb # [ 178.191286] niks3-server[1142]: 2026/09/23 16:15:06 INFO Received complete push request method=POST path=/api/pushes/84/complete worker1 # [ 178.203310] nix-grpc-daemon[1757]: time=2026-09-23T16:15:06.450Z level=INFO msg="Upload complete. (168ms)" lb # [ 178.202862] niks3-server[1142]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo worker1 # [ 178.207355] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 178.213170] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 worker1 # [ 178.224360] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 178.226775] nix-daemon[868]: reaped child process 2090, status = succeeded worker1 # [ 178.231702] nix-daemon[868]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 179.447670] nix-daemon[868]: reaped child process 2099, status = succeeded worker1 # [ 179.527135] nix-daemon[868]: accepted connection from pid 2115, user nix-grpc-daemon (trusted) worker1 # [ 179.539928] nix-daemon[868]: reaped child process 2118, status = succeeded lb # [ 179.543118] niks3-server[1142]: 2026/09/23 16:15:07 INFO Received push request method=POST path=/api/pushes worker1 # [ 179.555374] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.802Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 179.556894] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.804Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)" worker1 # [ 179.587246] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.834Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.583176] niks3-server[1142]: 2026/09/23 16:15:07 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign lb # [ 179.586589] niks3-server[1142]: 2026/09/23 16:15:07 INFO Signed narinfos id=85 count=1 lb # [ 179.597977] niks3-server[1142]: 2026/09/23 16:15:07 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst lb # [ 179.603124] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv lb # [ 179.613124] niks3-server[1142]: 2026/09/23 16:15:08 INFO Received complete push request method=POST path=/api/pushes/85/complete worker1 # [ 179.624865] nix-grpc-daemon[1757]: time=2026-09-23T16:15:07.872Z level=INFO msg="Upload complete. (186ms)" lb # [ 179.621918] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls lb # [ 179.624489] niks3-server[1142]: 2026/09/23 16:15:08 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo worker1 # [ 179.627843] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37004 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=1 worker1 # [ 179.635096] nix-daemon[868]: reaped child process 2097, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 12.35 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 179.762607] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 69eff944ceb040faa5efd13dc45be917 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 179.870334] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37004 client # [ 179.896350] nix[1581]: don't know how to build these paths: client # [ 179.897182] nix[1581]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv worker2 # [ 179.897460] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 client # [ 179.921408] nix[1581]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker worker2 # [ 179.917595] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:09Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 186.878956] envoy[921]: conn peer=[2001:db8:1::1]:59102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27047 tx=12197 ms=12177 client # [ 209.922730] nix[1581]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip} client # [ 209.931594] nix[1581]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed client # [ 209.964549] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 209.967899] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 209.972881] systemd[1]: gpu.service: Consumed 130ms CPU time over 30.208s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 43.05 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 212.081290] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:40Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker2 # [ 212.507414] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 212.513635] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 212.529824] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=0 worker2 # [ 212.535310] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 212.540836] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 212.553879] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'... worker2 # [ 212.565073] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 212.573893] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 212.670856] nix-daemon[860]: accepted connection from pid 2035, user nix-grpc-daemon (trusted) worker2 # [ 212.683389] nix-daemon[860]: reaped child process 2038, status = succeeded lb # [ 212.668582] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes worker2 # [ 212.701271] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 212.702723] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.874Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)" lb # [ 212.696169] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign worker2 # [ 212.727701] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.901Z level=INFO msg="Uploading 1 narinfos" lb # [ 212.704539] niks3-server[1142]: 2026/09/23 16:15:41 INFO Signed narinfos id=86 count=1 lb # [ 212.718557] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls lb # [ 212.727227] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst lb # [ 212.736758] niks3-server[1142]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/86/complete worker2 # [ 212.765208] nix-grpc-daemon[1744]: time=2026-09-23T16:15:41.938Z level=INFO msg="Upload complete. (178ms)" lb # [ 212.746557] niks3-server[1142]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo worker2 # [ 212.770544] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 212.776629] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 212.788140] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 212.798130] nix-daemon[860]: reaped child process 2033, status = succeeded worker2 # [ 212.801047] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051... client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 214.035839] nix-daemon[860]: reaped child process 2042, status = succeeded worker2 # [ 214.120744] nix-daemon[860]: accepted connection from pid 2058, user nix-grpc-daemon (trusted) worker2 # [ 214.132567] nix-daemon[860]: reaped child process 2061, status = succeeded lb # [ 214.115717] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received push request method=POST path=/api/pushes worker2 # [ 214.148307] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 214.150926] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.323Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)" lb # [ 214.153506] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign worker2 # [ 214.183989] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.357Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.160366] niks3-server[1142]: 2026/09/23 16:15:42 INFO Signed narinfos id=87 count=1 lb # [ 214.168431] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls lb # [ 214.172681] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv lb # [ 214.177145] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 214.182660] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received complete push request method=POST path=/api/pushes/87/complete worker2 # [ 214.210225] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.383Z level=INFO msg="Upload complete. (201ms)" lb # [ 214.189523] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo worker2 # [ 214.218409] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=1 worker2 # [ 214.227308] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=0 worker2 # [ 214.231378] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 214.235128] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 214.243348] nix-daemon[860]: reaped child process 2040, status = succeeded worker2 # [ 214.244679] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 214.250902] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.424Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 214.255914] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'... worker2 # [ 214.261836] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 214.343418] nix-daemon[860]: accepted connection from pid 2067, user nix-grpc-daemon (trusted) worker2 # [ 214.355388] nix-daemon[860]: reaped child process 2070, status = succeeded lb # [ 214.340116] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received push request method=POST path=/api/pushes worker2 # [ 214.372534] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.546Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 214.374634] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.547Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)" lb # [ 214.371186] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign worker2 # [ 214.399173] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.572Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.376116] niks3-server[1142]: 2026/09/23 16:15:42 INFO Signed narinfos id=88 count=1 lb # [ 214.381061] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst lb # [ 214.386164] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls lb # [ 214.394054] niks3-server[1142]: 2026/09/23 16:15:42 INFO Received complete push request method=POST path=/api/pushes/88/complete lb # [ 214.397838] niks3-server[1142]: 2026/09/23 16:15:42 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo worker2 # [ 214.424513] nix-grpc-daemon[1744]: time=2026-09-23T16:15:43.598Z level=INFO msg="Upload complete. (158ms)" worker2 # [ 214.427565] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 214.432257] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 214.441492] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 214.443144] nix-daemon[860]: reaped child process 2065, status = succeeded worker2 # [ 214.448149] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051... client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 215.884448] nix-daemon[860]: reaped child process 2074, status = succeeded worker2 # [ 215.968980] nix-daemon[860]: accepted connection from pid 2090, user nix-grpc-daemon (trusted) worker2 # [ 215.981811] nix-daemon[860]: reaped child process 2093, status = succeeded lb # [ 215.965955] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received push request method=POST path=/api/pushes worker2 # [ 215.997497] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.171Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 215.999294] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.172Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)" lb # [ 216.009224] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign worker2 # [ 216.037596] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.211Z level=INFO msg="Uploading 1 narinfos" lb # [ 216.015316] niks3-server[1142]: 2026/09/23 16:15:44 INFO Signed narinfos id=89 count=1 lb # [ 216.023422] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv lb # [ 216.028839] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls lb # [ 216.036726] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst lb # [ 216.047756] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received complete push request method=POST path=/api/pushes/89/complete lb # [ 216.054678] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo worker2 # [ 216.080993] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.254Z level=INFO msg="Upload complete. (222ms)" worker2 # [ 216.091265] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=1 worker2 # [ 216.099765] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=0 worker2 # [ 216.105538] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 216.110169] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 216.121566] nix-daemon[860]: reaped child process 2072, status = succeeded worker2 # [ 216.123227] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 216.130911] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.304Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 216.136461] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'... worker2 # [ 216.141380] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 216.228878] nix-daemon[860]: accepted connection from pid 2099, user nix-grpc-daemon (trusted) worker2 # [ 216.244050] nix-daemon[860]: reaped child process 2102, status = succeeded lb # [ 216.225900] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received push request method=POST path=/api/pushes worker2 # [ 216.256947] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.430Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 216.258720] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.432Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)" lb # [ 216.255974] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign lb # [ 216.260452] niks3-server[1142]: 2026/09/23 16:15:44 INFO Signed narinfos id=90 count=1 worker2 # [ 216.287364] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 216.269763] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst lb # [ 216.275872] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls lb # [ 216.289113] niks3-server[1142]: 2026/09/23 16:15:44 INFO Received complete push request method=POST path=/api/pushes/90/complete lb # [ 216.295307] niks3-server[1142]: 2026/09/23 16:15:44 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo worker2 # [ 216.324239] nix-grpc-daemon[1744]: time=2026-09-23T16:15:45.497Z level=INFO msg="Upload complete. (176ms)" worker2 # [ 216.328117] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 216.333704] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 worker2 # [ 216.346084] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) worker2 # [ 216.352924] nix-daemon[860]: reaped child process 2097, status = succeeded worker2 # [ 216.357801] nix-daemon[860]: accepted connection from pid 1664, user nix-grpc-daemon (trusted) client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051... client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv lb # [ 216.879615] envoy[921]: conn peer=[2001:db8:1::1]:36714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=30051 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 217.371808] nix-daemon[860]: reaped child process 2106, status = succeeded worker2 # [ 217.447065] nix-daemon[860]: accepted connection from pid 2122, user nix-grpc-daemon (trusted) worker2 # [ 217.458623] nix-daemon[860]: reaped child process 2125, status = succeeded lb # [ 217.442936] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes worker2 # [ 217.474366] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.647Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 217.476363] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.649Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)" lb # [ 217.484953] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign worker2 # [ 217.513891] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.687Z level=INFO msg="Uploading 1 narinfos" lb # [ 217.491144] niks3-server[1142]: 2026/09/23 16:15:45 INFO Signed narinfos id=91 count=1 lb # [ 217.499920] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls lb # [ 217.504046] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv lb # [ 217.511178] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst lb # [ 217.517570] niks3-server[1142]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/91/complete lb # [ 217.526409] niks3-server[1142]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo worker2 # [ 217.548142] nix-grpc-daemon[1744]: time=2026-09-23T16:15:46.721Z level=INFO msg="Upload complete. (183ms)" worker2 # [ 217.554654] nix-grpc-daemon[1664]: ts=2026-09-23T16:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36734 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=1 worker2 # [ 217.568909] nix-daemon[860]: reaped child process 2104, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.23 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.09 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 218.084186] nix-grpc-daemon[1697]: ts=2026-09-23T16:15:46Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.54 seconds) (finished: run the VM test script, in 219.81 seconds) test script finished in 219.98s cleanup kill QemuMachine (pid 49) client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T16:15:48Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T16:15:49Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.00 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh