tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm lb # Disk image does not exist, creating the virtualisation disk image... client # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 49) lb # Formatting '/build/vm-state-lb/tmp.buYwVSYIH9', fmt=raw size=1073741824 client: QEMU running (pid 50) worker2 # Disk image does not exist, creating the virtualisation disk image... client # Formatting '/build/vm-state-client/tmp.7qL4Dc2xYc', fmt=raw size=1073741824 lb # mke2fs 1.47.4 (6-Mar-2025) worker2: QEMU running (pid 52) worker2 # Formatting '/build/vm-state-worker2/tmp.uXJEC9kI4L', fmt=raw size=1073741824 client # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) client # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done client # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # Filesystem UUID: 935c7864-4bec-451b-9c6d-f9811e037d8e worker2 # Filesystem UUID: 267390b9-6407-422c-a794-9f1b4b2bdf9f lb # Discarding device blocks: 0/262144 done client # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: lb # Creating filesystem with 262144 4k blocks and 65536 inodes client # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 lb # Filesystem UUID: 342e5c28-2b4a-4c5a-9a1d-76ced8e4ca08 worker1: QEMU running (pid 51) worker2 # lb # Superblock backups stored on blocks: client # worker1 # Disk image does not exist, creating the virtualisation disk image... worker2 # Allocating group tables: 0/8 done client # Allocating group tables: 0/8 done worker1 # Formatting '/build/vm-state-worker1/tmp.I5xV9Cn87C', fmt=raw size=1073741824 worker2 # Writing inode tables: 0/8 done client # Writing inode tables: 0/8 done worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # Creating journal (8192 blocks): done client # Creating journal (8192 blocks): done worker1 # Discarding device blocks: 0/262144 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # client # worker1 # Filesystem UUID: 661a4feb-6ef4-4235-acef-9ac0c6c3851b worker2 # Virtualisation disk image created. client # Virtualisation disk image created. worker1 # Superblock backups stored on blocks: worker2 # Starting virtiofs daemons... client # Starting virtiofs daemons... worker1 # 32768, 98304, 163840, 229376 worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # Allocating group tables: 0/8 done (finished: start all VMs, in 0.23 seconds) worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... client: waiting for unit network-addresses-eth1.service worker1 # Writing inode tables: 0/8 done worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client: waiting for the VM to finish booting worker1 # Creating journal (8192 blocks): done worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # 32768, 98304, 163840, 229376 worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # worker1 # worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Allocating group tables: 0/8 done worker1 # Virtualisation disk image created. worker2 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Writing inode tables: 0/8 done worker1 # Starting virtiofs daemons... worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Creating journal (8192 blocks): done worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # Virtualisation disk image created. client # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # Starting virtiofs daemons... worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T04:41:17Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T04:41:17Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T04:41:17Z INFO virtiofsd] Client connected, servicing requests lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) lb # lb # client # client # lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)... lb # lb # lb # lb # client # Press Ctrl-B to configure iPXE (PCI 00:02.0)... client # client # client # client # lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker1 # worker1 # lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 lb # lb # client # Press Ctrl-B to configure iPXE (PCI 00:05.0)... client # client # worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker1 # worker1 # worker1 # worker1 # worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # worker2 # worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # worker1 # worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker2 # worker2 # worker2 # worker2 # worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker2 # worker2 # lb # Booting from ROM... client # Booting from ROM... worker1 # Booting from ROM... worker2 # Booting from ROM... lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.000000] BIOS-provided physical RAM map: lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test/init regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] BIOS-provided physical RAM map: lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved lb # [ 0.000000] NX (Execute Disable) protection: active client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.000000] APIC: Static calls initialized lb # [ 0.000000] SMBIOS 2.8 present. client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 lb # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.000000] NX (Execute Disable) protection: active worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mmcv7i4v7dgji0gb0y4pksgkj6hhjkk-nixos-system-worker1-test/init regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.000000] Hypervisor detected: KVM client # [ 0.000000] APIC: Static calls initialized worker1 # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] SMBIOS 2.8 present. lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 lb # [ 0.000001] kvm-clock: using sched offset of 459552412 cycles worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable lb # [ 0.000005] tsc: Detected 2400.012 MHz processor client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 lb # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000000] kvm-clock: using sched offset of 456535148 cycles worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved lb # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved lb # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT client # [ 0.000005] tsc: Detected 2400.012 MHz processor lb # [ 0.002749] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.000823] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.002761] Using GB pages for direct mapping worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved lb # [ 0.002900] RAMDISK: [mem 0x5e36e000-0x5ffcffff] client # [ 0.000850] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.002907] ACPI: Early table checksum verification disabled client # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.002911] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) client # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.000000] APIC: Static calls initialized worker1 # [ 0.000000] SMBIOS 2.8 present. client # [ 0.002762] Using GB pages for direct mapping lb # [ 0.002914] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002854] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 client # [ 0.002861] ACPI: Early table checksum verification disabled lb # [ 0.002919] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] DMI: Memory slots populated: 1/1 worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker1 # [ 0.000000] Hypervisor detected: KVM lb # [ 0.002926] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.002869] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 lb # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002874] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] kvm-clock: using sched offset of 455925500 cycles lb # [ 0.002931] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002880] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.002882] ACPI: FACS 0x000000005FFE0000 000040 worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zqb8x0sddns70mbblwgjmcf92snp08cl-nixos-system-worker2-test/init regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.002933] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor worker2 # [ 0.000000] BIOS-provided physical RAM map: worker1 # [ 0.000818] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002884] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002886] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved lb # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker1 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.002936] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] client # [ 0.002887] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable lb # [ 0.002937] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.002747] Using GB pages for direct mapping client # [ 0.002889] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002819] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.002937] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.002890] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker1 # [ 0.002827] ACPI: Early table checksum verification disabled worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved lb # [ 0.002938] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.002891] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.002830] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved lb # [ 0.002938] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] client # [ 0.002892] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.002834] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.002939] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.002892] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] lb # [ 0.003167] No NUMA configuration found worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker1 # [ 0.002838] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002893] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] lb # [ 0.003168] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker1 # [ 0.002844] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002893] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] lb # [ 0.003170] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker2 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.005438] Zone ranges: worker1 # [ 0.002846] ACPI: FACS 0x000000005FFE0000 000040 worker2 # [ 0.000000] APIC: Static calls initialized client # [ 0.002894] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker2 # [ 0.000000] SMBIOS 2.8 present. lb # [ 0.005439] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.003117] No NUMA configuration found worker1 # [ 0.002848] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.005441] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.003118] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] lb # [ 0.005442] Normal empty worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.002849] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.005443] Device empty client # [ 0.003121] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker2 # [ 0.000000] DMI: Memory slots populated: 1/1 lb # [ 0.005443] Movable zone start for each node client # [ 0.005768] Zone ranges: worker2 # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.002851] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.005444] Early memory node ranges client # [ 0.005769] DMA [mem 0x0000000000001000-0x0000000000ffffff] worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.005444] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.002852] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 client # [ 0.005771] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.005772] Normal empty lb # [ 0.005445] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.000001] kvm-clock: using sched offset of 480198118 cycles client # [ 0.005773] Device empty worker1 # [ 0.002854] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.005774] Movable zone start for each node lb # [ 0.005446] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.002855] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] client # [ 0.005775] Early memory node ranges lb # [ 0.005467] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.005775] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.002855] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor lb # [ 0.005746] On node 0, zone DMA: 97 pages in unavailable ranges client # [ 0.005776] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.000807] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.002856] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] lb # [ 0.033306] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.033782] ACPI: PM-Timer IO Port: 0x608 client # [ 0.005777] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.002856] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker2 # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs lb # [ 0.033795] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) client # [ 0.005797] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # [ 0.002857] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] lb # [ 0.033824] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 client # [ 0.006058] On node 0, zone DMA: 97 pages in unavailable ranges worker2 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.002857] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.034162] On node 0, zone DMA32: 40 pages in unavailable ranges worker2 # [ 0.002741] Using GB pages for direct mapping lb # [ 0.033827] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.003078] No NUMA configuration found client # [ 0.034638] ACPI: PM-Timer IO Port: 0x608 worker2 # [ 0.002833] RAMDISK: [mem 0x5e36f000-0x5ffcffff] lb # [ 0.033828] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.003079] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] client # [ 0.034652] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.002841] ACPI: Early table checksum verification disabled lb # [ 0.033830] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker1 # [ 0.003082] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker1 # [ 0.005367] Zone ranges: worker2 # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) client # [ 0.034679] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.033830] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker1 # [ 0.005368] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.034682] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker2 # [ 0.002848] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.033831] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker1 # [ 0.005370] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.034684] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.005371] Normal empty lb # [ 0.033834] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.002853] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.005372] Device empty client # [ 0.034685] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) lb # [ 0.033835] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker1 # [ 0.005372] Movable zone start for each node worker1 # [ 0.005373] Early memory node ranges worker2 # [ 0.002860] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.034686] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) lb # [ 0.033840] TSC deadline timer available worker2 # [ 0.002862] ACPI: FACS 0x000000005FFE0000 000040 lb # [ 0.033844] CPU topo: Max. logical packages: 1 worker1 # [ 0.005374] node 0: [mem 0x0000000000001000-0x000000000009efff] client # [ 0.034687] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) lb # [ 0.033845] CPU topo: Max. logical dies: 1 worker1 # [ 0.005375] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.002864] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.033845] CPU topo: Max. dies per package: 1 client # [ 0.034689] ACPI: Using ACPI (MADT) for SMP configuration information lb # [ 0.033848] CPU topo: Max. threads per core: 1 client # [ 0.034690] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker1 # [ 0.005375] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker2 # [ 0.002865] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.034695] TSC deadline timer available lb # [ 0.033849] CPU topo: Num. cores per package: 1 worker1 # [ 0.005395] On node 0, zone DMA: 1 pages in unavailable ranges client # [ 0.034700] CPU topo: Max. logical packages: 1 lb # [ 0.033849] CPU topo: Num. threads per package: 1 worker2 # [ 0.002867] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.005670] On node 0, zone DMA: 97 pages in unavailable ranges client # [ 0.034700] CPU topo: Max. logical dies: 1 lb # [ 0.033849] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs client # [ 0.034701] CPU topo: Max. dies per package: 1 worker1 # [ 0.033085] On node 0, zone DMA32: 40 pages in unavailable ranges worker2 # [ 0.002868] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.033547] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.033870] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.034704] CPU topo: Max. threads per core: 1 worker2 # [ 0.002870] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.034704] CPU topo: Num. cores per package: 1 worker1 # [ 0.033558] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) lb # [ 0.033904] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] client # [ 0.034705] CPU topo: Num. threads per package: 1 worker2 # [ 0.002871] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.033585] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.033906] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] client # [ 0.034705] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.002871] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.033588] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) lb # [ 0.033907] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.034730] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker1 # [ 0.033589] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) lb # [ 0.033909] Booting paravirtualized kernel on KVM worker2 # [ 0.002872] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.034764] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.033590] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker2 # [ 0.002872] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] lb # [ 0.033912] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns client # [ 0.034766] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker1 # [ 0.033592] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.002873] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] client # [ 0.034767] [mem 0x60000000-0xafffffff] available for PCI devices lb # [ 0.038443] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.033592] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker2 # [ 0.002873] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.034769] Booting paravirtualized kernel on KVM lb # [ 0.041235] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker2 # [ 0.003106] No NUMA configuration found worker1 # [ 0.033595] ACPI: Using ACPI (MADT) for SMP configuration information lb # [ 0.041279] kvm-guest: PV spinlocks disabled, single CPU worker2 # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker1 # [ 0.033596] ACPI: HPET id: 0x8086a201 base: 0xfed00000 client # [ 0.034773] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.033599] TSC deadline timer available worker2 # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker2 # [ 0.005446] Zone ranges: worker1 # [ 0.033603] CPU topo: Max. logical packages: 1 client # [ 0.039284] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.033604] CPU topo: Max. logical dies: 1 worker2 # [ 0.005447] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.041516] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker1 # [ 0.033604] CPU topo: Max. dies per package: 1 worker2 # [ 0.005449] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.041561] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.033608] CPU topo: Max. threads per core: 1 worker2 # [ 0.005450] Normal empty worker2 # [ 0.005451] Device empty worker1 # [ 0.033608] CPU topo: Num. cores per package: 1 lb # [ 0.041280] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.033609] CPU topo: Num. threads per package: 1 worker2 # [ 0.005452] Movable zone start for each node worker2 # [ 0.005452] Early memory node ranges worker1 # [ 0.033609] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.005453] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.033627] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() lb # [ 0.041374] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space. worker2 # [ 0.005454] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] lb # [ 0.041387] random: crng init done worker1 # [ 0.033657] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] client # [ 0.041563] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test/init regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.005455] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] lb # [ 0.041388] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker2 # [ 0.005475] On node 0, zone DMA: 1 pages in unavailable ranges worker1 # [ 0.033659] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.043551] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.005745] On node 0, zone DMA: 97 pages in unavailable ranges worker1 # [ 0.033660] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.041664] Unknown kernel command line parameters "regInfo=/nix/store/7vhifbnjabgrzv3p88abmcm4w5m6ld68-closure-info/registration", will be passed to user space. client # [ 0.041677] random: crng init done worker1 # [ 0.033661] Booting paravirtualized kernel on KVM worker2 # [ 0.033537] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.044647] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.044691] Fallback order for Node 0: 0 worker2 # [ 0.034021] ACPI: PM-Timer IO Port: 0x608 client # [ 0.041678] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker1 # [ 0.033665] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker2 # [ 0.034035] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) lb # [ 0.044694] Built 1 zonelists, mobility grouping on. Total pages: 393078 client # [ 0.043879] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.044695] Policy zone: DMA32 worker2 # [ 0.034067] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 worker1 # [ 0.038160] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 lb # [ 0.047253] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.044985] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.034069] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.040363] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 client # [ 0.045020] Fallback order for Node 0: 0 lb # [ 0.050600] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.040406] kvm-guest: PV spinlocks disabled, single CPU worker2 # [ 0.034071] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) lb # [ 0.054341] allocated 3145728 bytes of page_ext client # [ 0.045023] Built 1 zonelists, mobility grouping on. Total pages: 393078 client # [ 0.045024] Policy zone: DMA32 worker2 # [ 0.034072] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) lb # [ 0.064803] ftrace: allocating 48787 entries in 192 pages client # [ 0.048482] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.064805] ftrace: allocated 192 pages with 2 groups worker2 # [ 0.034073] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) lb # [ 0.065709] Dynamic Preempt: lazy client # [ 0.051834] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.034074] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) lb # [ 0.065845] rcu: Preemptible hierarchical RCU implementation. client # [ 0.055222] allocated 3145728 bytes of page_ext worker2 # [ 0.034076] ACPI: Using ACPI (MADT) for SMP configuration information lb # [ 0.065845] rcu: RCU event tracing is enabled. client # [ 0.065640] ftrace: allocating 48787 entries in 192 pages worker1 # [ 0.040408] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mmcv7i4v7dgji0gb0y4pksgkj6hhjkk-nixos-system-worker1-test/init regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.034077] ACPI: HPET id: 0x8086a201 base: 0xfed00000 lb # [ 0.065846] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.065642] ftrace: allocated 192 pages with 2 groups worker2 # [ 0.034083] TSC deadline timer available client # [ 0.066509] Dynamic Preempt: lazy lb # [ 0.065847] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.034087] CPU topo: Max. logical packages: 1 client # [ 0.066671] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.065848] Rude variant of Tasks RCU enabled. worker2 # [ 0.034088] CPU topo: Max. logical dies: 1 worker1 # [ 0.040510] Unknown kernel command line parameters "regInfo=/nix/store/dw847y1iw3fxd3sfgvs8sfqllmgw48g9-closure-info/registration", will be passed to user space. client # [ 0.066671] rcu: RCU event tracing is enabled. worker1 # [ 0.040524] random: crng init done lb # [ 0.065848] Tracing variant of Tasks RCU enabled. worker2 # [ 0.034088] CPU topo: Max. dies per package: 1 client # [ 0.066672] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.034091] CPU topo: Max. threads per core: 1 worker1 # [ 0.040525] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes lb # [ 0.065849] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker2 # [ 0.034092] CPU topo: Num. cores per package: 1 client # [ 0.066673] Trampoline variant of Tasks RCU enabled. lb # [ 0.065850] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.043012] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.066674] Rude variant of Tasks RCU enabled. worker2 # [ 0.034093] CPU topo: Num. threads per package: 1 client # [ 0.066674] Tracing variant of Tasks RCU enabled. worker1 # [ 0.044107] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.065906] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.034093] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker1 # [ 0.044140] Fallback order for Node 0: 0 client # [ 0.066675] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker2 # [ 0.034117] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() lb # [ 0.065908] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.044143] Built 1 zonelists, mobility grouping on. Total pages: 393078 client # [ 0.066676] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.044144] Policy zone: DMA32 worker2 # [ 0.034153] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] lb # [ 0.065909] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.046698] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.066698] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.034155] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.070334] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.050060] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.034157] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.066700] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.070620] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.053489] allocated 3145728 bytes of page_ext worker2 # [ 0.034158] Booting paravirtualized kernel on KVM worker1 # [ 0.064026] ftrace: allocating 48787 entries in 192 pages client # [ 0.066701] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.070626] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.064027] ftrace: allocated 192 pages with 2 groups worker2 # [ 0.034162] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns client # [ 0.071161] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.064886] Dynamic Preempt: lazy worker1 # [ 0.065030] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.070742] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.071448] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.038706] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 lb # [ 0.074332] Console: colour VGA+ 80x25 worker1 # [ 0.065031] rcu: RCU event tracing is enabled. worker2 # [ 0.040953] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 lb # [ 0.074336] printk: legacy console [tty0] enabled client # [ 0.071454] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.065032] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.041003] kvm-guest: PV spinlocks disabled, single CPU lb # [ 0.103961] printk: legacy console [ttyS0] enabled worker1 # [ 0.065033] Trampoline variant of Tasks RCU enabled. client # [ 0.071566] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.200382] ACPI: Core revision 20250807 worker1 # [ 0.065034] Rude variant of Tasks RCU enabled. client # [ 0.075058] Console: colour VGA+ 80x25 worker1 # [ 0.065034] Tracing variant of Tasks RCU enabled. client # [ 0.075062] printk: legacy console [tty0] enabled worker1 # [ 0.065035] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. client # [ 0.104659] printk: legacy console [ttyS0] enabled lb # [ 0.201251] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker1 # [ 0.065035] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.201345] ACPI: Core revision 20250807 lb # [ 0.202745] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.041005] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zqb8x0sddns70mbblwgjmcf92snp08cl-nixos-system-worker2-test/init regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.203718] x2apic enabled worker1 # [ 0.065090] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.202193] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns lb # [ 0.204484] APIC: Switched APIC routing to: physical x2apic worker1 # [ 0.065092] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.041101] Unknown kernel command line parameters "regInfo=/nix/store/3g4shldmj8ca1b4q39p7dmawv4im7shg-closure-info/registration", will be passed to user space. client # [ 0.203692] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.041114] random: crng init done client # [ 0.204638] x2apic enabled worker1 # [ 0.065093] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.041115] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker1 # [ 0.069500] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 lb # [ 0.206285] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.205387] APIC: Switched APIC routing to: physical x2apic worker2 # [ 0.043304] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.069792] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.044710] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.207270] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.044744] Fallback order for Node 0: 0 worker1 # [ 0.069799] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns client # [ 0.207191] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker2 # [ 0.044747] Built 1 zonelists, mobility grouping on. Total pages: 393078 worker2 # [ 0.044748] Policy zone: DMA32 lb # [ 0.208942] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker1 # [ 0.069903] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.073433] Console: colour VGA+ 80x25 worker2 # [ 0.047322] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.073436] printk: legacy console [tty0] enabled client # [ 0.208173] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.210025] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker2 # [ 0.050813] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.102964] printk: legacy console [ttyS0] enabled worker2 # [ 0.054227] allocated 3145728 bytes of page_ext lb # [ 0.211078] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker1 # [ 0.198724] ACPI: Core revision 20250807 client # [ 0.209958] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker2 # [ 0.064572] ftrace: allocating 48787 entries in 192 pages lb # [ 0.211940] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker2 # [ 0.064575] ftrace: allocated 192 pages with 2 groups client # [ 0.211042] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker2 # [ 0.065498] Dynamic Preempt: lazy worker1 # [ 0.199579] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker2 # [ 0.065640] rcu: Preemptible hierarchical RCU implementation. client # [ 0.212841] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker1 # [ 0.201047] APIC: Switch to symmetric I/O mode setup lb # [ 0.212943] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker2 # [ 0.065640] rcu: RCU event tracing is enabled. worker1 # [ 0.202014] x2apic enabled client # [ 0.213646] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker2 # [ 0.065641] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.214939] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.065643] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.202747] APIC: Switched APIC routing to: physical x2apic worker2 # [ 0.065643] Rude variant of Tasks RCU enabled. lb # [ 0.215940] Transient Scheduler Attacks: Vulnerable: No microcode client # [ 0.214713] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker2 # [ 0.065643] Tracing variant of Tasks RCU enabled. lb # [ 0.216939] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker2 # [ 0.065644] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker1 # [ 0.204535] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.215955] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl lb # [ 0.217940] Speculative Return Stack Overflow: Mitigation: Safe RET worker2 # [ 0.065645] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.216955] Transient Scheduler Attacks: Vulnerable: No microcode worker1 # [ 0.205508] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.065702] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.218672] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker2 # [ 0.065704] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.207155] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) client # [ 0.219667] Speculative Return Stack Overflow: Mitigation: Safe RET worker2 # [ 0.065705] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.208239] x86/cpu: User Mode Instruction Prevention (UMIP) activated lb # [ 0.218939] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker2 # [ 0.070502] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.209290] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 lb # [ 0.220947] Spectre V2 : Enabling IBPB for BPF worker2 # [ 0.070787] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.210837] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 lb # [ 0.221940] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.070794] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns client # [ 0.219955] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization lb # [ 0.222940] active return thunk: srso_alias_return_thunk client # [ 0.221962] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.211156] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker2 # [ 0.070906] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.074450] Console: colour VGA+ 80x25 lb # [ 0.223960] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.074454] printk: legacy console [tty0] enabled client # [ 0.222955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker1 # [ 0.213152] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.103919] printk: legacy console [ttyS0] enabled lb # [ 0.224940] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' client # [ 0.223955] active return thunk: srso_alias_return_thunk worker1 # [ 0.214153] Transient Scheduler Attacks: Vulnerable: No microcode worker2 # [ 0.200595] ACPI: Core revision 20250807 lb # [ 0.226651] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker1 # [ 0.215152] Spectre V2 : Mitigation: Enhanced / Automatic IBRS client # [ 0.224982] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' lb # [ 0.226939] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker2 # [ 0.201451] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker1 # [ 0.216152] Speculative Return Stack Overflow: Mitigation: Safe RET client # [ 0.225955] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker2 # [ 0.202927] APIC: Switch to symmetric I/O mode setup lb # [ 0.227939] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' client # [ 0.227672] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker2 # [ 0.203865] x2apic enabled lb # [ 0.228939] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' client # [ 0.228720] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker2 # [ 0.204623] APIC: Switched APIC routing to: physical x2apic lb # [ 0.229939] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' client # [ 0.229742] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' lb # [ 0.231835] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker1 # [ 0.217152] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization client # [ 0.230723] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.206509] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker1 # [ 0.219159] Spectre V2 : Enabling IBPB for BPF lb # [ 0.232648] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 client # [ 0.231730] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' lb # [ 0.233641] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.220153] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.207487] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns client # [ 0.232955] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 lb # [ 0.234633] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker1 # [ 0.221152] active return thunk: srso_alias_return_thunk client # [ 0.233955] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.209170] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) lb # [ 0.234939] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker1 # [ 0.222181] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' client # [ 0.234866] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker2 # [ 0.210253] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.223152] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' client # [ 0.235687] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.235939] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.211304] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker1 # [ 0.224152] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' client # [ 0.236668] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.212844] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker1 # [ 0.225152] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' client # [ 0.237697] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker1 # [ 0.226152] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker2 # [ 0.213171] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker1 # [ 0.227152] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.215167] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker1 # [ 0.228152] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.216167] Transient Scheduler Attacks: Vulnerable: No microcode worker1 # [ 0.229153] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker2 # [ 0.217166] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker1 # [ 0.230870] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.218112] Speculative Return Stack Overflow: Mitigation: Safe RET worker1 # [ 0.231869] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.232152] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker1 # [ 0.233152] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker1 # [ 0.234152] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.219166] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker2 # [ 0.221174] Spectre V2 : Enabling IBPB for BPF worker2 # [ 0.221940] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.223167] active return thunk: srso_alias_return_thunk worker2 # [ 0.224188] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.225167] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker2 # [ 0.226166] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker2 # [ 0.227167] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker2 # [ 0.228971] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker2 # [ 0.229984] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.231167] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.232167] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker2 # [ 0.233167] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.234167] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker2 # [ 0.235166] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker2 # [ 0.236166] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.237167] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. lb # [ 0.271546] Freeing SMP alternatives memory: 44K lb # [ 0.271943] pid_max: default: 32768 minimum: 301 lb # [ 0.273019] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.274052] landlock: Up and running. lb # [ 0.274940] Yama: becoming mindful. client # [ 0.273323] Freeing SMP alternatives memory: 44K client # [ 0.273958] pid_max: default: 32768 minimum: 301 lb # [ 0.275796] LSM support for eBPF active client # [ 0.275047] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.276077] landlock: Up and running. lb # [ 0.277118] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.276956] Yama: becoming mindful. client # [ 0.278149] LSM support for eBPF active lb # [ 0.278000] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.278903] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.281185] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.279992] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.269222] Freeing SMP alternatives memory: 44K lb # [ 0.282497] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker1 # [ 0.270005] pid_max: default: 32768 minimum: 301 lb # [ 0.282949] ... version: 2 client # [ 0.282847] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) lb # [ 0.283822] ... bit width: 48 worker1 # [ 0.270851] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.284480] ... generic counters: 6 worker1 # [ 0.271978] landlock: Up and running. worker1 # [ 0.272598] Yama: becoming mindful. lb # [ 0.284942] ... generic bitmap: 000000000000003f client # [ 0.283500] Performance Events: Fam17h+ core perfctr, AMD PMU driver. lb # [ 0.285839] ... fixed-purpose counters: 0 worker1 # [ 0.273365] LSM support for eBPF active client # [ 0.283962] ... version: 2 lb # [ 0.286528] ... fixed-purpose bitmap: 0000000000000000 client # [ 0.284780] ... bit width: 48 worker1 # [ 0.274239] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.285611] ... generic counters: 6 lb # [ 0.286942] ... value mask: 0000ffffffffffff worker1 # [ 0.275191] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.287833] ... max period: 00007fffffffffff client # [ 0.286021] ... generic bitmap: 000000000000003f client # [ 0.286959] ... fixed-purpose counters: 0 lb # [ 0.288656] ... global_ctrl mask: 000000000000003f lb # [ 0.289044] signal: max sigframe size: 3376 client # [ 0.287741] ... fixed-purpose bitmap: 0000000000000000 worker1 # [ 0.277688] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) lb # [ 0.289821] rcu: Hierarchical SRCU implementation. client # [ 0.287959] ... value mask: 0000ffffffffffff lb # [ 0.290534] rcu: Max phase no-delay instances is 400. client # [ 0.288959] ... max period: 00007fffffffffff worker1 # [ 0.278727] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker1 # [ 0.279157] ... version: 2 client # [ 0.289954] ... global_ctrl mask: 000000000000003f worker1 # [ 0.279854] ... bit width: 48 client # [ 0.291084] signal: max sigframe size: 3376 worker1 # [ 0.280155] ... generic counters: 6 client # [ 0.291966] rcu: Hierarchical SRCU implementation. worker1 # [ 0.280850] ... generic bitmap: 000000000000003f worker1 # [ 0.281160] ... fixed-purpose counters: 0 client # [ 0.292847] rcu: Max phase no-delay instances is 400. worker1 # [ 0.281864] ... fixed-purpose bitmap: 0000000000000000 lb # [ 0.295644] smp: Bringing up secondary CPUs ... worker1 # [ 0.282155] ... value mask: 0000ffffffffffff lb # [ 0.295956] smp: Brought up 1 node, 1 CPU worker1 # [ 0.283032] ... max period: 00007fffffffffff lb # [ 0.296624] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker1 # [ 0.283803] ... global_ctrl mask: 000000000000003f worker1 # [ 0.284249] signal: max sigframe size: 3376 worker2 # [ 0.274410] Freeing SMP alternatives memory: 44K worker1 # [ 0.285020] rcu: Hierarchical SRCU implementation. lb # [ 0.297092] Memory: 1454624K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved) client # [ 0.298268] smp: Bringing up secondary CPUs ... worker2 # [ 0.275159] pid_max: default: 32768 minimum: 301 worker1 # [ 0.285732] rcu: Max phase no-delay instances is 400. client # [ 0.298976] smp: Brought up 1 node, 1 CPU lb # [ 0.298215] devtmpfs: initialized worker2 # [ 0.275784] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.299029] x86/mm: Memory block size: 128MB client # [ 0.299738] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.276995] landlock: Up and running. worker2 # [ 0.277609] Yama: becoming mindful. worker2 # [ 0.278383] LSM support for eBPF active lb # [ 0.300846] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.301041] Memory: 1454636K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker2 # [ 0.279291] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.302288] devtmpfs: initialized lb # [ 0.301912] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.290761] smp: Bringing up secondary CPUs ... lb # [ 0.303017] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.280205] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.303196] x86/mm: Memory block size: 128MB worker1 # [ 0.291168] smp: Brought up 1 node, 1 CPU worker1 # [ 0.291829] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) lb # [ 0.304145] PM: RTC time: 04:41:17, date: 2026-09-23 worker2 # [ 0.283064] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.305087] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.292381] Memory: 1454580K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker1 # [ 0.293390] devtmpfs: initialized client # [ 0.305998] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.283727] Performance Events: Fam17h+ core perfctr, AMD PMU driver. lb # [ 0.307630] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.284172] ... version: 2 worker1 # [ 0.294200] x86/mm: Memory block size: 128MB client # [ 0.307066] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.284847] ... bit width: 48 client # [ 0.308292] PM: RTC time: 04:41:17, date: 2026-09-23 lb # [ 0.308434] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.285169] ... generic counters: 6 worker1 # [ 0.296066] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.286027] ... generic bitmap: 000000000000003f lb # [ 0.308964] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.286986] ... fixed-purpose counters: 0 worker1 # [ 0.297151] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.310236] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.287733] ... fixed-purpose bitmap: 0000000000000000 worker1 # [ 0.298270] pinctrl core: initialized pinctrl subsystem lb # [ 0.310955] audit: initializing netlink subsys (disabled) worker2 # [ 0.288171] ... value mask: 0000ffffffffffff client # [ 0.312117] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.299474] PM: RTC time: 04:41:17, date: 2026-09-23 worker2 # [ 0.289172] ... max period: 00007fffffffffff lb # [ 0.312291] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.313576] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.290171] ... global_ctrl mask: 000000000000003f lb # [ 0.312294] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.312944] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.313986] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.291305] signal: max sigframe size: 3376 worker2 # [ 0.292294] rcu: Hierarchical SRCU implementation. lb # [ 0.313946] audit: type=2000 audit(1790138478.464:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.303196] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.315254] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.293174] rcu: Max phase no-delay instances is 400. lb # [ 0.315944] thermal_sys: Registered thermal governor 'user_space' client # [ 0.315970] audit: initializing netlink subsys (disabled) worker1 # [ 0.304654] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.315945] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.317189] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.316945] cpuidle: using governor menu worker1 # [ 0.305178] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.317191] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.319147] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.306438] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.317962] audit: type=2000 audit(1790138478.470:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.307164] audit: initializing netlink subsys (disabled) lb # [ 0.320170] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] client # [ 0.319959] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.308299] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.299298] smp: Bringing up secondary CPUs ... client # [ 0.319960] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.320945] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.300163] smp: Brought up 1 node, 1 CPU worker1 # [ 0.308301] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.321956] PCI: Using configuration type 1 for base access client # [ 0.320926] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.309156] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.321770] cpuidle: using governor menu worker2 # [ 0.300941] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) lb # [ 0.322987] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker1 # [ 0.310083] thermal_sys: Registered thermal governor 'user_space' client # [ 0.323512] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.310863] audit: type=2000 audit(1790138478.473:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.302284] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved) client # [ 0.324207] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.312158] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.303412] devtmpfs: initialized worker1 # [ 0.312173] cpuidle: using governor menu client # [ 0.324961] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.304297] x86/mm: Memory block size: 128MB client # [ 0.325970] PCI: Using configuration type 1 for base access worker1 # [ 0.314365] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 lb # [ 0.328211] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.327074] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.306149] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.315434] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] lb # [ 0.328943] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.329911] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.316158] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.307208] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.317167] PCI: Using configuration type 1 for base access worker2 # [ 0.308254] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.318251] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.309367] PM: RTC time: 04:41:17, date: 2026-09-23 client # [ 0.334044] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.312984] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.334958] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.336943] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.323432] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.314504] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.324156] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.315197] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.316451] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.339959] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.317184] audit: initializing netlink subsys (disabled) client # [ 0.340958] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.318522] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.329155] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.318525] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.330155] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.345317] ACPI: Added _OSI(Module Device) worker2 # [ 0.319175] audit: type=2000 audit(1790138478.498:1): state=initialized audit_enabled=0 res=1 lb # [ 0.345944] ACPI: Added _OSI(Processor Device) lb # [ 0.346704] ACPI: Added _OSI(Processor Aggregator Device) worker2 # [ 0.321174] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.321176] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.322112] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.322921] cpuidle: using governor menu worker2 # [ 0.324389] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker2 # [ 0.325457] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] client # [ 0.351439] ACPI: Added _OSI(Module Device) client # [ 0.351960] ACPI: Added _OSI(Processor Device) worker2 # [ 0.326175] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry client # [ 0.352690] ACPI: Added _OSI(Processor Aggregator Device) worker1 # [ 0.340536] ACPI: Added _OSI(Module Device) worker2 # [ 0.327186] PCI: Using configuration type 1 for base access worker1 # [ 0.341157] ACPI: Added _OSI(Processor Device) worker1 # [ 0.341937] ACPI: Added _OSI(Processor Aggregator Device) lb # [ 0.355736] ACPI: 1 ACPI AML tables successfully acquired and loaded worker2 # [ 0.328411] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. lb # [ 0.359801] ACPI: Interpreter enabled lb # [ 0.360424] ACPI: PM: (supports S0 S3 S4 S5) lb # [ 0.360944] ACPI: Using IOAPIC for interrupt routing client # [ 0.360860] ACPI: 1 ACPI AML tables successfully acquired and loaded worker2 # [ 0.335493] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.363951] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug worker2 # [ 0.336171] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.364943] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.351949] ACPI: 1 ACPI AML tables successfully acquired and loaded worker2 # [ 0.337171] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.364928] ACPI: Interpreter enabled client # [ 0.365442] ACPI: PM: (supports S0 S3 S4 S5) client # [ 0.365960] ACPI: Using IOAPIC for interrupt routing lb # [ 0.368079] ACPI: Enabled 2 GPEs in block 00 to 3F worker1 # [ 0.355815] ACPI: Interpreter enabled client # [ 0.366869] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug worker1 # [ 0.356172] ACPI: PM: (supports S0 S3 S4 S5) worker1 # [ 0.356856] ACPI: Using IOAPIC for interrupt routing worker2 # [ 0.344170] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.359204] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.371958] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.360155] PCI: Using E820 reservations for host bridge windows client # [ 0.373129] ACPI: Enabled 2 GPEs in block 00 to 3F lb # [ 0.375711] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker1 # [ 0.363203] ACPI: Enabled 2 GPEs in block 00 to 3F lb # [ 0.376851] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] lb # [ 0.378029] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] lb # [ 0.379085] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker2 # [ 0.352418] ACPI: Added _OSI(Module Device) lb # [ 0.380364] PCI host bridge to bus 0000:00 worker2 # [ 0.353171] ACPI: Added _OSI(Processor Device) worker2 # [ 0.353877] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.380462] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) lb # [ 0.380948] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.381943] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.380965] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] lb # [ 0.382943] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.382041] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.371488] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.383082] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] lb # [ 0.383943] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.372161] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.384319] PCI host bridge to bus 0000:00 lb # [ 0.384943] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.373235] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] client # [ 0.384962] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.385943] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.374277] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] client # [ 0.385958] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] lb # [ 0.386943] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.375524] PCI host bridge to bus 0000:00 client # [ 0.386958] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker2 # [ 0.361776] ACPI: 1 ACPI AML tables successfully acquired and loaded worker1 # [ 0.376161] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.388076] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint client # [ 0.387958] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.377155] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.388958] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] lb # [ 0.389425] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker1 # [ 0.378155] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.389963] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.379159] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker2 # [ 0.365570] ACPI: Interpreter enabled client # [ 0.391005] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.366185] ACPI: PM: (supports S0 S3 S4 S5) worker1 # [ 0.380156] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.366916] ACPI: Using IOAPIC for interrupt routing client # [ 0.391988] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.381181] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.382156] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.367205] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.393410] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.368170] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.383143] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.384579] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.371270] ACPI: Enabled 2 GPEs in block 00 to 3F lb # [ 0.392035] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.392954] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.393907] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.395229] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] client # [ 0.396024] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.396667] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.396970] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] client # [ 0.397948] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker1 # [ 0.387222] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker2 # [ 0.379982] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) lb # [ 0.398979] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.399136] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker1 # [ 0.388168] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.399822] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker2 # [ 0.380942] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] worker1 # [ 0.389106] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.400681] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] client # [ 0.400700] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.400948] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker1 # [ 0.390340] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker2 # [ 0.383808] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] client # [ 0.402976] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.402719] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.384293] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker1 # [ 0.391885] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.403864] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker2 # [ 0.385515] PCI host bridge to bus 0000:00 client # [ 0.404718] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker2 # [ 0.386173] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.404981] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] client # [ 0.404964] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker1 # [ 0.394170] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] worker2 # [ 0.387169] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] lb # [ 0.405833] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker1 # [ 0.394986] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.406638] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] worker2 # [ 0.388170] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker1 # [ 0.395870] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] client # [ 0.406737] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.389174] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.396161] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] lb # [ 0.407550] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.390195] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] client # [ 0.408975] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker1 # [ 0.397999] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.391169] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] client # [ 0.409833] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.392170] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.410712] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.409949] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.400162] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker2 # [ 0.393181] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.410949] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker1 # [ 0.401003] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] client # [ 0.411525] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.411853] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker2 # [ 0.394620] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker1 # [ 0.401852] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.413307] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.402701] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.414125] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] lb # [ 0.414978] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] client # [ 0.414964] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] lb # [ 0.415831] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.415968] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] lb # [ 0.416684] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.405163] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] lb # [ 0.416948] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] worker1 # [ 0.406162] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] client # [ 0.417523] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.407091] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker2 # [ 0.397265] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.418928] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.418965] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.408542] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.398182] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] client # [ 0.419823] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] lb # [ 0.420823] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker2 # [ 0.399159] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] client # [ 0.420734] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.410163] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] lb # [ 0.421688] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker2 # [ 0.400351] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker1 # [ 0.410985] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.421821] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.422497] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.411861] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker2 # [ 0.401898] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.412161] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] client # [ 0.423499] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.423986] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] client # [ 0.424823] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] lb # [ 0.424964] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker1 # [ 0.414135] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.404210] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.425691] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker2 # [ 0.405152] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker1 # [ 0.415976] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] lb # [ 0.426913] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.426521] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.406045] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.416887] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] lb # [ 0.428714] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.407017] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker1 # [ 0.417725] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.427981] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] lb # [ 0.428965] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] client # [ 0.428942] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker2 # [ 0.408759] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.419167] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] lb # [ 0.430513] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.420094] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] client # [ 0.430723] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.410211] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] lb # [ 0.431924] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker2 # [ 0.411067] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] lb # [ 0.432686] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] client # [ 0.432836] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker1 # [ 0.421540] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.411917] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.433683] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker1 # [ 0.422902] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] lb # [ 0.433524] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.412709] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.423178] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] client # [ 0.434517] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.435986] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker1 # [ 0.424713] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.435938] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.436797] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker2 # [ 0.415178] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] client # [ 0.436725] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] lb # [ 0.437642] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.426167] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker2 # [ 0.416012] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker1 # [ 0.427087] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.416918] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] lb # [ 0.438516] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.438803] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.417764] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.428489] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.439979] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] client # [ 0.440977] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] client # [ 0.441850] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] lb # [ 0.440831] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.430181] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.419215] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] client # [ 0.442753] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.431015] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] lb # [ 0.441963] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker2 # [ 0.420079] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker1 # [ 0.431875] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] client # [ 0.444307] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.443556] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.420971] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.432713] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.422176] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.444526] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] client # [ 0.445976] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] lb # [ 0.445138] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.434170] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker2 # [ 0.423707] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.446965] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.434989] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.446508] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] client # [ 0.447923] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker2 # [ 0.424995] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker1 # [ 0.435870] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] lb # [ 0.447134] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.425898] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] client # [ 0.449377] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.436730] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.448557] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker2 # [ 0.426791] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.450515] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker1 # [ 0.437696] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] client # [ 0.451155] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.438373] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.428217] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] client # [ 0.452530] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker2 # [ 0.429193] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker1 # [ 0.439682] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] client # [ 0.453157] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.440346] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.430799] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.454509] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker1 # [ 0.441700] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker2 # [ 0.431975] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.432994] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker2 # [ 0.434692] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.449814] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.436121] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.451863] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker2 # [ 0.437191] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] lb # [ 0.452970] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker2 # [ 0.438710] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.454278] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO worker2 # [ 0.440184] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] client # [ 0.455190] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.455185] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.441085] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker1 # [ 0.442369] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.441904] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] client # [ 0.457525] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] lb # [ 0.457729] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] client # [ 0.458263] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint lb # [ 0.458559] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker2 # [ 0.442748] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.445730] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.459243] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO lb # [ 0.459488] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.444727] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.446446] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.460202] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint lb # [ 0.460477] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.445177] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.447456] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO worker2 # [ 0.446174] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] client # [ 0.461987] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] client # [ 0.462882] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker1 # [ 0.448413] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.447730] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.463974] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.464060] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.448725] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] lb # [ 0.465011] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker1 # [ 0.450940] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] lb # [ 0.466028] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 client # [ 0.465590] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker1 # [ 0.451162] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker2 # [ 0.449376] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.467039] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 lb # [ 0.468038] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker1 # [ 0.452541] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.450815] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] lb # [ 0.469060] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.453817] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.451623] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.470050] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.469025] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 lb # [ 0.471078] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker2 # [ 0.452721] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] client # [ 0.470066] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 lb # [ 0.471976] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 client # [ 0.471062] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 lb # [ 0.472935] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker1 # [ 0.457230] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.472054] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 lb # [ 0.473872] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker1 # [ 0.458257] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 lb # [ 0.474647] ACPI: PCI: Interrupt link GSID configured for IRQ 19 client # [ 0.473057] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker1 # [ 0.459255] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 lb # [ 0.474953] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 client # [ 0.474060] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.460253] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 lb # [ 0.475875] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 client # [ 0.475057] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 lb # [ 0.476672] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker1 # [ 0.461251] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 client # [ 0.476054] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 lb # [ 0.476957] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker1 # [ 0.462252] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 client # [ 0.476993] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker1 # [ 0.463261] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.477971] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 lb # [ 0.478993] iommu: Default domain type: Translated worker1 # [ 0.464259] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 client # [ 0.478969] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 lb # [ 0.479762] iommu: DMA domain TLB invalidation policy: lazy mode worker1 # [ 0.465189] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 client # [ 0.479947] ACPI: PCI: Interrupt link GSID configured for IRQ 19 lb # [ 0.480946] ACPI: bus type USB registered worker1 # [ 0.466114] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 client # [ 0.480701] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 lb # [ 0.481680] usbcore: registered new interface driver usbfs worker1 # [ 0.466868] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker2 # [ 0.453386] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.480970] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.481959] usbcore: registered new interface driver hub worker1 # [ 0.467171] ACPI: PCI: Interrupt link GSID configured for IRQ 19 client # [ 0.481902] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.482810] usbcore: registered new device driver usb worker2 # [ 0.455684] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker1 # [ 0.468099] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 client # [ 0.482684] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker1 # [ 0.468879] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.484474] NetLabel: Initializing worker2 # [ 0.456471] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint lb # [ 0.484943] NetLabel: domain hash size = 128 worker1 # [ 0.469171] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 client # [ 0.484077] iommu: Default domain type: Translated lb # [ 0.485682] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker1 # [ 0.470171] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 client # [ 0.484866] iommu: DMA domain TLB invalidation policy: lazy mode worker2 # [ 0.457474] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO lb # [ 0.485983] NetLabel: unlabeled traffic allowed by default client # [ 0.486002] ACPI: bus type USB registered lb # [ 0.486958] PCI: Using ACPI for IRQ routing worker2 # [ 0.458449] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint client # [ 0.486864] usbcore: registered new interface driver usbfs worker1 # [ 0.473252] iommu: Default domain type: Translated client # [ 0.487692] usbcore: registered new interface driver hub worker2 # [ 0.460089] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker1 # [ 0.474167] iommu: DMA domain TLB invalidation policy: lazy mode client # [ 0.487969] usbcore: registered new device driver usb worker2 # [ 0.460817] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker1 # [ 0.475414] ACPI: bus type USB registered client # [ 0.489667] NetLabel: Initializing worker1 # [ 0.476146] usbcore: registered new interface driver usbfs client # [ 0.489965] NetLabel: domain hash size = 128 worker2 # [ 0.461525] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker1 # [ 0.476835] usbcore: registered new interface driver hub client # [ 0.490718] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker1 # [ 0.477167] usbcore: registered new device driver usb worker2 # [ 0.462735] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.490997] NetLabel: unlabeled traffic allowed by default worker1 # [ 0.478874] NetLabel: Initializing client # [ 0.491965] PCI: Using ACPI for IRQ routing worker1 # [ 0.479162] NetLabel: domain hash size = 128 worker1 # [ 0.479970] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker1 # [ 0.480963] NetLabel: unlabeled traffic allowed by default worker1 # [ 0.481876] PCI: Using ACPI for IRQ routing worker2 # [ 0.466839] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker2 # [ 0.467280] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker2 # [ 0.468273] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker2 # [ 0.469268] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker2 # [ 0.470279] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker2 # [ 0.471266] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker2 # [ 0.472270] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker2 # [ 0.473266] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker2 # [ 0.474379] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.475187] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker2 # [ 0.476184] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker2 # [ 0.477181] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker2 # [ 0.478190] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker2 # [ 0.479186] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 worker2 # [ 0.480182] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker2 # [ 0.481182] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.483458] iommu: Default domain type: Translated worker2 # [ 0.484182] iommu: DMA domain TLB invalidation policy: lazy mode worker2 # [ 0.485442] ACPI: bus type USB registered worker2 # [ 0.486226] usbcore: registered new interface driver usbfs worker2 # [ 0.487170] usbcore: registered new interface driver hub worker2 # [ 0.488080] usbcore: registered new device driver usb worker2 # [ 0.489812] NetLabel: Initializing worker2 # [ 0.490170] NetLabel: domain hash size = 128 worker2 # [ 0.490919] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker2 # [ 0.491225] NetLabel: unlabeled traffic allowed by default worker2 # [ 0.492181] PCI: Using ACPI for IRQ routing lb # [ 0.531507] pci 0000:00:01.0: vgaarb: setting as boot VGA device lb # [ 0.531938] pci 0000:00:01.0: vgaarb: bridge control possible lb # [ 0.531938] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none lb # [ 0.531946] vgaarb: loaded lb # [ 0.532666] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 lb # [ 0.532943] hpet0: 3 comparators, 64-bit 100.000000 MHz counter lb # [ 0.536024] clocksource: Switched to clocksource kvm-clock client # [ 0.537054] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.537954] pci 0000:00:01.0: vgaarb: bridge control possible worker1 # [ 0.525735] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.537954] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.537962] vgaarb: loaded lb # [ 0.539519] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.526151] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.538710] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 lb # [ 0.540181] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.526151] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none lb # [ 0.541511] pnp: PnP ACPI init client # [ 0.538958] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker1 # [ 0.526160] vgaarb: loaded lb # [ 0.542252] ACPI: IRQ 4 override to edge(!), high(!) worker1 # [ 0.526909] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker1 # [ 0.527157] hpet0: 3 comparators, 64-bit 100.000000 MHz counter lb # [ 0.543205] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.544705] pnp: PnP ACPI: found 5 devices client # [ 0.543047] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.531251] clocksource: Switched to clocksource kvm-clock client # [ 0.546657] VFS: Disk quotas dquot_6.6.0 client # [ 0.547393] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.548808] pnp: PnP ACPI init worker1 # [ 0.534846] VFS: Disk quotas dquot_6.6.0 lb # [ 0.552234] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns client # [ 0.549656] ACPI: IRQ 4 override to edge(!), high(!) worker1 # [ 0.535548] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.553677] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.536863] pnp: PnP ACPI init client # [ 0.550626] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.554674] NET: Registered PF_INET protocol family worker1 # [ 0.537614] ACPI: IRQ 4 override to edge(!), high(!) client # [ 0.552221] pnp: PnP ACPI: found 5 devices worker1 # [ 0.538532] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.555785] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.540008] pnp: PnP ACPI: found 5 devices client # [ 0.559899] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns client # [ 0.561452] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.547612] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns client # [ 0.562456] NET: Registered PF_INET protocol family worker1 # [ 0.549005] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.549980] NET: Registered PF_INET protocol family worker2 # [ 0.536101] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.563673] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.536166] pci 0000:00:01.0: vgaarb: bridge control possible worker1 # [ 0.551142] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.536166] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker2 # [ 0.536173] vgaarb: loaded worker2 # [ 0.536930] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker2 # [ 0.537170] hpet0: 3 comparators, 64-bit 100.000000 MHz counter lb # [ 0.571937] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.573447] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.574712] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.541258] clocksource: Switched to clocksource kvm-clock lb # [ 0.576025] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.577240] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.578352] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.579639] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.545215] VFS: Disk quotas dquot_6.6.0 lb # [ 0.580767] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.545939] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.581954] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.547341] pnp: PnP ACPI init lb # [ 0.582918] NET: Registered PF_XDP protocol family worker2 # [ 0.548148] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.583757] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.549177] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved client # [ 0.580816] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.584744] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker1 # [ 0.567756] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.550731] pnp: PnP ACPI: found 5 devices lb # [ 0.585732] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] client # [ 0.582408] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.569262] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.586800] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] client # [ 0.583712] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.587878] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.570593] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.585082] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.588958] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.571937] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.586365] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.590635] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.573209] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.587495] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.574321] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.588815] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.592775] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.575611] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.589956] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.558416] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.576693] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.591219] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.594792] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.559837] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.577887] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.592195] NET: Registered PF_XDP protocol family worker2 # [ 0.560833] NET: Registered PF_INET protocol family worker1 # [ 0.578830] NET: Registered PF_XDP protocol family client # [ 0.593102] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] lb # [ 0.596688] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.579669] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.562054] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.594156] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker1 # [ 0.580663] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] lb # [ 0.598294] PCI: CLS 0 bytes, default 64 client # [ 0.595168] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker1 # [ 0.581682] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] client # [ 0.596317] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] lb # [ 0.599236] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker1 # [ 0.582791] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] client # [ 0.597405] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] lb # [ 0.600938] Trying to unpack rootfs image as initramfs... worker1 # [ 0.583888] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] client # [ 0.598492] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.584971] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] client # [ 0.600388] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.586703] ACPI: \_SB_.GSIA: Enabled at IRQ 16 client # [ 0.602726] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.589114] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.591014] ACPI: \_SB_.GSIC: Enabled at IRQ 18 client # [ 0.604995] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.592998] ACPI: \_SB_.GSID: Enabled at IRQ 19 client # [ 0.607005] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.594741] PCI: CLS 0 bytes, default 64 client # [ 0.608717] PCI: CLS 0 bytes, default 64 worker1 # [ 0.595640] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns client # [ 0.609667] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.579044] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.597403] Trying to unpack rootfs image as initramfs... client # [ 0.611406] Trying to unpack rootfs image as initramfs... worker2 # [ 0.580590] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.581931] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.583358] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.584728] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.585899] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.587217] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.588393] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.589658] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.590666] NET: Registered PF_XDP protocol family worker2 # [ 0.591548] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.592591] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.593649] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.594797] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker2 # [ 0.595937] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.597084] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker2 # [ 0.598953] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker2 # [ 0.601337] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.603561] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.605573] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.607294] PCI: CLS 0 bytes, default 64 lb # [ 0.643830] Initialise system trusted keyrings worker2 # [ 0.608288] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.610090] Trying to unpack rootfs image as initramfs... lb # [ 0.646398] workingset: timestamp_bits=40 max_order=19 bucket_order=0 client # [ 0.655084] Initialise system trusted keyrings worker1 # [ 0.643760] Initialise system trusted keyrings client # [ 0.657578] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker1 # [ 0.644695] workingset: timestamp_bits=40 max_order=19 bucket_order=0 lb # [ 0.667294] Key type asymmetric registered lb # [ 0.670347] Asymmetric key parser 'x509' registered lb # [ 0.671155] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) lb # [ 0.674467] io scheduler mq-deadline registered lb # [ 0.675207] io scheduler kyber registered lb # [ 0.679353] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.680760] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A client # [ 0.682569] Key type asymmetric registered worker1 # [ 0.669172] Key type asymmetric registered client # [ 0.683313] Asymmetric key parser 'x509' registered worker1 # [ 0.669980] Asymmetric key parser 'x509' registered lb # [ 0.687535] Linux agpgart interface v0.103 lb # [ 0.688263] ACPI: bus type drm_connector registered client # [ 0.686592] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker2 # [ 0.655560] Initialise system trusted keyrings lb # [ 0.691680] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.674444] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) client # [ 0.689572] io scheduler mq-deadline registered lb # [ 0.692791] usbserial: USB Serial support registered for generic client # [ 0.690359] io scheduler kyber registered worker2 # [ 0.658567] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker1 # [ 0.678452] io scheduler mq-deadline registered lb # [ 0.695350] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker1 # [ 0.679428] io scheduler kyber registered client # [ 0.693218] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.695350] Please enable it if your BIOS has the CPPC option. lb # [ 0.697690] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled client # [ 0.696651] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.683074] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.702496] drop_monitor: Initializing network drop monitor service lb # [ 0.703737] NET: Registered PF_INET6 protocol family worker1 # [ 0.686732] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A client # [ 0.701815] Linux agpgart interface v0.103 client # [ 0.702596] ACPI: bus type drm_connector registered lb # [ 0.706853] Segment Routing with IPv6 lb # [ 0.709373] In-situ OAM (IOAM) with IPv6 client # [ 0.705908] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.692604] Linux agpgart interface v0.103 lb # [ 0.710467] IPI shorthand broadcast: enabled client # [ 0.707035] usbserial: USB Serial support registered for generic worker1 # [ 0.693574] ACPI: bus type drm_connector registered client # [ 0.709533] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. client # [ 0.709533] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.696981] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.680458] Key type asymmetric registered client # [ 0.711827] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker2 # [ 0.681216] Asymmetric key parser 'x509' registered worker2 # [ 0.682111] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) lb # [ 0.718769] sched_clock: Marking stable (584014458, 134312978)->(788620747, -70293311) worker1 # [ 0.698347] usbserial: USB Serial support registered for generic client # [ 0.716691] drop_monitor: Initializing network drop monitor service client # [ 0.717917] NET: Registered PF_INET6 protocol family worker1 # [ 0.703644] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.687510] io scheduler mq-deadline registered worker1 # [ 0.703644] Please enable it if your BIOS has the CPPC option. worker2 # [ 0.688271] io scheduler kyber registered lb # [ 0.723552] registered taskstats version 1 client # [ 0.720987] Segment Routing with IPv6 lb # [ 0.724570] Loading compiled-in X.509 certificates worker2 # [ 0.692414] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.724546] In-situ OAM (IOAM) with IPv6 client # [ 0.725617] IPI shorthand broadcast: enabled worker2 # [ 0.693740] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.711396] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.713076] drop_monitor: Initializing network drop monitor service worker2 # [ 0.698818] Linux agpgart interface v0.103 worker1 # [ 0.716545] NET: Registered PF_INET6 protocol family worker2 # [ 0.699641] ACPI: bus type drm_connector registered worker1 # [ 0.719703] Segment Routing with IPv6 worker1 # [ 0.720436] In-situ OAM (IOAM) with IPv6 client # [ 0.733811] sched_clock: Marking stable (599013206, 134500728)->(805725211, -72211277) worker2 # [ 0.702910] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.704026] usbserial: USB Serial support registered for generic worker1 # [ 0.723663] IPI shorthand broadcast: enabled client # [ 0.738723] registered taskstats version 1 client # [ 0.739714] Loading compiled-in X.509 certificates worker2 # [ 0.707405] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. lb # [ 0.743341] Demotion targets for Node 0: null worker2 # [ 0.707405] Please enable it if your BIOS has the CPPC option. lb # [ 0.744229] Key type .fscrypt registered worker2 # [ 0.709764] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled lb # [ 0.747331] Key type fscrypt-provisioning registered lb # [ 0.748290] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.749227] ima: Allocated hash algorithm: sha1 worker1 # [ 0.731962] sched_clock: Marking stable (598013882, 133352743)->(805681532, -74314907) worker2 # [ 0.714547] drop_monitor: Initializing network drop monitor service lb # [ 0.750022] ima: No architecture policies found worker2 # [ 0.715719] NET: Registered PF_INET6 protocol family lb # [ 0.752339] PM: Magic number: 10:302:667 worker1 # [ 0.735500] registered taskstats version 1 worker1 # [ 0.736529] Loading compiled-in X.509 certificates worker2 # [ 0.718896] Segment Routing with IPv6 lb # [ 0.753974] RAS: Correctable Errors collector initialized. worker2 # [ 0.719600] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.722706] IPI shorthand broadcast: enabled client # [ 0.758524] Demotion targets for Node 0: null client # [ 0.759462] Key type .fscrypt registered lb # [ 0.763003] clk: Disabling unused clocks worker2 # [ 0.728865] sched_clock: Marking stable (594014623, 134372398)->(805979190, -77592169) client # [ 0.761529] Key type fscrypt-provisioning registered client # [ 0.762494] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.766365] PM: genpd: Disabling unused power domains client # [ 0.763448] ima: Allocated hash algorithm: sha1 worker2 # [ 0.735613] registered taskstats version 1 client # [ 0.767564] ima: No architecture policies found worker1 # [ 0.754386] Demotion targets for Node 0: null worker2 # [ 0.736619] Loading compiled-in X.509 certificates client # [ 0.768578] PM: Magic number: 10:302:667 client # [ 0.770254] RAS: Correctable Errors collector initialized. worker1 # [ 0.757424] Key type .fscrypt registered worker1 # [ 0.758119] Key type fscrypt-provisioning registered worker1 # [ 0.759179] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.761397] ima: Allocated hash algorithm: sha1 worker1 # [ 0.762305] ima: No architecture policies found worker1 # [ 0.764382] PM: Magic number: 10:302:667 worker1 # [ 0.766141] RAS: Correctable Errors collector initialized. client # [ 0.782212] clk: Disabling unused clocks client # [ 0.782983] PM: genpd: Disabling unused power domains worker2 # [ 0.753062] Demotion targets for Node 0: null worker2 # [ 0.756443] Key type .fscrypt registered worker2 # [ 0.757134] Key type fscrypt-provisioning registered worker2 # [ 0.758126] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.777209] clk: Disabling unused clocks worker2 # [ 0.760404] ima: Allocated hash algorithm: sha1 worker2 # [ 0.761207] ima: No architecture policies found worker1 # [ 0.779388] PM: genpd: Disabling unused power domains worker2 # [ 0.763399] PM: Magic number: 10:302:667 worker2 # [ 0.765028] RAS: Correctable Errors collector initialized. worker2 # [ 0.774075] clk: Disabling unused clocks worker2 # [ 0.778420] PM: genpd: Disabling unused power domains lb # [ 0.940281] Freeing initrd memory: 29064K lb # [ 0.943590] Freeing unused decrypted memory: 2028K lb # [ 0.946224] Freeing unused kernel image (initmem) memory: 3652K lb # [ 0.947345] Write protecting the kernel read-only data: 32768k lb # [ 0.949210] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.950797] Freeing unused kernel image (rodata/data gap) memory: 720K client # [ 0.954554] Freeing initrd memory: 29056K client # [ 0.957860] Freeing unused decrypted memory: 2028K client # [ 0.960634] Freeing unused kernel image (initmem) memory: 3652K client # [ 0.961769] Write protecting the kernel read-only data: 32768k worker1 # [ 0.949231] Freeing initrd memory: 29056K client # [ 0.963721] Freeing unused kernel image (text/rodata gap) memory: 1184K client # [ 0.965297] Freeing unused kernel image (rodata/data gap) memory: 720K worker1 # [ 0.952465] Freeing unused decrypted memory: 2028K worker1 # [ 0.955114] Freeing unused kernel image (initmem) memory: 3652K worker1 # [ 0.956191] Write protecting the kernel read-only data: 32768k worker1 # [ 0.958061] Freeing unused kernel image (text/rodata gap) memory: 1184K worker1 # [ 0.959558] Freeing unused kernel image (rodata/data gap) memory: 720K worker2 # [ 0.946273] Freeing initrd memory: 29060K worker2 # [ 0.949527] Freeing unused decrypted memory: 2028K worker2 # [ 0.952192] Freeing unused kernel image (initmem) memory: 3652K worker2 # [ 0.953293] Write protecting the kernel read-only data: 32768k worker2 # [ 0.955207] Freeing unused kernel image (text/rodata gap) memory: 1184K worker2 # [ 0.956746] Freeing unused kernel image (rodata/data gap) memory: 720K lb # [ 1.001859] x86/mm: Checked W+X mappings: passed, no W+X pages found. lb # [ 1.002985] Run /init as init process lb # [ 1.013929] systemd[1]: Inserted module 'autofs4' client # [ 1.016316] x86/mm: Checked W+X mappings: passed, no W+X pages found. client # [ 1.017483] Run /init as init process worker1 # [ 1.010323] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker1 # [ 1.011437] Run /init as init process client # [ 1.028442] systemd[1]: Inserted module 'autofs4' lb # [ 1.033821] fuse: init (API version 7.45) worker1 # [ 1.022421] systemd[1]: Inserted module 'autofs4' lb # [ 1.041176] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker2 # [ 1.007551] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker2 # [ 1.008737] Run /init as init process lb # [ 1.043798] ACPI: \_SB_.GSIH: Enabled at IRQ 23 lb # [ 1.046787] ACPI: \_SB_.GSIE: Enabled at IRQ 20 lb # [ 1.049111] ACPI: \_SB_.GSIF: Enabled at IRQ 21 client # [ 1.048611] fuse: init (API version 7.45) lb # [ 1.054687] virtiofs virtio5: discovered new tag: nix-store worker2 # [ 1.019917] systemd[1]: Inserted module 'autofs4' lb # [ 1.056260] virtiofs virtio5: virtio_fs_setup_dax: No cache capability client # [ 1.056184] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.042537] fuse: init (API version 7.45) client # [ 1.058684] ACPI: \_SB_.GSIH: Enabled at IRQ 23 lb # [ 1.064823] virtiofs virtio6: discovered new tag: shared client # [ 1.061803] ACPI: \_SB_.GSIE: Enabled at IRQ 20 lb # [ 1.066596] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 1.063820] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.050033] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.052548] ACPI: \_SB_.GSIH: Enabled at IRQ 23 lb # [ 1.070078] virtiofs virtio7: discovered new tag: xchg lb # [ 1.071654] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 1.055537] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.069379] virtiofs virtio5: discovered new tag: nix-store client # [ 1.071052] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker1 # [ 1.057527] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker2 # [ 1.039930] fuse: init (API version 7.45) worker1 # [ 1.062837] virtiofs virtio5: discovered new tag: nix-store client # [ 1.077922] virtiofs virtio6: discovered new tag: shared worker1 # [ 1.064426] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker2 # [ 1.047306] ACPI: \_SB_.GSIG: Enabled at IRQ 22 client # [ 1.079810] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.049889] ACPI: \_SB_.GSIH: Enabled at IRQ 23 client # [ 1.083979] virtiofs virtio7: discovered new tag: xchg worker2 # [ 1.052805] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.085443] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 1.054773] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.072345] virtiofs virtio6: discovered new tag: shared worker1 # [ 1.074002] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 1.077170] virtiofs virtio7: discovered new tag: xchg worker2 # [ 1.059826] virtiofs virtio5: discovered new tag: nix-store worker1 # [ 1.078645] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 1.061444] virtiofs virtio5: virtio_fs_setup_dax: No cache capability lb # [ 1.096555] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.068759] virtiofs virtio6: discovered new tag: shared worker2 # [ 1.070515] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.075594] virtiofs virtio7: discovered new tag: xchg client # [ 1.108296] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.077205] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 1.102655] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.102689] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.433290] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.448817] systemd[1]: Detected virtualization kvm. lb # [ 1.451669] systemd[1]: Detected architecture x86-64. lb # [ 1.454426] systemd[1]: Running in initrd. client # [ 1.444614] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.457635] systemd[1]: Initializing machine ID from random generator. client # [ 1.456714] systemd[1]: Detected virtualization kvm. worker1 # [ 1.438428] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.461517] systemd[1]: Hostname set to . client # [ 1.458854] systemd[1]: Detected architecture x86-64. worker1 # [ 1.445260] systemd[1]: Detected virtualization kvm. client # [ 1.461173] systemd[1]: Running in initrd. worker1 # [ 1.447437] systemd[1]: Detected architecture x86-64. client # [ 1.462346] systemd[1]: Initializing machine ID from random generator. client # [ 1.463575] systemd[1]: Hostname set to . worker1 # [ 1.449531] systemd[1]: Running in initrd. worker1 # [ 1.452044] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.454963] systemd[1]: Hostname set to . worker2 # [ 1.439289] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.451331] systemd[1]: Detected virtualization kvm. worker2 # [ 1.453730] systemd[1]: Detected architecture x86-64. worker2 # [ 1.455866] systemd[1]: Running in initrd. worker2 # [ 1.458327] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.461225] systemd[1]: Hostname set to . client # [ 1.657007] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.671953] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.695164] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.684063] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.699834] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.701038] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.702492] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.671606] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.703724] systemd[1]: Reached target Path Units. client # [ 1.704605] systemd[1]: Reached target Slice Units. client # [ 1.705481] systemd[1]: Reached target Swaps. client # [ 1.706303] systemd[1]: Reached target Timer Units. client # [ 1.707298] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.708540] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.711948] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.709687] systemd[1]: Listening on Journal Sockets. client # [ 1.710697] systemd[1]: Listening on udev Control Socket. client # [ 1.711759] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.715734] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.712781] systemd[1]: Reached target Socket Units. lb # [ 1.716910] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.714633] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.718380] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.719511] systemd[1]: Reached target Path Units. lb # [ 1.720424] systemd[1]: Reached target Slice Units. lb # [ 1.721286] systemd[1]: Reached target Swaps. client # [ 1.718337] systemd[1]: Starting Load Kernel Module configfs... lb # [ 1.722098] systemd[1]: Reached target Timer Units. lb # [ 1.723068] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.724339] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.725532] systemd[1]: Listening on Journal Sockets. lb # [ 1.726513] systemd[1]: Listening on udev Control Socket. lb # [ 1.727572] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.728641] systemd[1]: Reached target Socket Units. lb # [ 1.730557] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.734410] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.727987] systemd[1]: Starting Journal Service... worker2 # [ 1.709050] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.713693] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.714919] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.716404] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.717536] systemd[1]: Reached target Path Units. worker2 # [ 1.718447] systemd[1]: Reached target Slice Units. worker2 # [ 1.719299] systemd[1]: Reached target Swaps. lb # [ 1.743524] systemd[1]: Starting Journal Service... worker2 # [ 1.720129] systemd[1]: Reached target Timer Units. worker2 # [ 1.721093] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.739312] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.753691] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.722429] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.723556] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.724569] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.725618] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.743741] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.726592] systemd[1]: Reached target Socket Units. client # [ 1.757621] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.745021] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.728267] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.746520] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.747765] systemd[1]: Reached target Path Units. worker1 # [ 1.748679] systemd[1]: Reached target Slice Units. worker1 # [ 1.749567] systemd[1]: Reached target Swaps. worker1 # [ 1.750414] systemd[1]: Reached target Timer Units. worker2 # [ 1.732503] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.764296] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.751407] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.769520] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.752667] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.753815] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.754828] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.755922] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.773441] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.756943] systemd[1]: Reached target Socket Units. worker1 # [ 1.758875] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.777023] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.780411] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.763447] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.778474] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.741997] systemd[1]: Starting Journal Service... client # [ 1.784967] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 1.793572] systemd-journald[66]: Collecting audit messages is disabled. client # [ 1.791919] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.795487] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.766575] systemd[1]: Starting Load Kernel Modules... client # [ 1.797920] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.773460] systemd[1]: Starting Journal Service... lb # [ 1.803002] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 1.770485] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.808780] systemd[1]: Finished Load Kernel Module configfs. worker2 # [ 1.777159] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.809597] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.814780] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.799013] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.803503] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.789942] systemd-journald[66]: Collecting audit messages is disabled. client # [ 1.821682] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.791098] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.826419] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.795238] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 1.813602] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.829602] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.802765] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.837505] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.825921] systemd-journald[66]: Collecting audit messages is disabled. worker2 # [ 1.808782] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.827882] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.848552] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.833054] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 1.849048] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.818450] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.840786] systemd[1]: Finished Load Kernel Module configfs. client # [ 1.858708] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.845823] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker2 # [ 1.831112] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.869931] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.870038] systemd[1]: Finished Load Kernel Modules. worker1 # [ 1.858451] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.840559] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.878548] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.879615] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.869884] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.890404] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.855911] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.879404] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.899540] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.864639] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.898827] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.766324] systemd-modules-load[68]: Inserted module 'dm_mod' client # [ 1.768379] systemd-modules-load[68]: Inserted module 'virtio_balloon' client # [ 1.904742] systemd[1]: Started Journal Service. client # [ 1.771297] systemd-modules-load[68]: Inserted module 'virtio_gpu' worker2 # [ 1.881484] systemd[1]: Finished Load Kernel Modules. lb # [ 1.783331] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.918892] systemd[1]: Started Journal Service. client # [ 1.782188] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.902448] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # lb # [ 1.788358] systemd-modules-load[68]: Inserted module 'virtio_balloon' client # [ 1.785073] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.786040] systemd[1]: Reached target Local File Systems. lb # [ 1.789426] systemd-modules-load[68]: Inserted module 'virtio_gpu' worker2 # [ 1.889583] systemd[1]: Starting Apply Kernel Variables... client # [ 1.788975] systemd[1]: Starting Create System Files and Directories... lb # [ 1.793235] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.794321] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.911588] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.795551] systemd[1]: Reached target Local File Systems. client # [ 1.795335] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.800286] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.923442] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.905501] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.806468] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.909607] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.813823] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.914512] systemd[1]: Reached target Local File Systems. worker1 # [ 1.934586] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.782542] systemd-modules-load[67]: Inserted module 'dm_mod' worker2 # [ 1.786052] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker2 # [ 1.787120] systemd-modules-load[67]: Inserted module 'virtio_gpu' client # [ 1.819102] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.923288] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.930579] systemd[1]: Started Journal Service. lb # [ 1.833621] systemd[1]: Finished Create System Files and Directories. worker1 # [ 1.818387] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.953490] systemd[1]: Started Journal Service. worker2 # [ 1.803181] systemd[1]: Finished Apply Kernel Variables. worker1 # worker1 # [ 1.823316] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.824392] systemd-modules-load[67]: Inserted module 'virtio_gpu' client # [ 1.842388] systemd-udevd[82]: Using default interface naming scheme 'v261'. worker2 # [ 1.813149] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.832083] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.840114] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.841178] systemd[1]: Reached target Local File Systems. worker1 # [ 1.845033] systemd[1]: Starting Create System Files and Directories... lb # [ 1.864343] systemd-udevd[76]: Using default interface naming scheme 'v261'. worker1 # [ 1.851677] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.866245] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.858831] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.841572] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.849313] systemd-udevd[74]: Using default interface naming scheme 'v261'. lb # [ 1.889092] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.888123] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.871878] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.908667] systemd-udevd[80]: Using default interface naming scheme 'v261'. client # [ 1.921726] systemd[1]: Finished Coldplug All udev Devices. client # [ 1.922685] systemd[1]: Reached target System Initialization. client # [ 1.923511] systemd[1]: Reached target Basic System. lb # [ 1.944181] systemd[1]: Finished Coldplug All udev Devices. lb # [ 1.945075] systemd[1]: Reached target System Initialization. lb # [ 1.945856] systemd[1]: Reached target Basic System. worker1 # [ 1.932077] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.933589] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 1.934450] systemd[1]: Reached target System Initialization. worker2 # [ 1.935265] systemd[1]: Reached target Basic System. worker1 # [ 1.988171] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 1.989710] systemd[1]: Reached target System Initialization. worker1 # [ 1.990591] systemd[1]: Reached target Basic System. client # [ 2.247579] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.248377] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 client # [ 2.252714] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.256842] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.259537] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 client # [ 2.260381] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.261528] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.263405] usb usb1: Product: EHCI Host Controller client # [ 2.264524] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd client # [ 2.265245] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.269866] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.268676] hub 1-0:1.0: USB hub found client # [ 2.269789] hub 1-0:1.0: 6 ports detected client # [ 2.272106] uhci_hcd 0000:00:1d.0: UHCI Host Controller lb # [ 2.275740] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.272890] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.275371] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.279443] serio: i8042 KBD port at 0x60,0x64 irq 1 worker2 # [ 2.249261] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.285259] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.283627] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.289095] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.286695] uhci_hcd 0000:00:1d.0: detected 2 ports lb # [ 2.289943] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 client # [ 2.287447] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.291929] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.289789] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 2.258170] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 lb # [ 2.296210] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.298368] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker2 # [ 2.264763] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 2.300822] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.298287] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.301955] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.302949] serio: i8042 AUX port at 0x60,0x64 irq 12 client # [ 2.299397] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.306332] usb usb1: Product: EHCI Host Controller lb # [ 2.306990] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd lb # [ 2.308338] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.309997] hub 1-0:1.0: USB hub found worker2 # [ 2.276459] ehci-pci 0000:00:1d.7: EHCI Host Controller lb # [ 2.312230] hub 1-0:1.0: 6 ports detected worker2 # [ 2.277168] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker2 # [ 2.278897] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 worker2 # [ 2.281259] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.316246] uhci_hcd 0000:00:1d.0: UHCI Host Controller client # [ 2.313607] usb usb2: Product: UHCI Host Controller client # [ 2.314288] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.286435] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker2 # [ 2.287453] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 worker2 # [ 2.288585] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.289991] usb usb1: Product: EHCI Host Controller client # [ 2.322397] usb usb2: SerialNumber: 0000:00:1d.0 worker2 # [ 2.290773] serio: i8042 AUX port at 0x60,0x64 irq 12 worker2 # [ 2.291591] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd lb # [ 2.326781] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker2 # [ 2.292328] usb usb1: SerialNumber: 0000:00:1d.7 client # [ 2.324163] hub 2-0:1.0: USB hub found worker1 # [ 2.311141] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 2.294836] hub 1-0:1.0: USB hub found worker2 # [ 2.297434] hub 1-0:1.0: 6 ports detected worker1 # [ 2.317721] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.332808] hub 2-0:1.0: 2 ports detected lb # [ 2.339416] uhci_hcd 0000:00:1d.0: detected 2 ports client # [ 2.339403] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker2 # [ 2.307900] uhci_hcd 0000:00:1d.0: UHCI Host Controller client # [ 2.340156] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 lb # [ 2.344127] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker1 # [ 2.326677] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 2.332041] ehci-pci 0000:00:1d.7: EHCI Host Controller worker1 # [ 2.332817] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker1 # [ 2.334078] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.355001] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.356155] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.218921] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.321540] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker1 # [ 2.340392] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker1 # [ 2.341279] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.221312] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.343069] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.344232] usb usb1: Product: EHCI Host Controller worker1 # [ 2.344902] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.346298] serio: i8042 KBD port at 0x60,0x64 irq 1 worker1 # [ 2.346994] serio: i8042 AUX port at 0x60,0x64 irq 12 worker1 # [ 2.348726] usb usb1: SerialNumber: 0000:00:1d.7 client # [ 2.227298] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.229255] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.333448] uhci_hcd 0000:00:1d.0: detected 2 ports worker1 # [ 2.352144] hub 1-0:1.0: USB hub found worker2 # [ 2.334257] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker1 # [ 2.353151] hub 1-0:1.0: 6 ports detected lb # [ 2.370804] usb usb2: Product: UHCI Host Controller worker2 # [ 2.340133] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.341338] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.364556] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker1 # [ 2.365271] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 lb # [ 2.383866] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.385555] uhci_hcd 0000:00:1d.1: detected 2 ports lb # [ 2.389426] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.386317] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker1 # [ 2.376873] uhci_hcd 0000:00:1d.0: detected 2 ports client # [ 2.256323] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.394409] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker2 # [ 2.361131] usb usb2: Product: UHCI Host Controller client # [ 2.258643] systemd[1]: Finished Virtual Console Setup. lb # [ 2.397783] SCSI subsystem initialized lb # [ 2.400938] hub 2-0:1.0: USB hub found worker2 # [ 2.366743] SCSI subsystem initialized lb # [ 2.267630] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.384977] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker2 # [ 2.369408] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.370143] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.401475] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.277331] (udev-worker)[88]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.394581] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.409188] SCSI subsystem initialized worker1 # [ 2.395693] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.410478] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.381548] hub 2-0:1.0: USB hub found client # [ 2.415606] usb usb3: Product: UHCI Host Controller lb # [ 2.419734] hub 2-0:1.0: 2 ports detected client # [ 2.416336] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.250257] (udev-worker)[87]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.252662] systemd[1]: Starting Virtual Console Setup... client # [ 2.419039] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker2 # [ 2.388141] hub 2-0:1.0: 2 ports detected lb # [ 2.288892] (udev-worker)[85]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.290897] (udev-worker)[85]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.423113] usb usb3: SerialNumber: 0000:00:1d.1 worker2 # [ 2.393591] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker1 # [ 2.411895] usb usb2: Product: UHCI Host Controller client # [ 2.430563] hub 3-0:1.0: USB hub found client # [ 2.297836] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 2.433234] hub 3-0:1.0: 2 ports detected client # [ 2.299512] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.420651] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.304070] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.428431] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.443151] uhci_hcd 0000:00:1d.2: UHCI Host Controller client # [ 2.443924] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.312831] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.314575] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.416799] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker1 # [ 2.302640] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.437454] SCSI subsystem initialized client # [ 2.452543] uhci_hcd 0000:00:1d.2: detected 2 ports lb # [ 2.455890] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.453298] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 lb # [ 2.456681] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 client # [ 2.458434] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.445492] hub 2-0:1.0: USB hub found worker2 # [ 2.292467] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.459986] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.313296] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.462544] usb usb4: Product: UHCI Host Controller client # [ 2.463258] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.469525] uhci_hcd 0000:00:1d.1: detected 2 ports client # [ 2.466544] usb usb4: SerialNumber: 0000:00:1d.2 worker1 # [ 2.453517] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.468086] hub 4-0:1.0: USB hub found lb # [ 2.336989] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 2.301773] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.304067] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.470644] hub 4-0:1.0: 2 ports detected lb # [ 2.340230] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.457536] hub 2-0:1.0: 2 ports detected lb # [ 2.342142] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.325654] (udev-worker)[88]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.478366] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker1 # [ 2.328320] (udev-worker)[88]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.342224] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 2.480413] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.312263] systemd[1]: Finished Virtual Console Setup. lb # [ 2.481586] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.448944] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 client # [ 2.348656] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 2.491700] usb usb3: Product: UHCI Host Controller worker2 # [ 2.457114] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.458252] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 lb # [ 2.493383] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.494119] usb usb3: SerialNumber: 0000:00:1d.1 worker2 # [ 2.464468] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.500440] hub 3-0:1.0: USB hub found worker2 # [ 2.465692] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.484413] uhci_hcd 0000:00:1d.1: UHCI Host Controller lb # [ 2.502422] hub 3-0:1.0: 2 ports detected worker1 # [ 2.485169] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 lb # [ 2.510285] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.511035] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker2 # [ 2.342277] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 2.343427] systemd[1]: Reached target Initrd Root Device. client # [ 2.510624] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.479409] usb usb3: Product: UHCI Host Controller worker2 # [ 2.480110] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.515475] uhci_hcd 0000:00:1d.2: detected 2 ports worker1 # [ 2.365292] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.347222] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 2.382652] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 2.518443] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.370135] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.505006] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.487395] usb usb3: SerialNumber: 0000:00:1d.1 lb # [ 2.389493] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 2.489712] hub 3-0:1.0: USB hub found lb # [ 2.525405] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.491122] hub 3-0:1.0: 2 ports detected lb # [ 2.526679] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.511517] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 client # [ 2.527313] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker1 # [ 2.380899] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 2.528425] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.531906] usb usb4: Product: UHCI Host Controller worker1 # [ 2.381950] systemd[1]: Reached target Initrd Root Device. client # [ 2.529761] ahci 0000:00:1f.2: flags: 64bit ncq only client # [ 2.532815] scsi host0: ahci worker1 # [ 2.386068] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.502427] uhci_hcd 0000:00:1d.2: UHCI Host Controller client # [ 2.534597] scsi host1: ahci worker2 # [ 2.503185] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 client # [ 2.535812] scsi host2: ahci worker1 # [ 2.521493] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.539151] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.540036] usb usb4: SerialNumber: 0000:00:1d.2 worker1 # [ 2.522631] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.538183] scsi host3: ahci lb # [ 2.541971] hub 4-0:1.0: USB hub found client # [ 2.539896] scsi host4: ahci lb # [ 2.544420] hub 4-0:1.0: 2 ports detected worker2 # [ 2.509291] uhci_hcd 0000:00:1d.2: detected 2 ports client # [ 2.541425] scsi host5: ahci worker2 # [ 2.510338] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 client # [ 2.542472] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.529370] usb usb3: Product: UHCI Host Controller worker1 # [ 2.530071] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.544163] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 client # [ 2.545368] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 lb # [ 2.549372] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.514760] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.546592] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 client # [ 2.547761] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.534523] usb usb3: SerialNumber: 0000:00:1d.1 client # [ 2.548979] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker2 # [ 2.517423] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.520475] usb usb4: Product: UHCI Host Controller worker2 # [ 2.521257] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.540495] hub 3-0:1.0: USB hub found worker1 # [ 2.542381] hub 3-0:1.0: 2 ports detected worker2 # [ 2.393498] systemd-fsck[107]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 2.549439] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker2 # [ 2.531934] usb usb4: SerialNumber: 0000:00:1d.2 worker1 # [ 2.550151] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.567768] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.400956] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.553623] uhci_hcd 0000:00:1d.2: detected 2 ports worker2 # [ 2.537880] hub 4-0:1.0: USB hub found worker2 # [ 2.540686] hub 4-0:1.0: 2 ports detected worker2 # [ 2.541426] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 2.578060] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.579528] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.565872] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 lb # [ 2.583416] scsi host0: ahci lb # [ 2.584679] scsi host1: ahci worker1 # [ 2.433935] systemd-fsck[107]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 2.586835] scsi host2: ahci lb # [ 2.588391] scsi host3: ahci lb # [ 2.590009] scsi host4: ahci worker1 # [ 2.440526] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 2.591776] scsi host5: ahci lb # [ 2.593228] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 lb # [ 2.594533] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 lb # [ 2.595807] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.578432] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.579766] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.597003] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 lb # [ 2.598179] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 lb # [ 2.599334] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker1 # [ 2.583338] usb usb4: Product: UHCI Host Controller worker1 # [ 2.584262] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.586694] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.573170] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.574281] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker1 # [ 2.592666] hub 4-0:1.0: USB hub found worker1 # [ 2.593230] hub 4-0:1.0: 2 ports detected worker1 # [ 2.595522] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.578053] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.598940] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.586485] scsi host0: ahci worker2 # [ 2.588012] scsi host1: ahci worker2 # [ 2.589892] scsi host2: ahci worker2 # [ 2.591801] scsi host3: ahci worker1 # [ 2.610010] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker2 # [ 2.593425] scsi host4: ahci worker2 # [ 2.595013] scsi host5: ahci worker1 # [ 2.613653] ahci 0000:00:1f.2: flags: 64bit ncq only worker2 # [ 2.596160] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker2 # [ 2.597880] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker2 # [ 2.599025] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker2 # [ 2.600318] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker2 # [ 2.601528] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker2 # [ 2.602718] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.639949] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.630122] scsi host0: ahci client # [ 2.642855] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker1 # [ 2.632907] scsi host1: ahci client # [ 2.646095] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.634636] scsi host2: ahci client # [ 2.647882] usb 1-1: Manufacturer: QEMU worker1 # [ 2.636418] scsi host3: ahci client # [ 2.649599] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker1 # [ 2.638165] scsi host4: ahci worker1 # [ 2.639818] scsi host5: ahci worker1 # [ 2.641265] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.642558] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker1 # [ 2.643762] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.644922] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker1 # [ 2.646102] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.647323] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 lb # [ 2.680302] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 client # [ 2.680037] hid: raw HID events driver (C) Jiri Kosina lb # [ 2.683172] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 lb # [ 2.686454] usb 1-1: Product: QEMU USB Tablet lb # [ 2.688227] usb 1-1: Manufacturer: QEMU lb # [ 2.690307] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker2 # [ 2.670973] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker2 # [ 2.673948] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker2 # [ 2.676577] usb 1-1: Product: QEMU USB Tablet worker2 # [ 2.678068] usb 1-1: Manufacturer: QEMU worker2 # [ 2.679470] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 lb # [ 2.724498] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.705893] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 2.725490] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.728410] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker1 # [ 2.731630] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.733464] usb 1-1: Manufacturer: QEMU worker1 # [ 2.735102] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker1 # [ 2.758024] hid: raw HID events driver (C) Jiri Kosina client # [ 2.639085] systemd[1]: Mounting /sysroot... lb # [ 2.659561] systemd[1]: Mounting /sysroot... worker2 # [ 2.654260] systemd[1]: Mounting /sysroot... worker1 # [ 2.690504] systemd[1]: Mounting /sysroot... client # [ 2.862126] ata6: SATA link down (SStatus 0 SControl 300) client # [ 2.863022] ata2: SATA link down (SStatus 0 SControl 300) client # [ 2.864106] ata1: SATA link down (SStatus 0 SControl 300) client # [ 2.865040] ata4: SATA link down (SStatus 0 SControl 300) client # [ 2.865953] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) client # [ 2.866981] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 client # [ 2.867831] ata3.00: applying bridge limits client # [ 2.868558] ata5: SATA link down (SStatus 0 SControl 300) client # [ 2.869525] ata3.00: configured for UDMA/100 client # [ 2.870428] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.910850] ata2: SATA link down (SStatus 0 SControl 300) lb # [ 2.913227] ata1: SATA link down (SStatus 0 SControl 300) lb # [ 2.916122] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) lb # [ 2.919064] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 lb # [ 2.921852] ata3.00: applying bridge limits lb # [ 2.924206] ata4: SATA link down (SStatus 0 SControl 300) lb # [ 2.927291] ata5: SATA link down (SStatus 0 SControl 300) lb # [ 2.929964] ata6: SATA link down (SStatus 0 SControl 300) lb # [ 2.932409] ata3.00: configured for UDMA/100 lb # [ 2.935072] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 client # [ 2.938614] usbcore: registered new interface driver usbhid client # [ 2.939378] usbhid: USB HID core driver worker2 # [ 2.918998] ata1: SATA link down (SStatus 0 SControl 300) client # [ 2.952582] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker2 # [ 2.921284] ata2: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.923994] ata6: SATA link down (SStatus 0 SControl 300) client # [ 2.957957] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker2 # [ 2.926329] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker2 # [ 2.928852] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker2 # [ 2.930969] ata3.00: applying bridge limits worker2 # [ 2.932896] ata4: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.935174] ata5: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.937466] ata3.00: configured for UDMA/100 worker1 # [ 2.956141] ata1: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.939739] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker1 # [ 2.963581] ata2: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.965708] ata5: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.968123] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker1 # [ 2.970689] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker1 # [ 2.972800] ata3.00: applying bridge limits worker1 # [ 2.974976] ata6: SATA link down (SStatus 0 SControl 300) client # [ 2.990096] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker1 # [ 2.977294] ata4: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.979629] ata3.00: configured for UDMA/100 worker1 # [ 2.982055] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 client # [ 2.998629] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 3.000188] EXT4-fs (vda): mounted filesystem 935c7864-4bec-451b-9c6d-f9811e037d8e r/w with ordered data mode. Quota mode: none. client # [ 2.869651] systemd[1]: Mounted /sysroot. lb # [ 3.008600] usbcore: registered new interface driver usbhid client # [ 2.871160] systemd[1]: Reached target Initrd Root File System. client # [ 2.875339] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.015812] usbhid: USB HID core driver lb # [ 3.028747] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 client # [ 2.891526] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 2.894793] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 2.897117] systemd[1]: Reloading... lb # [ 3.033851] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker2 # [ 3.021517] usbcore: registered new interface driver usbhid worker2 # [ 3.022293] usbhid: USB HID core driver worker1 # [ 3.057504] usbcore: registered new interface driver usbhid worker1 # [ 3.058451] usbhid: USB HID core driver worker2 # [ 3.040138] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker2 # [ 3.042096] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 lb # [ 3.077623] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker2 # [ 3.049046] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 3.086613] EXT4-fs (vda): mounted filesystem 342e5c28-2b4a-4c5a-9a1d-76ced8e4ca08 r/w with ordered data mode. Quota mode: none. lb # [ 3.088569] cdrom: Uniform CD-ROM driver Revision: 3.20 lb # [ 2.957504] systemd[1]: Mounted /sysroot. lb # [ 2.958919] systemd[1]: Reached target Initrd Root File System. lb # [ 2.962493] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.068484] cdrom: Uniform CD-ROM driver Revision: 3.20 worker1 # [ 3.092952] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 3.095072] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker2 # [ 3.080582] EXT4-fs (vda): mounted filesystem 267390b9-6407-422c-a794-9f1b4b2bdf9f r/w with ordered data mode. Quota mode: none. client # [ 2.978727] systemd[1]: Reloading finished in 82 ms. worker1 # [ 3.099817] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 2.982942] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 2.950666] systemd[1]: Mounted /sysroot. worker2 # [ 2.952161] systemd[1]: Reached target Initrd Root File System. lb # [ 2.987961] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 2.990150] systemd[1]: Reloading... client # [ 2.987075] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker2 # [ 2.955759] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.110547] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 2.991226] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... worker1 # [ 3.113399] EXT4-fs (vda): mounted filesystem 661a4feb-6ef4-4235-acef-9ac0c6c3851b r/w with ordered data mode. Quota mode: none. client # [ 2.995125] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 2.996451] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 2.997786] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 2.986847] systemd[1]: Mounted /sysroot. worker1 # [ 2.989960] systemd[1]: Reached target Initrd Root File System. worker2 # [ 2.972243] systemd-sysroot-fstab-check[141]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 2.992284] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 2.977093] systemd[1]: Reload requested from client PID 141 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 2.978657] systemd[1]: Reloading... worker1 # [ 3.007990] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.011166] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.012735] systemd[1]: Reloading... lb # [ 3.082447] systemd[1]: Reloading finished in 92 ms. lb # [ 3.090745] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.058092] systemd[1]: Reloading finished in 81 ms. lb # [ 3.093249] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... lb # [ 3.098814] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.100319] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.066506] systemd-sysroot-fstab-check[141]: Requesting initrd-fs.target/start/replace... lb # [ 3.102228] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.071166] systemd-sysroot-fstab-check[141]: Requesting swap.target/start/replace... worker2 # [ 3.075347] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.076721] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.077734] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.104055] systemd[1]: Reloading finished in 89 ms. worker1 # [ 3.111644] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.114147] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... worker1 # [ 3.120612] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.122111] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.123570] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.640732] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.643745] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.648139] systemd[1]: Mounting /sysroot/run... client # [ 3.652138] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.656247] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.661193] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.667180] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.674984] systemd[1]: Mounting /sysroot/run... lb # [ 3.683391] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.681140] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.685712] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.654109] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.687949] systemd[1]: Mounted /sysroot/run. client # [ 3.688807] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.658804] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.691611] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 3.661786] systemd[1]: Mounting /sysroot/run... lb # [ 3.697363] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.694812] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.665779] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.669329] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.705083] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.691975] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.706455] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.708490] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 3.700189] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.708366] systemd[1]: Mounting /sysroot/run... client # [ 3.728743] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.729872] systemd[1]: Reached target Initrd File Systems. lb # [ 3.734740] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.731490] systemd[1]: Starting Find NixOS closure... worker1 # [ 3.719803] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.735136] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 3.740882] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.742684] systemd[1]: Mounted /sysroot/run. lb # [ 3.743581] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.744505] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.730617] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.748709] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.714302] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.716072] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.716799] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.717477] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.720381] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.753133] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 3.759437] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.724692] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.761100] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.758413] systemd[1]: Finished Find NixOS closure. client # [ 3.759733] systemd[1]: Reached target Initrd Default Target. client # [ 3.761976] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 3.735436] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.736911] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.740142] systemd[1]: Mounting /sysroot/nix/store... client # [ 3.774852] systemd[1]: Stopped target Initrd Default Target. client # [ 3.776194] systemd[1]: Stopped target Basic System. client # [ 3.777072] systemd[1]: Stopped target Initrd Root Device. client # [ 3.778027] systemd[1]: Stopped target Path Units. client # [ 3.779153] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.780150] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 3.781321] systemd[1]: Stopped target Slice Units. client # [ 3.782212] systemd[1]: Stopped target Socket Units. client # [ 3.783157] systemd[1]: Stopped target System Initialization. client # [ 3.785081] systemd[1]: Stopped target Swaps. client # [ 3.785685] systemd[1]: Stopped target Timer Units. client # [ 3.786447] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 3.774323] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.787251] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 3.788031] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 3.788939] systemd[1]: Stopped Find NixOS closure. worker1 # [ 3.777301] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.790206] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.778099] systemd[1]: Mounted /sysroot/run. client # [ 3.791110] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.760196] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.792250] systemd[1]: Stopped Apply Kernel Variables. client # [ 3.793329] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 3.762077] systemd[1]: Reached target Initrd File Systems. worker1 # [ 3.781722] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.763336] systemd[1]: Starting Find NixOS closure... client # [ 3.795231] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 3.782781] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.796249] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 3.797433] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 3.798715] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 3.767142] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 3.786675] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.800048] systemd[1]: Stopped Create System Files and Directories. client # [ 3.801096] systemd[1]: Stopped target Local File Systems. client # [ 3.801963] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 3.803165] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 3.805183] systemd[1]: Stopped Coldplug All udev Devices. client # [ 3.807619] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 3.808914] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 3.809892] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 3.799221] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.800703] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.802957] systemd[1]: Mounting /sysroot/nix/store... client # [ 3.815766] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 3.818927] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 3.787375] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 3.792384] systemd[1]: Finished Find NixOS closure. worker2 # [ 3.793691] systemd[1]: Reached target Initrd Default Target. client # [ 3.826722] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 3.796184] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 3.828827] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 3.829971] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.831262] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.832668] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 3.834138] systemd[1]: Closed udev Control Socket. client # [ 3.835651] systemd[1]: Starting Cleanup udev Database... client # [ 3.836486] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 3.838202] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 3.826092] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.839240] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 3.827518] systemd[1]: Reached target Initrd File Systems. client # [ 3.840426] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 3.841480] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 3.829237] systemd[1]: Starting Find NixOS closure... client # [ 3.842787] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 3.812299] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 3.832344] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 3.813697] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 3.815976] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 3.816965] systemd[1]: Stopped target Basic System. worker2 # [ 3.818174] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 3.819077] systemd[1]: Stopped target Path Units. worker2 # [ 3.819801] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 3.822132] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 3.823165] systemd[1]: Stopped target Slice Units. worker2 # [ 3.823822] systemd[1]: Stopped target Socket Units. worker2 # [ 3.824557] systemd[1]: Stopped target System Initialization. worker2 # [ 3.825373] systemd[1]: Stopped target Swaps. client # [ 3.856764] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 3.825976] systemd[1]: Stopped target Timer Units. client # [ 3.857963] systemd[1]: Finished Cleanup udev Database. worker2 # [ 3.827077] systemd[1]: dbus.socket: Deactivated successfully. client # [ 3.859163] systemd[1]: Reached target Switch Root. worker2 # [ 3.828117] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 3.860628] systemd[1]: Starting NixOS Activation... worker2 # [ 3.828919] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 3.830191] systemd[1]: Stopped Find NixOS closure. worker2 # [ 3.831103] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.832114] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.833101] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.834100] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 3.835097] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 3.854129] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 3.836235] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 3.837296] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 3.839049] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 3.839967] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 3.841528] systemd[1]: Stopped target Local File Systems. worker1 # [ 3.860602] systemd[1]: Finished Find NixOS closure. worker1 # [ 3.862097] systemd[1]: Reached target Initrd Default Target. worker2 # [ 3.843479] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 3.844487] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.863663] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 3.845483] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 3.846283] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 3.847330] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 3.850461] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 3.877826] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 3.858911] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 3.878947] systemd[1]: Stopped target Basic System. worker1 # [ 3.881119] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 3.881900] systemd[1]: Stopped target Path Units. worker2 # [ 3.863502] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 3.882623] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 3.864628] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 3.883624] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 3.865664] systemd[1]: Closed udev Control Socket. worker1 # [ 3.885069] systemd[1]: Stopped target Slice Units. worker1 # [ 3.885777] systemd[1]: Stopped target Socket Units. worker1 # [ 3.886553] systemd[1]: Stopped target System Initialization. worker1 # [ 3.887376] systemd[1]: Stopped target Swaps. worker2 # [ 3.869036] systemd[1]: Starting Cleanup udev Database... worker1 # [ 3.888060] systemd[1]: Stopped target Timer Units. worker2 # [ 3.869802] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 3.888932] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 3.870802] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 3.890176] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 3.871700] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 3.891253] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 3.873075] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 3.892303] systemd[1]: Stopped Find NixOS closure. worker2 # [ 3.874104] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 3.893898] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.875727] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 3.895096] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.876703] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.896229] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.878158] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.898051] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 3.899690] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 3.900443] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 3.901616] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 3.903080] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 3.904093] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 3.906115] systemd[1]: Stopped target Local File Systems. worker1 # [ 3.906963] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 3.907936] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.909657] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 3.912567] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 3.913564] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 3.894950] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 3.914528] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 3.896569] systemd[1]: Finished Cleanup udev Database. worker2 # [ 3.897811] systemd[1]: Reached target Switch Root. worker2 # [ 3.899638] systemd[1]: Starting NixOS Activation... client # [ 3.932974] initrd-nixos-activation-start[190]: booting system configuration /nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test worker1 # [ 3.923635] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 3.928064] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 3.929068] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 3.930059] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 3.936268] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 3.938088] systemd[1]: Closed udev Control Socket. worker1 # [ 3.939151] systemd[1]: Starting Cleanup udev Database... worker1 # [ 3.942143] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 3.943229] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 3.944093] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 3.945352] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 3.947165] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 3.948136] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 3.948966] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.950141] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.963529] initrd-nixos-activation-start[190]: running activation script... worker1 # [ 3.967290] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 3.968942] systemd[1]: Finished Cleanup udev Database. worker1 # [ 3.970209] systemd[1]: Reached target Switch Root. worker1 # [ 3.971947] systemd[1]: Starting NixOS Activation... worker2 # [ 3.973791] initrd-nixos-activation-start[192]: booting system configuration /nix/store/zqb8x0sddns70mbblwgjmcf92snp08cl-nixos-system-worker2-test worker2 # [ 4.000893] initrd-nixos-activation-start[192]: running activation script... worker1 # [ 4.039549] initrd-nixos-activation-start[190]: booting system configuration /nix/store/8mmcv7i4v7dgji0gb0y4pksgkj6hhjkk-nixos-system-worker1-test worker1 # [ 4.063482] initrd-nixos-activation-start[190]: running activation script... client # [ 4.157971] initrd-nixos-activation-start[213]: setting up /etc... worker2 # [ 4.200618] initrd-nixos-activation-start[215]: setting up /etc... worker1 # [ 4.242927] initrd-nixos-activation-start[213]: setting up /etc... client # [ 4.264420] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 4.266097] systemd[1]: Finished NixOS Activation. client # [ 4.268069] systemd[1]: Starting Switch Root... client # [ 4.284093] systemd[1]: Switching root. worker2 # [ 4.306777] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 4.309077] systemd[1]: Finished NixOS Activation. worker2 # [ 4.310568] systemd[1]: Starting Switch Root... worker1 # [ 4.337215] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 4.338819] systemd[1]: Finished NixOS Activation. worker1 # [ 4.340698] systemd[1]: Starting Switch Root... worker2 # [ 4.324428] systemd[1]: Switching root. worker1 # [ 4.353097] systemd[1]: Switching root. client # [ 4.546587] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker2 # [ 4.585248] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker1 # [ 4.615197] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 4.651707] NET: Registered PF_VSOCK protocol family worker1 # [ 4.686295] NET: Registered PF_VSOCK protocol family worker2 # [ 4.689385] NET: Registered PF_VSOCK protocol family lb # [ 4.659574] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.685890] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.688087] systemd[1]: Reached target Initrd File Systems. lb # [ 4.689848] systemd[1]: Starting Find NixOS closure... lb # [ 4.692749] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.713255] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.716793] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.722658] systemd[1]: Finished Find NixOS closure. lb # [ 4.723773] systemd[1]: Reached target Initrd Default Target. lb # [ 4.725576] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.739223] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.740248] systemd[1]: Stopped target Basic System. lb # [ 4.741152] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.743132] systemd[1]: Stopped target Path Units. lb # [ 4.743799] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.744822] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.745845] systemd[1]: Stopped target Slice Units. lb # [ 4.746589] systemd[1]: Stopped target Socket Units. lb # [ 4.747565] systemd[1]: Stopped target System Initialization. lb # [ 4.748446] systemd[1]: Stopped target Swaps. lb # [ 4.749137] systemd[1]: Stopped target Timer Units. lb # [ 4.750114] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.751134] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.752104] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.753560] systemd[1]: Stopped Find NixOS closure. lb # [ 4.754624] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.756163] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.757251] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.758214] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.759344] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.760349] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.762173] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.763426] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.764584] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.765910] systemd[1]: Stopped target Local File Systems. lb # [ 4.766789] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.768291] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.770142] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.771831] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.772893] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.775232] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.782183] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.786181] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 4.788846] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.790894] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.794154] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.795234] systemd[1]: Closed udev Control Socket. lb # [ 4.796988] systemd[1]: Starting Cleanup udev Database... lb # [ 4.798136] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.799172] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.800115] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.802134] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.803105] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.803990] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 4.805278] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.806670] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.821891] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 4.823030] systemd[1]: Finished Cleanup udev Database. lb # [ 4.824443] systemd[1]: Reached target Switch Root. lb # [ 4.826222] systemd[1]: Starting NixOS Activation... client # [ 5.010675] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 5.021283] systemd[1]: Detected virtualization kvm. client # [ 5.023252] systemd[1]: Detected architecture x86-64. client # [ 5.025459] systemd[1]: Detected first boot. client # [ 5.029915] systemd[1]: Initializing machine ID from random generator. lb # [ 4.902736] initrd-nixos-activation-start[190]: booting system configuration /nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test lb # [ 4.931686] initrd-nixos-activation-start[190]: running activation script... worker1 # [ 5.042640] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 5.052307] systemd[1]: Detected virtualization kvm. worker1 # [ 5.054105] systemd[1]: Detected architecture x86-64. worker1 # [ 5.056061] systemd[1]: Detected first boot. worker1 # [ 5.060068] systemd[1]: Initializing machine ID from random generator. worker2 # [ 5.043539] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 5.053296] systemd[1]: Detected virtualization kvm. worker2 # [ 5.055126] systemd[1]: Detected architecture x86-64. worker2 # [ 5.057048] systemd[1]: Detected first boot. worker2 # [ 5.060657] systemd[1]: Initializing machine ID from random generator. client # [ 5.169223] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 5.138216] initrd-nixos-activation-start[213]: setting up /etc... client # [ 5.273099] systemd[1]: Applying preset policy. worker1 # [ 5.300310] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 5.294084] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 5.262956] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.265065] systemd[1]: Finished NixOS Activation. lb # [ 5.267647] systemd[1]: Starting Switch Root... lb # [ 5.285155] systemd[1]: Switching root. worker2 # [ 5.400520] systemd[1]: Applying preset policy. worker1 # [ 5.421513] systemd[1]: Applying preset policy. client # [ 5.451275] systemd[1]: Populated /etc with preset unit settings. lb # [ 5.546137] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.568523] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 5.592436] systemd[1]: Populated /etc with preset unit settings. client # [ 5.640405] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 5.641778] systemd[1]: Stopped initrd-switch-root.service. client # [ 5.643958] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 5.646072] systemd[1]: Created slice Slice /system/getty. client # [ 5.647376] systemd[1]: Created slice User and Session Slice. client # [ 5.648294] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 5.649465] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 5.650571] systemd[1]: Expecting device /dev/hvc0... client # [ 5.651253] systemd[1]: Expecting device /dev/ttyS0... client # [ 5.652013] systemd[1]: Reached target Local Encrypted Volumes. client # [ 5.652873] systemd[1]: Stopped target initrd-fs.target. client # [ 5.653652] systemd[1]: Stopped target initrd-root-fs.target. client # [ 5.654414] systemd[1]: Stopped target initrd-switch-root.target. client # [ 5.655280] systemd[1]: Reached target Virtual Machines and Containers. client # [ 5.656177] systemd[1]: Reached target Path Units. client # [ 5.656887] systemd[1]: Reached target Remote File Systems. client # [ 5.657668] systemd[1]: Reached target Slice Units. client # [ 5.658360] systemd[1]: Reached target Swaps. client # [ 5.660135] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 5.662615] systemd[1]: Listening on Process Core Dump Socket. client # [ 5.664323] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 5.677750] systemd[1]: Listening on Factory Reset Management. client # [ 5.678802] systemd[1]: Listening on Hostname Service Socket. client # [ 5.681820] systemd[1]: Starting Journal Log Access Socket... client # [ 5.683083] systemd[1]: Listening on Journal Audit Socket. client # [ 5.685470] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 5.686734] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 5.688354] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.689734] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 5.693537] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 5.694672] systemd[1]: Listening on udev Control Socket. client # [ 5.695554] systemd[1]: Listening on udev Varlink Socket. client # [ 5.697674] systemd[1]: Mounting Huge Pages File System... client # [ 5.700042] systemd[1]: Mounting POSIX Message Queue File System... client # [ 5.706925] systemd[1]: Mounting Kernel Debug File System... client # [ 5.714635] systemd[1]: Mounting Kernel Trace File System... client # [ 5.724747] systemd[1]: Starting Create List of Static Device Nodes... client # [ 5.734341] systemd[1]: Starting Load Kernel Module configfs... client # [ 5.742762] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 5.750375] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 5.760236] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 5.771853] systemd[1]: Mounting FUSE Control File System... client # [ 5.774702] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 5.748934] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 5.750282] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 5.752771] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 5.754831] systemd[1]: Created slice Slice /system/getty. worker2 # [ 5.756264] systemd[1]: Created slice User and Session Slice. worker2 # [ 5.757214] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.758477] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 5.759562] systemd[1]: Expecting device /dev/hvc0... client # [ 5.791343] systemd[1]: Starting Journal Service... worker2 # [ 5.760233] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 5.760993] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 5.778610] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 5.761822] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 5.780032] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 5.762573] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 5.763340] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 5.764175] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 5.765082] systemd[1]: Reached target Path Units. client # [ 5.797176] systemd[1]: Starting Load Kernel Modules... worker1 # [ 5.782688] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 5.765799] systemd[1]: Reached target Remote File Systems. worker2 # [ 5.766592] systemd[1]: Reached target Slice Units. worker1 # [ 5.784855] systemd[1]: Created slice Slice /system/getty. worker1 # [ 5.786212] systemd[1]: Created slice User and Session Slice. worker1 # [ 5.787131] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.788442] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 5.789502] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 5.790212] systemd[1]: Expecting device /dev/ttyS0... worker1 # [ 5.790977] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 5.791820] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 5.792581] systemd[1]: Stopped target initrd-root-fs.target. client # [ 5.807084] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 5.793373] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 5.794227] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 5.795150] systemd[1]: Reached target Path Units. worker1 # [ 5.795889] systemd[1]: Reached target Remote File Systems. worker2 # [ 5.767265] systemd[1]: Reached target Swaps. worker1 # [ 5.796655] systemd[1]: Reached target Slice Units. worker1 # [ 5.797351] systemd[1]: Reached target Swaps. worker2 # [ 5.780220] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 5.799054] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 5.782500] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 5.801487] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 5.784085] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 5.803215] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 5.785845] systemd[1]: Listening on Factory Reset Management. worker2 # [ 5.786802] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 5.789634] systemd[1]: Starting Journal Log Access Socket... client # [ 5.821621] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 5.790773] systemd[1]: Listening on Journal Audit Socket. client # [ 5.824591] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.793308] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 5.794829] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 5.795988] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.797300] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 5.805283] systemd[1]: Listening on Factory Reset Management. worker1 # [ 5.817807] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 5.800910] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 5.801956] systemd[1]: Listening on udev Control Socket. worker2 # [ 5.802861] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 5.820794] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 5.821901] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 5.804990] systemd[1]: Mounting Huge Pages File System... client # [ 5.837626] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 5.824231] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 5.825817] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 5.808916] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 5.826976] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.828275] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 5.842820] systemd-journald[283]: Collecting audit messages is enabled. worker1 # [ 5.832314] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 5.833397] systemd[1]: Listening on udev Control Socket. worker2 # [ 5.815791] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 5.834275] systemd[1]: Listening on udev Varlink Socket. client # [ 5.850275] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 5.836501] systemd[1]: Mounting Huge Pages File System... client # [ 5.851246] systemd[1]: Mounted Huge Pages File System. worker1 # [ 5.839828] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 5.822897] systemd[1]: Mounting Kernel Trace File System... client # [ 5.858910] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.845931] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 5.832651] systemd[1]: Starting Create List of Static Device Nodes... client # [ 5.866461] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 5.854426] systemd[1]: Mounting Kernel Trace File System... client # [ 5.734518] systemd[1]: Queued start job for default target Multi-User System. client # [ 5.737327] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 5.842619] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 5.862613] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 5.845713] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 5.880095] systemd[1]: Started Journal Service. client # [ 5.883899] loop: module loaded worker1 # [ 5.872261] systemd[1]: Starting Load Kernel Module configfs... client # [ 5.751981] systemd-modules-load[284]: Inserted module 'loop' worker2 # [ 5.854485] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 5.755179] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 5.875611] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 5.758219] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 5.862992] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 5.763299] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 5.883518] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 5.766415] systemd[1]: Finished Load Kernel Module configfs. client # [ 5.768416] systemd[1]: Finished Load Kernel Modules. worker2 # [ 5.873011] systemd[1]: Mounting FUSE Control File System... client # [ 5.771262] systemd[1]: Mounted FUSE Control File System. worker1 # [ 5.894167] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 5.878531] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 5.912534] EXT4-fs (vda): re-mounted 935c7864-4bec-451b-9c6d-f9811e037d8e. worker1 # [ 5.904967] systemd[1]: Mounting FUSE Control File System... client # [ 5.787204] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 5.890143] systemd[1]: Starting Journal Service... worker1 # [ 5.909544] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 5.791105] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 5.894553] systemd[1]: Starting Load Kernel Modules... client # [ 5.795102] systemd[1]: Mounting Kernel Configuration File System... client # [ 5.798146] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 5.799128] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 5.921632] systemd[1]: Starting Journal Service... worker2 # [ 5.903866] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 5.803802] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 5.927596] systemd[1]: Starting Load Kernel Modules... worker2 # [ 5.909973] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 5.912713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.813246] systemd-oomd[285]: No swap; memory pressure usage will be degraded worker2 # [ 5.917004] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 5.936751] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 5.922097] systemd[1]: Listening on Journal Log Access Socket. client # [ 5.821056] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 5.924003] systemd[1]: Mounted Huge Pages File System. worker1 # [ 5.941846] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 5.926007] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.944452] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.928579] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 5.930231] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 5.932876] systemd[1]: Mounted FUSE Control File System. client # [ 5.830644] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 5.831655] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.952927] systemd[1]: Starting Coldplug All udev Devices... client # [ 5.832875] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 5.936771] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 5.958420] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 5.940806] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 5.960452] systemd[1]: Mounted Huge Pages File System. worker1 # [ 5.962382] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.964410] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 5.965322] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 5.969110] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 5.971553] systemd[1]: Mounted FUSE Control File System. worker1 # [ 5.979967] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 5.998414] NET: Registered PF_VSOCK protocol family worker2 # [ 5.974025] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 5.976146] systemd[1]: Finished Load Kernel Module configfs. worker1 # [ 5.993665] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 5.995289] systemd[1]: Finished Load Kernel Module configfs. worker2 # [ 5.980450] systemd[1]: Mounting Kernel Configuration File System... client # [ 6.012778] systemd-journald[283]: Received client request to flush runtime journal. worker1 # [ 5.999576] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 6.025529] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.008085] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.019697] EXT4-fs (vda): re-mounted 267390b9-6407-422c-a794-9f1b4b2bdf9f. worker2 # [ 6.026628] systemd-journald[285]: Collecting audit messages is enabled. worker1 # [ 6.046637] systemd-journald[283]: Collecting audit messages is enabled. worker2 # [ 6.029010] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.031726] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 6.033452] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 6.056419] EXT4-fs (vda): re-mounted 661a4feb-6ef4-4235-acef-9ac0c6c3851b. worker2 # [ 6.041502] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.042266] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.060919] loop: module loaded worker2 # [ 6.044600] loop: module loaded worker1 # [ 6.067800] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.050276] systemd[1]: Finished Load Kernel Modules. worker1 # [ 6.068971] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 6.071443] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.056831] systemd[1]: Starting Firewall... worker1 # [ 6.076709] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 6.078421] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.082428] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.064662] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 6.088582] systemd[1]: Starting Firewall... client # [ 5.971613] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 6.092850] systemd[1]: Starting Apply Kernel Variables... client # [ 5.974808] systemd[1]: Reached target First Boot Complete. client # [ 5.976272] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 5.963750] systemd[1]: Queued start job for default target Multi-User System.[ 6.098029] systemd[1]: Started Journal Service. client # [ 5.979207] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 5.949091] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.084433] systemd[1]: Started Journal Service. client # [ 5.981958] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 5.953389] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # worker1 # [ 5.977206] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 5.958756] systemd-modules-load[286]: Inserted module 'loop' worker1 # [ 5.982712] systemd-modules-load[284]: Inserted module 'loop' worker1 # [ 5.987049] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 5.968524] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 5.987576] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 5.991074] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 6.017229] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 6.020343] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 6.056039] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 6.026586] systemd[1]: Finished Load/Save OS Random Seed. client # [ 6.058155] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 6.027509] systemd[1]: Reached target First Boot Complete. client # [ 6.060353] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 6.061146] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 6.062135] systemd[1]: Reached target First Boot Complete. worker2 # [ 6.179149] systemd-journald[285]: Received client request to flush runtime journal. worker1 # [ 6.198414] systemd-journald[283]: Received client request to flush runtime journal. client # [ 6.085068] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 6.086159] systemd[1]: Reached target Preparation for Local File Systems. client # [ 6.088516] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 6.119645] systemd-udevd[314]: Using default interface naming scheme 'v261'. client # [ 6.152194] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 6.161455] systemd-oomd[286]: No swap; memory pressure usage will be degraded worker1 # [ 6.167209] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 6.168196] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 6.171096] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 6.162094] systemd-oomd[287]: No swap; memory pressure usage will be degraded worker2 # [ 6.168110] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 6.169058] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 6.169762] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 6.170682] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 6.172673] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 6.192080] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 6.173732] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 6.193224] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 6.197109] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 6.357256] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.362180] systemd[1]: Detected virtualization kvm. lb # [ 6.363123] systemd[1]: Detected architecture x86-64. lb # [ 6.364115] systemd[1]: Detected first boot. lb # [ 6.366371] systemd[1]: Initializing machine ID from random generator. client # [ 6.231959] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.280125] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 6.251609] systemd-udevd[318]: Using default interface naming scheme 'v261'. worker1 # [ 6.274491] systemd-udevd[318]: Using default interface naming scheme 'v261'. client # [ 6.296078] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. client # [ 6.316848] (udev-worker)[323]: Network interface NamePolicy= disabled on kernel command line. client # [ 6.320341] (udev-worker)[321]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 6.322880] (udev-worker)[321]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.324340] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 6.351443] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 6.395496] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 6.396665] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.398075] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.400367] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.401831] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.403315] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.395355] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 6.380776] systemd[1]: Finished Coldplug All udev Devices. lb # [ 6.575751] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.599561] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 client # [ 6.605757] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 6.489049] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 6.473951] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.514527] systemd[1]: Mounting /run/wrappers... lb # [ 6.664944] systemd[1]: Applying preset policy. worker2 # [ 6.544835] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 6.574538] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 6.588675] systemd[1]: Mounted /run/wrappers. client # [ 6.597159] systemd[1]: Reached target Local File Systems. client # [ 6.599112] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.603191] systemd[1]: Starting register-nix-paths.service... client # [ 6.606188] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 6.610826] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 6.617096] ACPI: button: Power Button [PWRF] client # [ 6.615962] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 6.617694] systemd[1]: Starting Create System Files and Directories... worker1 # [ 6.609745] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker2 # [ 6.593336] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker1 # [ 6.632154] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.617171] systemd[1]: Mounting /run/wrappers... worker2 # [ 6.619154] (udev-worker)[358]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 6.639935] (udev-worker)[354]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 6.622183] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 6.642528] (udev-worker)[354]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.625678] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 6.649076] systemd[1]: Mounting /run/wrappers... client # [ 6.621770] mousedev: PS/2 mouse device common for all mice client # [ 6.669713] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 6.661393] systemd[1]: Mounted /run/wrappers. worker2 # [ 6.662868] systemd[1]: Reached target Local File Systems. lb # [ 6.836001] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 6.669544] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.707374] systemd[1]: Finished Create System Files and Directories. worker1 # [ 6.697675] systemd[1]: Mounted /run/wrappers. worker2 # [ 6.678789] systemd[1]: Starting register-nix-paths.service... worker1 # [ 6.698969] systemd[1]: Reached target Local File Systems. client # [ 6.713214] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 6.682204] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 6.681973] rtc_cmos PNP0B00:00: registered as rtc0 worker1 # [ 6.703067] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.682040] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T04:41:24 UTC (1790138484) client # [ 6.682119] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker2 # [ 6.685433] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 6.720254] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 6.708975] systemd[1]: Starting register-nix-paths.service... worker2 # [ 6.690352] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 6.711075] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 6.714203] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 6.697124] systemd[1]: Starting Create System Files and Directories... worker1 # [ 6.721250] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 6.735682] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 6.724656] systemd[1]: Starting Create System Files and Directories... client # [ 6.744606] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 6.785430] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 6.682716] systemd-journald[283]: Time jumped backwards, rotating. client # [ 6.709467] parport_pc 00:02: reported by Plug and Play ACPI client # [ 6.717086] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 client # [ 6.814113] systemd[1]: Finished Rebuild Journal Catalog. client # [ 6.816088] systemd[1]: Starting Update is Completed... worker1 # [ 6.819217] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 6.823823] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 6.720059] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 6.744097] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] client # [ 6.749305] Console: switching to colour dummy device 80x25 worker2 # [ 6.820885] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 6.823705] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 6.858890] systemd[1]: Finished Update is Completed. worker2 # [ 6.830104] systemd[1]: Finished Create System Files and Directories. worker2 # [ 6.835767] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 6.838717] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 7.009153] [drm] Found bochs VGA, ID 0xb0c5. client # [ 7.009632] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 6.867214] systemd[1]: Finished Create System Files and Directories. client # [ 7.015773] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt client # [ 7.016862] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker1 # [ 6.871306] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 7.021980] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.023402] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 6.873213] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 7.025761] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 7.027762] systemd[1]: Created slice Slice /system/getty. lb # [ 7.029107] systemd[1]: Created slice User and Session Slice. lb # [ 7.030054] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.031233] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.032368] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.033043] systemd[1]: Expecting device /dev/ttyS0... lb # [ 7.033830] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.034702] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.035488] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.036250] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 7.037097] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.038037] systemd[1]: Reached target Path Units. lb # [ 7.038781] systemd[1]: Reached target Remote File Systems. lb # [ 7.039589] systemd[1]: Reached target Slice Units. lb # [ 7.040263] systemd[1]: Reached target Swaps. client # [ 7.038860] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic lb # [ 7.042107] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.039625] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 lb # [ 7.044588] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.046433] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.045741] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 client # [ 7.047442] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 7.048234] systemd[1]: Listening on Factory Reset Management. lb # [ 7.060758] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.063703] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.064892] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.067682] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.068814] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.069938] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.071236] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.075154] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.076154] systemd[1]: Listening on udev Control Socket. lb # [ 7.077059] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.079149] systemd[1]: Mounting Huge Pages File System... lb # [ 7.081694] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 6.913245] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 7.086952] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.095609] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 6.950963] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 7.104232] systemd[1]: Starting Create List of Static Device Nodes... client # [ 6.970936] systemd[1]: Starting Virtual Console Setup... lb # [ 7.113663] systemd[1]: Starting Load Kernel Module configfs... lb # [ 7.116239] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.126814] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 6.969752] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 7.138988] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 6.971550] systemd[1]: Starting Update is Completed... lb # [ 7.151174] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.007198] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 7.157673] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.009941] systemd[1]: Starting Update is Completed... lb # [ 7.172014] systemd[1]: Starting Journal Service... lb # [ 7.178452] systemd[1]: Starting Load Kernel Modules... lb # [ 7.188084] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.045145] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 7.046387] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.047858] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.199896] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.049888] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.053825] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.206521] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.056333] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.058207] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.216205] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.224261] systemd-journald[283]: Collecting audit messages is enabled. worker2 # [ 7.055105] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 7.225854] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.057938] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.229715] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.059917] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.232507] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.084382] systemd[1]: Finished Update is Completed. worker2 # [ 7.064870] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.236640] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.066975] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.068490] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.239571] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.070590] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.247843] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.082377] systemd[1]: Finished Update is Completed. lb # [ 7.117761] systemd[1]: Queued start job for default target Multi-User System. lb # [ 7.254612] systemd[1]: Started Journal Service. lb # [ 7.121842] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 7.255420] Console: switching to colour frame buffer device 160x50 lb # [ 7.129731] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 7.135135] systemd[1]: Finished Load Kernel Module configfs. lb # [ 7.135934] systemd[1]: Mounted FUSE Control File System. lb # [ 7.276847] loop: module loaded lb # [ 7.148889] systemd-modules-load[284]: Inserted module 'loop' client # [ 7.283564] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 7.158541] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.157178] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 7.296267] EXT4-fs (vda): re-mounted 342e5c28-2b4a-4c5a-9a1d-76ced8e4ca08. client # [ 7.160212] systemd[1]: Stopped Virtual Console Setup. lb # [ 7.167107] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.167553] systemd[1]: Starting Virtual Console Setup... lb # [ 7.171607] systemd[1]: Finished Load Kernel Modules. lb # [ 7.176098] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.177813] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.309558] ppdev: user-space parallel port driver lb # [ 7.187158] systemd[1]: Starting Firewall... worker2 # [ 7.287145] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console lb # [ 7.192456] systemd-oomd[286]: No swap; memory pressure usage will be degraded lb # [ 7.195140] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 7.295333] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 lb # [ 7.195991] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.200462] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.334256] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) lb # [ 7.212074] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.212856] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.219073] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 7.229526] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 7.232052] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 7.369950] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker1 # [ 7.229749] systemd[1]: Finished Firewall. client # [ 7.245588] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.248068] systemd[1]: Stopped Virtual Console Setup. client # [ 7.253367] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.230675] systemd[1]: Finished Firewall. lb # [ 7.417964] systemd-journald[283]: Received client request to flush runtime journal. worker1 # [ 7.460171] mousedev: PS/2 mouse device common for all mice worker1 # [ 7.463472] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console client # [ 7.480904] kvm_amd: TSC scaling supported client # [ 7.481317] kvm_amd: Nested Virtualization enabled client # [ 7.482026] kvm_amd: Nested Paging enabled client # [ 7.482428] kvm_amd: LBR virtualization supported client # [ 7.483106] kvm_amd: Virtual VMLOAD VMSAVE supported client # [ 7.483921] kvm_amd: Virtual GIF supported client # [ 7.484315] kvm_amd: Virtual NMI enabled worker1 # [ 7.345379] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 7.350469] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 7.338690] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 7.341755] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 7.384248] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.386386] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.389407] systemd[1]: Reached target First Boot Complete. client # [ 7.521581] EDAC MC: Ver: 3.0.0 lb # [ 7.390656] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.392239] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 7.394194] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.395096] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.351617] ACPI: button: Power Button [PWRF] worker2 # [ 7.457089] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 7.502161] rtc_cmos PNP0B00:00: registered as rtc0 worker2 # [ 7.502229] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T04:41:25 UTC (1790138485) worker2 # [ 7.502339] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker2 # [ 7.502717] systemd-journald[285]: Time jumped backwards, rotating. worker2 # [ 7.510320] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 worker2 # [ 7.514929] parport_pc 00:02: reported by Plug and Play ACPI worker2 # [ 7.515008] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker2 # [ 7.520607] Console: switching to colour dummy device 80x25 lb # [ 7.491252] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 7.524532] mousedev: PS/2 mouse device common for all mice lb # [ 7.501173] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.502403] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.505496] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 7.469691] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input3 worker1 # [ 7.486251] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input4 worker2 # [ 7.568140] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized worker2 # [ 7.638198] [drm] Found bochs VGA, ID 0xb0c5. worker2 # [ 7.638200] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. lb # [ 7.549501] systemd-udevd[322]: Using default interface naming scheme 'v261'. client # [ 7.548746] systemd[1]: Finished register-nix-paths.service. worker2 # [ 7.654654] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker2 # [ 7.655290] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 7.504905] ACPI: button: Power Button [PWRF] worker1 # [ 7.537986] rtc_cmos PNP0B00:00: RTC can wake from S4 client # [ 7.557255] systemd-vconsole-setup[419]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 7.560407] systemd[1]: Finished Virtual Console Setup. client # [ 7.562312] systemd[1]: Reached target System Initialization. client # [ 7.563587] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 7.564930] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 7.566137] systemd[1]: Reached target Timer Units. client # [ 7.567210] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 7.568516] systemd[1]: Listening on Nix Daemon Socket. client # [ 7.569335] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 7.554909] rtc_cmos PNP0B00:00: registered as rtc0 worker1 # [ 7.554979] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T04:41:25 UTC (1790138485) client # [ 7.571141] systemd[1]: Reached target Socket Units. worker1 # [ 7.555053] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs client # [ 7.571821] systemd[1]: Reached target Basic System. client # [ 7.573180] systemd[1]: Started backdoor.service. client # [ 7.575960] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 7.583123] systemd[1]: Starting nftables firewall... worker2 # [ 7.686673] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker2 # [ 7.687332] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD client # [ 7.586464] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 7.598226] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 7.597109] systemd[1]: Starting Post-Boot Actions... worker1 # [ 7.555610] systemd-journald[283]: Time jumped backwards, rotating. worker1 # [ 7.578249] parport_pc 00:02: reported by Plug and Play ACPI worker1 # [ 7.588605] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker1 # [ 7.597744] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 7.607808] systemd[1]: Started Reset console on configuration changes. client # [ 7.627267] systemd[1]: Starting resolvconf update... worker1 # [ 7.642449] Console: switching to colour dummy device 80x25 worker1 # [ 7.702821] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker1 # [ 7.702938] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD client # connecting to host... client # [ 7.665071] systemd[1]: Finished Post-Boot Actions. worker1 # [ 7.789972] [drm] Found bochs VGA, ID 0xb0c5. worker1 # [ 7.789975] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 7.795216] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client: Guest shell says: b'Spawning backdoor root shell...\n' client # [ 7.678083] nsncd[427]: Sep 23 04:41:25.626 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 7.673190] systemd[1]: Finished register-nix-paths.service. client # [ 7.679886] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 7.674332] systemd[1]: Finished register-nix-paths.service. client # [ 7.691383] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 7.712825] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.677311] systemd[1]: Reached target System Initialization. client # [ 7.693104] systemd[1]: Reached target User and Group Name Lookups. client: connected to guest root shell worker2 # [ 7.678367] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 7.681782] systemd[1]: Started Daily Cleanup of Temporary Directories. client: (connecting took 8.41 seconds) client # [ 7.700231] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 7.682720] systemd[1]: Reached target Timer Units. client: (finished: waiting for the VM to finish booting, in 8.41 seconds) client # [ 7.707777] systemd[1]: Starting User Login Management... worker2 # [ 7.684081] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 7.713952] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 7.688120] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 7.808051] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker2 # [ 7.688901] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 7.689940] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 7.809148] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker2 # [ 7.693142] systemd[1]: Reached target Socket Units. worker1 # [ 7.678565] systemd[1]: Reached target System Initialization. worker2 # [ 7.693800] systemd[1]: Reached target Basic System. worker1 # [ 7.682732] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 7.697204] systemd[1]: Started backdoor.service. worker1 # [ 7.683697] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 7.697842] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 7.684579] systemd[1]: Reached target Timer Units. worker2 # [ 7.700078] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 7.685669] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 7.705075] systemd[1]: Starting Post-Boot Actions... worker1 # [ 7.687206] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 7.705760] systemd[1]: Started Reset console on configuration changes. worker1 # [ 7.688962] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 7.715352] systemd[1]: Starting resolvconf update... worker1 # [ 7.691616] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 7.721406] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 7.692738] systemd[1]: Reached target Socket Units. worker1 # [ 7.693445] systemd[1]: Reached target Basic System. worker1 # [ 7.827488] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 7.696635] systemd[1]: Started backdoor.service. worker1 # [ 7.698199] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 7.705357] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 7.707682] systemd[1]: Starting Post-Boot Actions... worker1 # [ 7.714188] systemd[1]: Started Reset console on configuration changes. worker1 # [ 7.718937] systemd[1]: Starting resolvconf update... worker1 # [ 7.729260] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 7.735626] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.752840] systemd[1]: Starting Virtual Console Setup... lb # [ 7.791985] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 7.816516] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker2 # [ 7.792492] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 7.793584] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 7.796241] systemd[1]: Starting Virtual Console Setup... client # [ 7.828191] dbus-broker-launch[456]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 7.749482] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker1 # [ 7.820051] systemd[1]: Finished Post-Boot Actions. lb # [ 7.847837] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. client # [ 7.845274] systemd-logind[457]: New seat seat0. client # [ 7.849634] systemd-logind[457]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 7.818688] systemd[1]: Finished Post-Boot Actions. client # [ 7.851251] systemd-logind[457]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 7.852419] dbus-broker-launch[456]: NSS returned no entry for 'systemd-timesync' client # [ 7.853399] dbus-broker-launch[456]: Invalid user-name in /nix/store/c4mprcfibsx637q0hk2g869f3wvd2k8k-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 7.859099] (udev-worker)[346]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 7.856349] systemd-logind[457]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) lb # [ 7.860973] (udev-worker)[346]: Network interface NamePolicy= disabled on kernel command line. client # [ 7.860219] systemd[1]: Started User Login Management. worker1 # connecting to host... client # [ 7.867718] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 7.868614] systemd[1]: Stopping Host and Network Name Lookups... client # [ 7.869516] systemd[1]: Stopped target User and Group Name Lookups. client # [ 7.870432] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 7.860742] nsncd[514]: Sep 23 04:41:25.934 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 7.864751] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 7.879610] systemd[1]: Starting linger-users.service... client # [ 7.880458] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 7.868135] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 7.868995] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 7.871911] systemd[1]: Starting User Login Management... worker2 # [ 7.853532] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 7.854474] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 7.856147] systemd[1]: Reached target User and Group Name Lookups. lb # [ 7.892476] systemd[1]: Mounting /run/wrappers... client # [ 7.891108] systemd[1]: nscd.service: Deactivated successfully. worker2 # connecting to host... worker2 # [ 7.861438] systemd[1]: Starting User Login Management... client # [ 7.894063] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 7.863501] nsncd[516]: Sep 23 04:41:25.981 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 7.892249] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 7.912690] systemd[1]: Started D-Bus System Message Bus. lb # [ 7.928157] systemd[1]: Mounted /run/wrappers. lb # [ 7.929584] systemd[1]: Reached target Local File Systems. client # [ 7.929866] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 7.933366] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 7.749726] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 7.938298] systemd[1]: Starting register-nix-paths.service... worker1 # [ 7.917787] dbus-broker-launch[518]: Looking up NSS user entry for 'systemd-timesync'... client # [ 7.937497] dbus-broker-launch[456]: Ready lb # [ 7.942155] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.944577] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.953148] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 7.939845] dbus-broker-launch[518]: NSS returned no entry for 'systemd-timesync' worker1 # [ 7.941426] dbus-broker-launch[518]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 7.968779] systemd[1]: Starting Create System Files and Directories... worker2 # [ 7.843957] Console: switching to colour frame buffer device 160x50 worker2 # [ 8.070010] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker2 # [ 7.898817] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 7.963424] systemd[1]: Started D-Bus System Message Bus. client # [ 7.976477] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 7.944703] dbus-broker-launch[520]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 7.948509] dbus-broker-launch[520]: NSS returned no entry for 'systemd-timesync' client # [ 7.980565] systemd[1]: Finished linger-users.service. worker2 # [ 7.950668] dbus-broker-launch[520]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 7.989442] systemd[1]: Finished resolvconf update. client # [ 7.992901] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 7.993982] nsncd[521]: Sep 23 04:41:25.944 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 7.996711] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 7.965676] systemd[1]: Started D-Bus System Message Bus. client # [ 7.997729] systemd[1]: Reached target User and Group Name Lookups. client # [ 8.001592] systemd[1]: Starting DHCP Client... worker1 # [ 8.013561] dbus-broker-launch[518]: Ready worker2 # [ 8.008382] dbus-broker-launch[520]: Ready worker2 # [ 8.182410] ppdev: user-space parallel port driver worker2 # [ 8.057555] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 8.059938] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 8.061605] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 8.068990] systemd[1]: Stopping User and Group Name Lookups... client # [ 8.103178] dhcpcd[552]: dhcpcd-10.3.2 starting worker2 # [ 8.072622] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 8.074132] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 8.074895] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 8.108146] dhcpcd[560]: dev: loaded udev worker2 # [ 8.077217] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 8.251141] 8021q: 802.1Q VLAN Support v1.8 lb # [ 8.121764] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.085169] systemd-logind[541]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 8.090292] systemd-logind[541]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 8.095210] systemd-logind[541]: New seat seat0. worker2 # [ 8.095839] systemd[1]: Started User Login Management. lb # [ 8.133200] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.100092] systemd[1]: Starting linger-users.service... lb # [ 8.142293] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.138959] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 8.299650] Console: switching to colour frame buffer device 160x50 lb # [ 8.187228] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 8.191141] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 8.194455] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.326193] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.196220] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 8.163367] systemd-logind[541]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 8.200136] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 8.201710] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.167835] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 8.204361] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.170413] nsncd[589]: Sep 23 04:41:26.299 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 8.174617] systemd[1]: Reached target Host and Network Name Lookups. client # [ 8.341535] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 8.342724] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 8.176438] systemd[1]: Reached target User and Group Name Lookups. client # [ 8.343922] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 8.178120] systemd[1]: linger-users.service: Deactivated successfully. client # [ 8.344799] cfg80211: failed to load regulatory.db worker2 # [ 8.178946] systemd[1]: Finished linger-users.service. worker2 # [ 8.202791] systemd[1]: Finished resolvconf update. worker2 # [ 8.204970] systemd[1]: Reached target Preparation for Network. worker2 # [ 8.207953] systemd[1]: Starting DHCP Client... worker1 # [ 8.358520] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker1 # [ 8.080368] systemd-logind[542]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 8.210953] systemd[1]: Starting Address configuration of eth1... worker2 # [ 8.212680] systemd[1]: Starting Extra networking commands.... client # [ 8.254924] systemd[1]: Finished nftables firewall. worker1 # [ 8.236859] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 8.243964] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 8.261670] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.258154] systemd[1]: Reached target Preparation for Network. client # [ 8.261103] systemd[1]: Starting Address configuration of eth1... worker2 # [ 8.229866] systemd-vconsole-setup[523]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 8.231387] systemd[1]: Finished Virtual Console Setup. client # [ 8.265161] systemd[1]: Starting Extra networking commands.... worker1 # [ 8.251610] nsncd[589]: Sep 23 04:41:26.239 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 8.401389] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 8.271778] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.269159] dhcpcd[560]: eth0: waiting for carrier worker1 # [ 8.256488] systemd[1]: Stopped target User and Group Name Lookups. client # [ 8.270270] dhcpcd[560]: eth0: carrier acquired worker1 # [ 8.261831] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 8.265863] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 8.280935] dhcpcd[560]: DUID 00:01:00:01:32:46:14:f6:52:54:00:12:34:56 worker1 # [ 8.269694] systemd[1]: nscd.service: Deactivated successfully. client # [ 8.282670] dhcpcd[560]: eth0: IAID 00:12:34:56 worker1 # [ 8.270674] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 8.271623] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 8.284462] dhcpcd[560]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 8.272569] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 8.273459] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 8.279058] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 8.279985] systemd[1]: Finished resolvconf update. lb # [ 8.296588] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.280782] systemd[1]: Reached target Preparation for Network. worker1 # [ 8.281694] systemd[1]: Starting DHCP Client... worker1 # [ 8.282494] systemd[1]: Starting Address configuration of eth1... worker1 # [ 8.288248] systemd[1]: Starting Extra networking commands.... worker1 # [ 8.289531] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.291168] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.296070] systemd-logind[542]: New seat seat0. worker1 # [ 8.296882] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.299708] systemd-logind[542]: Watching system buttons on /dev/input/event3 (Power Button) worker1 # [ 8.434899] ppdev: user-space parallel port driver worker1 # [ 8.302461] systemd-logind[542]: Watching system buttons on /dev/input/event2 (QEMU Virtio Keyboard) lb # [ 8.319300] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.305557] systemd[1]: Started User Login Management. worker1 # [ 8.307821] systemd[1]: Starting linger-users.service... client # [ 8.456324] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 8.327474] systemd[1]: Starting Update is Completed... client # [ 8.334130] network-addresses-eth1-start[583]: adding address 192.168.1.1/24... done worker2 # [ 8.444249] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 8.345948] network-addresses-eth1-start[583]: adding address 2001:db8:1::1/64... done client # [ 8.361712] systemd[1]: Finished Address configuration of eth1. worker2 # [ 8.334954] network-addresses-eth1-start[619]: adding address 192.168.1.4/24... done worker1 # [ 8.371278] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 8.352447] network-addresses-eth1-start[619]: adding address 2001:db8:1::4/64... done worker1 # [ 8.372248] systemd[1]: Finished linger-users.service. client # [ 8.387561] systemd[1]: Finished Extra networking commands.. client # [ 8.389294] systemd[1]: Reached target Network. client # [ 8.391886] systemd[1]: Starting Permit User Sessions... worker1 # [ 8.384313] network-addresses-eth1-start[614]: adding address 192.168.1.3/24... done worker2 # [ 8.375601] systemd[1]: Finished Address configuration of eth1. client # [ 8.407555] systemd[1]: Finished Permit User Sessions. client # [ 8.410913] systemd[1]: Started Getty on tty1. client # [ 8.412259] systemd[1]: Reached target Login Prompts. worker2 # [ 8.382252] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 8.383277] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 8.383973] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 8.406290] network-addresses-eth1-start[614]: adding address 2001:db8:1::3/64... done worker2 # [ 8.388230] systemd[1]: Starting Virtual Console Setup... lb # [ 8.424661] systemd[1]: Finished Update is Completed. lb # [ 8.573920] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 8.431730] systemd[1]: Finished Address configuration of eth1. lb # [ 8.583019] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker1 # [ 8.571167] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker2 # [ 8.555378] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker2 # [ 8.493657] dhcpcd[631]: dhcpcd-10.3.2 starting worker2 # [ 8.503440] systemd[1]: Finished Extra networking commands.. worker1 # [ 8.656875] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker2 # [ 8.505468] systemd[1]: Reached target Network. worker2 # [ 8.507495] systemd[1]: Starting Permit User Sessions... worker2 # [ 8.510810] dhcpcd[688]: dev: loaded udev worker1 # [ 8.544472] systemd[1]: Finished Extra networking commands.. worker1 # [ 8.545429] systemd[1]: Reached target Network. worker2 # [ 8.662973] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 8.547936] systemd[1]: Starting Permit User Sessions... worker2 # [ 8.665376] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 8.551266] dhcpcd[627]: dhcpcd-10.3.2 starting worker1 # [ 8.561476] systemd-vconsole-setup[619]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.567440] systemd[1]: Finished Virtual Console Setup. worker1 # [ 8.568459] dhcpcd[688]: dev: loaded udev worker1 # [ 8.574609] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.575835] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.577134] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 8.581186] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.722910] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 8.725623] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 8.590071] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 8.632279] systemd[1]: Finished Permit User Sessions. worker2 # [ 8.619739] systemd[1]: Finished Permit User Sessions. worker1 # [ 8.644178] systemd[1]: Started Getty on tty1. worker1 # [ 8.644926] systemd[1]: Reached target Login Prompts. worker2 # [ 8.635796] systemd[1]: Started Getty on tty1. worker2 # [ 8.636502] systemd[1]: Reached target Login Prompts. worker2 # [ 8.808103] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 8.700076] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 8.822116] kvm_amd: TSC scaling supported worker2 # [ 8.822575] kvm_amd: Nested Virtualization enabled worker2 # [ 8.823018] kvm_amd: Nested Paging enabled worker2 # [ 8.824109] kvm_amd: LBR virtualization supported worker2 # [ 8.824857] kvm_amd: Virtual VMLOAD VMSAVE supported worker2 # [ 8.825544] kvm_amd: Virtual GIF supported worker2 # [ 8.825934] kvm_amd: Virtual NMI enabled worker2 # [ 8.843475] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 8.844124] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 8.846725] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 8.847582] cfg80211: failed to load regulatory.db lb # [ 8.751416] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 8.636588] ACPI: button: Power Button [PWRF] lb # [ 8.756295] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 8.636831] mousedev: PS/2 mouse device common for all mice worker1 # [ 8.883994] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.639918] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 8.874039] EDAC MC: Ver: 3.0.0 lb # [ 8.681745] rtc_cmos PNP0B00:00: registered as rtc0 lb # [ 8.681818] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T04:41:26 UTC (1790138486) worker1 # [ 8.914380] kvm_amd: TSC scaling supported worker1 # [ 8.914823] kvm_amd: Nested Virtualization enabled worker1 # [ 8.915262] kvm_amd: Nested Paging enabled worker1 # [ 8.916121] kvm_amd: LBR virtualization supported worker1 # [ 8.916836] kvm_amd: Virtual VMLOAD VMSAVE supported worker1 # [ 8.917303] kvm_amd: Virtual GIF supported worker1 # [ 8.917969] kvm_amd: Virtual NMI enabled worker2 # [ 8.900151] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 8.767169] dhcpcd[688]: libudev: received NULL device worker2 # [ 8.767933] dhcpcd[688]: eth0: waiting for carrier worker2 # [ 8.769388] dhcpcd[688]: libudev: received NULL device lb # [ 8.804688] systemd[1]: Finished register-nix-paths.service. worker2 # [ 8.770211] dhcpcd[688]: eth0: carrier acquired lb # [ 8.805659] systemd[1]: Reached target System Initialization. lb # [ 8.808245] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 8.809254] systemd[1]: Started niks3 garbage collection timer. lb # [ 8.810133] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 8.775640] dhcpcd[688]: DUID 00:01:00:01:32:46:14:f6:52:54:00:12:34:56 lb # [ 8.810970] systemd[1]: Reached target Timer Units. lb # [ 8.811678] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 8.776825] dhcpcd[688]: eth0: IAID 00:12:34:56 worker2 # [ 8.777777] dhcpcd[688]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 8.931914] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 8.932599] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 8.816081] systemd[1]: Listening on niks3 server socket. lb # [ 8.816842] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 8.934035] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 8.934874] cfg80211: failed to load regulatory.db lb # [ 8.817725] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 8.820124] systemd[1]: Reached target Socket Units. worker2 # [ 8.784476] systemd-vconsole-setup[657]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 8.820858] systemd[1]: Reached target Basic System. worker2 # [ 8.786867] systemd[1]: Finished Virtual Console Setup. lb # [ 8.681892] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 8.682613] systemd-journald[283]: Time jumped backwards, rotating. lb # [ 8.750238] parport_pc 00:02: reported by Plug and Play ACPI worker2 # [ 8.789188] dhcpcd[688]: eth0: soliciting a DHCP lease lb # [ 8.826374] systemd[1]: Started backdoor.service. lb # [ 8.829492] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 8.931289] NET: Registered PF_PACKET protocol family worker2 # [ 8.800402] dhcpcd[688]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 8.803235] dhcpcd[688]: eth0: probing address 10.0.2.15/24 lb # [ 8.840097] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 8.850160] systemd[1]: Starting Post-Boot Actions... worker1 # [ 8.970616] EDAC MC: Ver: 3.0.0 worker1 # [ 8.979930] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 8.863556] systemd[1]: Started Reset console on configuration changes. worker1 # [ 8.849964] dhcpcd[688]: eth0: waiting for carrier worker1 # [ 8.851454] dhcpcd[688]: libudev: received NULL device worker1 # [ 8.852499] dhcpcd[688]: libudev: received NULL device worker1 # [ 8.853743] dhcpcd[688]: eth0: carrier acquired lb # [ 8.783140] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker1 # [ 8.858855] dhcpcd[688]: DUID 00:01:00:01:32:46:14:f6:52:54:00:12:34:56 worker1 # [ 8.860921] dhcpcd[688]: eth0: IAID 00:12:34:56 worker1 # [ 8.861748] dhcpcd[688]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 8.863282] dhcpcd[688]: eth0: soliciting a DHCP lease lb # [ 8.880080] systemd[1]: Starting resolvconf update... lb # [ 8.790666] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized worker1 # [ 9.003771] NET: Registered PF_PACKET protocol family worker1 # [ 8.874409] dhcpcd[688]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 8.876176] dhcpcd[688]: eth0: probing address 10.0.2.15/24 lb # [ 8.901637] systemd[1]: Started rustfs.service. lb # [ 8.929308] systemd[1]: Starting rustfs-bucket.service... worker1 # [ 8.914395] systemd-vconsole-setup[691]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.918203] systemd[1]: Finished Virtual Console Setup. lb # [ 8.806734] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 8.889248] Console: switching to colour dummy device 80x25 lb # [ 8.944894] systemd[1]: Finished Firewall. lb # connecting to host... lb # [ 8.965474] systemd[1]: Finished Post-Boot Actions. lb # [ 8.971878] nsncd[518]: Sep 23 04:41:26.923 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client: (finished: waiting for unit network-addresses-eth1.service, in 9.62 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 8.990219] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 8.972779] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt lb # [ 8.972890] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.01 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb # [ 9.004955] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.007100] systemd[1]: Reached target User and Group Name Lookups. lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.01 seconds) lb: (finished: waiting for the VM to finish booting, in 0.01 seconds) lb # [ 9.016419] systemd[1]: Starting D-Bus System Message Bus... lb # [ 9.039191] systemd[1]: Starting User Login Management... lb # [ 9.187913] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 lb # [ 9.087340] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 9.188493] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 9.269757] [drm] Found bochs VGA, ID 0xb0c5. lb # [ 9.269759] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. lb # [ 9.342258] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic lb # [ 9.342987] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 lb # [ 9.264406] systemd[1]: Starting Virtual Console Setup... lb # [ 9.286312] dbus-broker-launch[550]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 9.322600] systemd-logind[553]: New seat seat0. lb # [ 9.325889] dbus-broker-launch[550]: NSS returned no entry for 'systemd-timesync' lb # [ 9.328124] dbus-broker-launch[550]: Invalid user-name in /nix/store/xhp17752g31xx87ahxdgrix2v9kism59-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 9.333300] systemd[1]: Started User Login Management. lb # [ 9.344444] systemd-logind[553]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) lb # [ 9.352080] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.355844] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.356696] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 9.357577] systemd[1]: Stopping User and Group Name Lookups... lb # [ 9.362940] systemd[1]: Starting linger-users.service... lb # [ 9.363711] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 9.372109] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.375451] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 9.390082] systemd[1]: Started D-Bus System Message Bus. lb # [ 9.403877] systemd-logind[553]: Watching system buttons on /dev/input/event2 (Power Button) lb # [ 9.417409] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.428411] dbus-broker-launch[550]: Ready lb # [ 9.704381] ppdev: user-space parallel port driver lb # [ 9.744234] Console: switching to colour frame buffer device 160x50 lb # [ 9.822985] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) lb # [ 9.836200] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 9.481421] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 9.704326] nsncd[630]: Sep 23 04:41:27.466 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.706877] systemd[1]: Finished linger-users.service. lb # [ 9.711941] network-addresses-eth1-start[655]: adding address 192.168.1.2/24... done lb # [ 9.714277] systemd[1]: Finished resolvconf update. lb # [ 9.719389] systemd[1]: Reached target Preparation for Network. lb # [ 9.721596] systemd[1]: Starting DHCP Client... lb # [ 9.727497] systemd[1]: Starting Address configuration of eth1... lb # [ 9.728472] systemd[1]: Starting Extra networking commands.... lb # [ 9.729603] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.731173] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.735212] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.736130] systemd-logind[553]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 9.737317] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.738299] systemd[1]: Stopped Virtual Console Setup. lb # [ 9.752409] systemd[1]: Starting Virtual Console Setup... lb # [ 9.762105] network-addresses-eth1-start[655]: adding address 2001:db8:1::2/64... done client # [ 9.776815] dhcpcd[560]: eth0: soliciting a DHCP lease lb # [ 9.915776] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) client # [ 9.928346] NET: Registered PF_PACKET protocol family lb # [ 9.799642] systemd[1]: Finished Address configuration of eth1. client # [ 9.799841] dhcpcd[560]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 9.804146] dhcpcd[560]: eth0: probing address 10.0.2.15/24 lb # [ 9.857581] dhcpcd[665]: dhcpcd-10.3.2 starting lb # [ 9.872715] dhcpcd[722]: dev: loaded udev lb # [ 10.033079] 8021q: 802.1Q VLAN Support v1.8 lb # [ 10.037332] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 9.925476] systemd[1]: Finished Extra networking commands.. lb # [ 9.927774] systemd[1]: Reached target Network. lb # [ 9.936291] systemd[1]: Starting PostgreSQL Server... lb # [ 9.942490] systemd[1]: Starting Permit User Sessions... lb # [ 9.951517] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.963278] systemd[1]: Stopped Virtual Console Setup. lb # [ 10.023364] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 10.035282] systemd[1]: Starting Virtual Console Setup... lb # [ 10.083470] systemd[1]: Finished Permit User Sessions. lb # [ 10.118294] systemd[1]: Started Getty on tty1. lb # [ 10.118992] systemd[1]: Reached target Login Prompts. worker2 # [ 10.093414] dhcpcd[688]: eth0: soliciting an IPv6 router worker2 # [ 10.094909] dhcpcd[688]: eth0: Router Advertisement from fe80::2 worker2 # [ 10.096172] dhcpcd[688]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 10.097346] dhcpcd[688]: eth0: adding route to fec0::/64 worker2 # [ 10.098330] dhcpcd[688]: eth0: adding default route via fe80::2 lb # [ 10.316103] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 10.344020] kvm_amd: TSC scaling supported lb # [ 10.345584] kvm_amd: Nested Virtualization enabled lb # [ 10.346079] kvm_amd: Nested Paging enabled lb # [ 10.347618] kvm_amd: LBR virtualization supported lb # [ 10.348096] kvm_amd: Virtual VMLOAD VMSAVE supported lb # [ 10.350243] kvm_amd: Virtual GIF supported lb # [ 10.351648] kvm_amd: Virtual NMI enabled lb # [ 10.372917] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 10.373704] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 10.379267] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 10.380191] cfg80211: failed to load regulatory.db lb # [ 10.437944] EDAC MC: Ver: 3.0.0 lb # [ 10.466053] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 10.332905] dhcpcd[722]: eth0: waiting for carrier lb # [ 10.334460] dhcpcd[722]: libudev: received NULL device lb # [ 10.335463] dhcpcd[722]: libudev: received NULL device lb # [ 10.336716] dhcpcd[722]: eth0: carrier acquired lb # [ 10.348880] dhcpcd[722]: DUID 00:01:00:01:32:46:14:f8:52:54:00:12:34:56 lb # [ 10.350231] dhcpcd[722]: eth0: IAID 00:12:34:56 lb # [ 10.350868] dhcpcd[722]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 10.382380] postgresql-pre-start[746]: The files belonging to this database system will be owned by user "postgres". lb # [ 10.384774] postgresql-pre-start[746]: This user must also own the server process. lb # [ 10.388066] postgresql-pre-start[746]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 10.390297] postgresql-pre-start[746]: The default database encoding has accordingly been set to "UTF8". lb # [ 10.391508] postgresql-pre-start[746]: The default text search configuration will be set to "english". lb # [ 10.392708] postgresql-pre-start[746]: Data page checksums are enabled. lb # [ 10.393646] postgresql-pre-start[746]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 10.394870] postgresql-pre-start[746]: creating subdirectories ... ok lb # [ 10.395717] postgresql-pre-start[746]: selecting dynamic shared memory implementation ... posix client # [ 10.421395] dhcpcd[560]: eth0: soliciting an IPv6 router client # [ 10.423243] dhcpcd[560]: eth0: Router Advertisement from fe80::2 client # [ 10.424523] dhcpcd[560]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 10.425956] dhcpcd[560]: eth0: adding route to fec0::/64 client # [ 10.427116] dhcpcd[560]: eth0: adding default route via fe80::2 lb: (finished: waiting for unit network-addresses-eth1.service, in 1.43 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 10.506478] postgresql-pre-start[746]: selecting default "max_connections" ... 100 worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.02 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.02 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 10.557647] postgresql-pre-start[746]: selecting default "shared_buffers" ... 128MB lb # [ 10.573286] systemd-vconsole-setup[736]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 10.577562] systemd[1]: Finished Virtual Console Setup. worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service lb # [ 11.116932] postgresql-pre-start[746]: selecting default time zone ... UTC lb # [ 11.121534] postgresql-pre-start[746]: creating configuration files ... ok worker1 # [ 11.179834] dhcpcd[688]: eth0: soliciting an IPv6 router worker1 # [ 11.181220] dhcpcd[688]: eth0: Router Advertisement from fe80::2 worker1 # [ 11.182082] dhcpcd[688]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 11.182973] dhcpcd[688]: eth0: adding route to fec0::/64 worker1 # [ 11.183891] dhcpcd[688]: eth0: adding default route via fe80::2 lb # [ 11.286781] postgresql-pre-start[746]: running bootstrap script ... ok lb # [ 11.673272] postgresql-pre-start[746]: performing post-bootstrap initialization ... ok lb # [ 11.824117] postgresql-pre-start[746]: syncing data to disk ... ok lb # [ 11.824976] postgresql-pre-start[746]: initdb: warning: enabling "trust" authentication for local connections lb # [ 11.826288] postgresql-pre-start[746]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 11.828160] postgresql-pre-start[746]: Success. You can now start the database server using: lb # [ 11.829141] postgresql-pre-start[746]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 11.932942] dhcpcd[722]: eth0: soliciting a DHCP lease lb # [ 12.075484] NET: Registered PF_PACKET protocol family lb # [ 11.946455] dhcpcd[722]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.947538] dhcpcd[722]: eth0: probing address 10.0.2.15/24 lb # [ 11.970299] postgres[799]: [799] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 11.974460] postgres[799]: [799] LOG: listening on IPv6 address "::1", port 5432 lb # [ 11.975648] postgres[799]: [799] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 12.000850] postgres[799]: [799] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 12.004694] dhcpcd[722]: eth0: soliciting an IPv6 router lb # [ 12.005923] dhcpcd[722]: eth0: Router Advertisement from fe80::2 lb # [ 12.006780] dhcpcd[722]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 12.007998] dhcpcd[722]: eth0: adding route to fec0::/64 lb # [ 12.008809] dhcpcd[722]: eth0: adding default route via fe80::2 lb # [ 12.021370] postgres[810]: [810] LOG: database system was shut down at 2026-09-23 04:41:29 GMT lb # [ 12.027682] postgres[799]: [799] LOG: database system is ready to accept connections lb # [ 12.031935] systemd[1]: Started PostgreSQL Server. lb # [ 12.038376] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 12.156414] postgresql-setup-start[821]: CREATE DATABASE lb # [ 12.183927] postgresql-setup-start[826]: CREATE ROLE lb # [ 12.195513] postgresql-setup-start[828]: ALTER DATABASE lb # [ 12.199929] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 12.201551] systemd[1]: Reached target PostgreSQL. worker1 # [ 13.102905] dhcpcd[688]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 13.105505] dhcpcd[688]: eth0: adding route to 10.0.2.0/24 worker1 # [ 13.107386] dhcpcd[688]: eth0: adding default route via 10.0.2.2 worker1 # [ 13.185575] systemd[1]: Started DHCP Client. worker1 # [ 13.187697] systemd[1]: Reached target Multi-User System. worker1 # [ 13.188912] systemd[1]: Startup finished in 881ms (kernel) + 3.654s (initrd) + 8.651s (userspace) = 13.187s. worker2 # [ 14.689531] dhcpcd[688]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 14.691672] dhcpcd[688]: eth0: adding route to 10.0.2.0/24 worker2 # [ 14.695351] dhcpcd[688]: eth0: adding default route via 10.0.2.2 worker2 # [ 14.776488] systemd[1]: Started DHCP Client. worker2 # [ 14.778553] systemd[1]: Reached target Multi-User System. worker2 # [ 14.779442] systemd[1]: Startup finished in 877ms (kernel) + 3.660s (initrd) + 10.239s (userspace) = 14.778s. client # [ 15.441785] dhcpcd[560]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 15.444513] dhcpcd[560]: eth0: adding route to 10.0.2.0/24 client # [ 15.446953] dhcpcd[560]: eth0: adding default route via 10.0.2.2 client # [ 15.528422] systemd[1]: Started DHCP Client. client # [ 15.530463] systemd[1]: Reached target Multi-User System. client # [ 15.531466] systemd[1]: Startup finished in 886ms (kernel) + 3.608s (initrd) + 11.036s (userspace) = 15.531s. lb # [ 16.253504] dhcpcd[722]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 16.256474] dhcpcd[722]: eth0: adding route to 10.0.2.0/24 lb # [ 16.258667] dhcpcd[722]: eth0: adding default route via 10.0.2.2 lb # [ 16.342178] systemd[1]: Started DHCP Client. lb # [ 16.344251] systemd[1]: Reached target Network is Online. lb # [ 16.347223] systemd[1]: Started Envoy reverse proxy. lb # [ 16.352777] systemd[1]: Started mock-oidc.service. lb # [ 16.439678] mock-oidc-server[928]: Mock OIDC Server running lb # [ 16.441076] mock-oidc-server[928]: OIDC Address: 192.168.1.2:8080 lb # [ 16.441961] mock-oidc-server[928]: Issue Address: 0.0.0.0:8081 lb # [ 16.443244] mock-oidc-server[928]: Issuer: http://192.168.1.2:8080/oidc lb # [ 16.444271] mock-oidc-server[928]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 16.445661] mock-oidc-server[928]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 16.446903] mock-oidc-server[928]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 16.614628] envoy[927]: [2026-09-23 04:41:34.566][927][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104) lb # [ 16.617103] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 16.619098] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 16.621189] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 16.624996] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 16.627452] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 16.629130] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 16.630658] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 16.632830] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 16.636114] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 16.638440] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 16.640128] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 16.641740] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 16.643714] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 16.645682] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 16.648586] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 16.652135] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 16.653853] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 16.657171] envoy[927]: [2026-09-23 04:41:34.568][927][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 16.658800] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 16.660433] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 16.662437] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 16.664114] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 16.667171] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 16.683868] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 16.686443] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 16.689084] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 16.691913] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 16.694262] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 16.696807] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 16.699253] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 16.700958] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 16.704226] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 16.705959] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 16.708497] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 16.717149] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 16.723140] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 16.724617] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 16.725992] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 16.730062] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 16.735889] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 16.740895] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 16.744452] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 16.745954] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 16.747446] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 16.749100] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 16.773314] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 16.777266] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 16.779641] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.781946] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 16.783837] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 16.790537] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 16.793101] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 16.797592] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.799507] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 16.801515] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 16.804325] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 16.807547] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 16.810094] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 16.811780] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 16.813719] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 16.815289] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 16.817082] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 16.822100] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 16.825081] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 16.830658] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.834134] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 16.836719] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 16.838994] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 16.841441] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 16.843087] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 16.847031] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 16.849335] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 16.851628] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 16.853712] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 16.856865] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 16.858921] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 16.861023] envoy[927]: [2026-09-23 04:41:34.569][927][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 16.862852] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 16.869396] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 16.871065] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 16.873445] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.875253] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 16.877413] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 16.879401] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 16.881716] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 16.884494] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 16.886492] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 16.891239] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 16.892766] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 16.894638] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 16.900495] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 16.903380] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 16.906094] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 16.911947] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 16.913556] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 16.916083] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 16.917858] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 16.921073] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 16.931785] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 16.933576] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 16.935825] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 16.937471] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 16.940287] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 16.942513] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 16.943956] envoy[927]: [2026-09-23 04:41:34.571][927][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 16.945321] envoy[927]: [2026-09-23 04:41:34.629][927][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 16.946596] envoy[927]: [2026-09-23 04:41:34.630][927][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 16.959053] envoy[927]: [2026-09-23 04:41:34.630][927][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 16.960419] envoy[927]: [2026-09-23 04:41:34.630][927][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 16.967862] envoy[927]: [2026-09-23 04:41:34.630][927][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 16.971436] envoy[927]: [2026-09-23 04:41:34.923][927][info][main] [source/server/server.cc:968] runtime: {} lb # [ 16.973966] envoy[927]: [2026-09-23 04:41:34.926][927][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 16.975556] envoy[927]: [2026-09-23 04:41:34.926][927][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 16.977194] envoy[927]: [2026-09-23 04:41:34.929][927][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 16.979387] envoy[927]: [2026-09-23 04:41:34.931][927][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 16.981436] envoy[927]: [2026-09-23 04:41:34.933][927][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 16.982835] envoy[927]: [2026-09-23 04:41:34.933][927][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 17.003802] envoy[927]: [2026-09-23 04:41:34.956][927][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker2 # [ 16.970262] systemd[1]: Reached target Network is Online. worker2 # [ 16.973431] systemd[1]: Starting scheduler-token.service... lb # [ 17.007998] envoy[927]: [2026-09-23 04:41:34.960][927][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 17.010092] envoy[927]: [2026-09-23 04:41:34.962][927][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 17.012968] envoy[927]: [2026-09-23 04:41:34.965][927][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 16.997343] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 17.009752] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 17.011405] systemd[1]: Finished scheduler-token.service. worker2 # [ 17.012704] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 37ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 17.016976] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 17.118450] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 17.108547] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 17.129392] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 17.131600] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 17.118684] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 17.120855] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 17.146179] nix-grpc-daemon[781]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 17.148131] nix-grpc-daemon[781]: E0000 00:00:1790138495.224099 781 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 17.133610] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 17.135223] nix-grpc-daemon[783]: E0000 00:00:1790138495.265294 783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 17.154662] nix-grpc-daemon[781]: E0000 00:00:1790138495.232766 781 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 17.157195] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 17.140894] nix-grpc-daemon[783]: E0000 00:00:1790138495.272968 783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 17.161572] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 17.144055] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 17.147337] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 17.179092] nix-grpc-daemon[781]: E0000 00:00:1790138495.257380 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 17.163934] nix-grpc-daemon[783]: E0000 00:00:1790138495.295258 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.203781] envoy[927]: [2026-09-23 04:41:35.155][927][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 17.207925] envoy[927]: [2026-09-23 04:41:35.155][927][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized worker1 # [ 17.195172] nix-grpc-daemon[781]: E0000 00:00:1790138495.273421 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 17.180116] nix-grpc-daemon[783]: E0000 00:00:1790138495.311696 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 17.199156] nix-grpc-daemon[781]: E0000 00:00:1790138495.275727 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.213526] envoy[927]: [2026-09-23 04:41:35.155][927][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager worker2 # [ 17.182175] nix-grpc-daemon[783]: E0000 00:00:1790138495.311782 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.219411] envoy[927]: [2026-09-23 04:41:35.171][927][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker1 # [ 17.210360] nix-grpc-daemon[781]: E0000 00:00:1790138495.288626 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 17.195713] nix-grpc-daemon[783]: E0000 00:00:1790138495.327713 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 17.223613] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.225852] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 17.208505] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.210181] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 17.426458] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.428221] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 17.412448] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.415813] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 17.629084] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.612949] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.614788] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 17.633104] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 17.833518] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.835323] nix-grpc-daemon[781]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 17.816083] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.819815] nix-grpc-daemon[783]: ts=2026-09-23T04:41:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.036160] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.038338] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.020890] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.024602] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.239335] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.241270] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.226776] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.229894] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.442609] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.427227] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.446797] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.430747] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.647201] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.629115] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.651183] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.632910] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.851962] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.833645] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.855920] nix-grpc-daemon[781]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.837216] nix-grpc-daemon[783]: ts=2026-09-23T04:41:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 18.950352] rustfs-bucket-start[963]: mb s3://farm lb # [ 18.962457] systemd[1]: Finished rustfs-bucket.service. lb # [ 18.968097] systemd[1]: Starting niks3 server... worker2 # [ 19.038332] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.056941] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.061183] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.042198] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 19.090302] postgres[975]: [975] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 19.093188] postgres[975]: [975] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 19.115078] niks3-server[970]: 2026/09/23 04:41:37 OK 20241026095416_initial_model.sql (11.84ms) lb # [ 19.118317] niks3-server[970]: 2026/09/23 04:41:37 OK 20251210153512_drop_unused_gin_index.sql (3.49ms) lb # [ 19.123118] niks3-server[970]: 2026/09/23 04:41:37 OK 20251218171726_add_pins.sql (4.86ms) lb # [ 19.129065] niks3-server[970]: 2026/09/23 04:41:37 OK 20260628120000_add_object_size_and_stats.sql (3.78ms) lb # [ 19.137354] niks3-server[970]: 2026/09/23 04:41:37 OK 20260905000000_add_claims.sql (8.33ms) lb # [ 19.141963] niks3-server[970]: 2026/09/23 04:41:37 OK 20260920000000_drop_claims.sql (4.49ms) lb # [ 19.143681] niks3-server[970]: 2026/09/23 04:41:37 goose: successfully migrated database to version: 20260920000000 lb # [ 19.148268] niks3-server[970]: 2026/09/23 04:41:37 OK 1_commit_pending_closure.sql (6.26ms) lb # [ 19.152442] niks3-server[970]: 2026/09/23 04:41:37 OK 2_object_stats_trigger.sql (4.07ms) lb # [ 19.153521] niks3-server[970]: 2026/09/23 04:41:37 goose: up to current file version: 2 lb # [ 19.157604] niks3-server[970]: 2026/09/23 04:41:37 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 19.185518] niks3-server[970]: 2026/09/23 04:41:37 INFO Read proxy enabled — serving cache objects from S3 lb # [ 19.187598] niks3-server[970]: 2026/09/23 04:41:37 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 19.191091] systemd[1]: Started niks3 server. lb # [ 19.191846] niks3-server[970]: 2026/09/23 04:41:37 INFO systemd watchdog enabled interval=15s lb # [ 19.193724] systemd[1]: Reached target Multi-User System. lb # [ 19.195496] niks3-server[970]: 2026/09/23 04:41:37 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 19.197137] systemd[1]: Startup finished in 872ms (kernel) + 4.967s (initrd) + 13.356s (userspace) = 19.195s. worker1 # [ 19.262116] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.244207] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.264216] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.247359] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.465093] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.445425] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.467510] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.450385] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.668492] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.651136] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.672662] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.654917] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.873402] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.856637] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.877560] nix-grpc-daemon[781]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.859846] nix-grpc-daemon[783]: ts=2026-09-23T04:41:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.057884] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.078483] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.062124] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.082636] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 9.62 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.04 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.02 seconds) lb: waiting for TCP port 50051 on localhost worker2 # [ 20.264122] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.284631] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.267448] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.287968] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 20.315356] envoy[927]: conn peer=[::1]:52576 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=2 lb: (finished: waiting for TCP port 50051 on localhost, in 0.02 seconds) worker2 # [ 20.386431] systemd[1]: Starting scheduler-token.service... worker2 # [ 20.425231] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 20.426942] systemd[1]: Finished scheduler-token.service. worker2 # [ 20.428374] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 42ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of x86_64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 20.464783] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.466639] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.485181] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.489231] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.06 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2 # [ 20.667844] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.690071] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.671632] nix-grpc-daemon[783]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.694143] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.872618] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.876379] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.895994] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.899481] nix-grpc-daemon[781]: ts=2026-09-23T04:41:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.077445] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.097349] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.081205] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.101601] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.283346] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.303835] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.286475] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.306948] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.484665] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.486730] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.504465] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.508546] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2 # [ 21.687402] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.690458] nix-grpc-daemon[783]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.709715] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.714485] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.891092] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.892679] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.911792] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.915797] nix-grpc-daemon[781]: ts=2026-09-23T04:41:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.093315] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.095068] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.116709] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.120856] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.296108] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.298218] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.323186] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.326400] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.499525] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.503248] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.524110] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.528077] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2 # [ 22.704387] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.708212] nix-grpc-daemon[783]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.728743] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.733337] nix-grpc-daemon[781]: ts=2026-09-23T04:41:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.908699] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.911340] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.933957] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.938198] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.112403] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.116597] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.139247] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.143284] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.318372] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.321930] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.346197] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.349948] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.522581] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.524308] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.546377] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.548578] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 23.725450] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.729308] nix-grpc-daemon[783]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.749677] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.753706] nix-grpc-daemon[781]: ts=2026-09-23T04:41:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.930347] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.934102] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.954501] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.958484] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.134522] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.137906] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.159123] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.163479] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.339026] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.340689] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.365679] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.369502] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.542142] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.546608] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.570430] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.574631] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 24.747551] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.750778] nix-grpc-daemon[783]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.775397] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.779189] nix-grpc-daemon[781]: ts=2026-09-23T04:41:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.951413] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.953666] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.979981] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.984204] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.154835] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.158620] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.185254] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.189776] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.360700] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.364300] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.392253] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.395349] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.564779] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.566559] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.592891] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.596717] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 25.767273] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.769084] nix-grpc-daemon[783]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.797733] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.801462] nix-grpc-daemon[781]: ts=2026-09-23T04:41:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.969726] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.971600] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.999415] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.003280] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.172186] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.173839] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.203620] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.207310] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.376515] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.380208] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.409432] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.412510] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.577706] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.582335] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.609977] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.613913] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 26.782780] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.784682] nix-grpc-daemon[783]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.814856] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.818881] nix-grpc-daemon[781]: ts=2026-09-23T04:41:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.985875] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.989707] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.019763] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.023634] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.190919] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.195185] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.224122] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.228147] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.395629] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.397436] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.430201] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.433353] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.598702] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.601786] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.630863] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.634684] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 27.802732] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.806508] nix-grpc-daemon[783]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.835658] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.839676] nix-grpc-daemon[781]: ts=2026-09-23T04:41:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.007702] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.011630] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.040519] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.045985] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.212226] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.216175] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.245284] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.249375] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.417708] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.421482] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.451869] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.455118] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.621922] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.623589] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.652764] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.657438] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 28.824957] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 28.828839] nix-grpc-daemon[783]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.858915] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.863722] nix-grpc-daemon[781]: ts=2026-09-23T04:41:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.029805] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.033573] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.064786] nix-grpc-daemon[781]: ts=2026-09-23T04:41:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.069137] nix-grpc-daemon[781]: ts=2026-09-23T04:41:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.172775] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=scheduler_take_over lb # [ 29.207889] niks3-server[970]: 2026/09/23 04:41:47 INFO lead: acquired remote=192.168.1.4:35722 worker2 # [ 29.233849] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.240731] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 29.244295] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:60028 worker1 # [ 29.269549] nix-grpc-daemon[781]: ts=2026-09-23T04:41:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.248230] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 29.254778] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:60028 worker2 # [ 29.258444] nix-grpc-daemon[783]: ts=2026-09-23T04:41:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 9.71 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 30.407296] nix-grpc-daemon[781]: ts=2026-09-23T04:41:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 30.417409] systemd[1]: Started Nix Daemon. worker1 # [ 30.529497] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 30.545906] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5) worker1 # [ 30.593451] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5) worker1 # [ 30.766220] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 3/5) worker1 # [ 30.845618] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 4/5) worker1 # [ 30.953559] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 30.958946] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 30.974475] nix-grpc-daemon[781]: ts=2026-09-23T04:41:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 30.982581] nix-grpc-daemon[781]: ts=2026-09-23T04:41:49Z level=debug event=path_info_miss path=63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test error="error: path '/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 30.992265] nix-grpc-daemon[781]: ts=2026-09-23T04:41:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 30.999194] nix-grpc-daemon[781]: ts=2026-09-23T04:41:49Z level=debug event=path_info_miss path=63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test error="error: path '/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds) client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds) worker2 # [ 31.206855] nix-grpc-daemon[783]: ts=2026-09-23T04:41:49Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:60040 worker2 # [ 31.222238] systemd[1]: Started Nix Daemon. worker2 # [ 31.337876] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 31.352252] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker2 # [ 31.412466] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 102 ms (attempt 2/5) worker2 # [ 31.516952] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5) worker2 # [ 31.814389] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 4/5) worker2 # [ 31.947792] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 31.955381] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 31.997801] nix-grpc-daemon[781]: ts=2026-09-23T04:41:50Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 32.002220] nix-grpc-daemon[781]: ts=2026-09-23T04:41:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:56694 worker1 # [ 32.010459] nix-grpc-daemon[781]: ts=2026-09-23T04:41:50Z level=debug event=path_info_miss path=63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test error="error: path '/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 32.002764] nix-grpc-daemon[783]: ts=2026-09-23T04:41:50Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:60040 worker2 # [ 32.014630] nix-grpc-daemon[783]: ts=2026-09-23T04:41:50Z level=debug event=path_info_miss path=63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test error="error: path '/nix/store/63q6fv5d7k3dddraqacyv4gqrs1j7rnh-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 33.232073] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 33.367113] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.19 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker2 # [ 33.504280] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker2 # [ 33.509903] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 33.530034] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=0 worker2 # [ 33.531711] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 33.582274] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 33.595383] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying 1 paths to 'lb:50051'... worker2 # [ 33.608137] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'... worker2 # [ 33.636825] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 33.667309] nix-grpc-daemon[783]: ts=2026-09-23T04:41:51Z level=info event=niks3_push_started pid=883 worker2 # [ 33.837857] nix-grpc-daemon[883]: time=2026-09-23T04:41:51.967Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 33.941107] nix-daemon[869]: accepted connection from pid 896, user nix-grpc-daemon (trusted) worker2 # [ 33.961378] nix-daemon[869]: reaped child process 899, status = succeeded lb # [ 33.996845] niks3-server[970]: 2026/09/23 04:41:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 33.975188] nix-grpc-daemon[883]: time=2026-09-23T04:41:52.107Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 33.976673] nix-grpc-daemon[883]: time=2026-09-23T04:41:52.108Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)" lb # [ 34.048451] niks3-server[970]: 2026/09/23 04:41:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 34.074116] niks3-server[970]: 2026/09/23 04:41:52 INFO Signed narinfos id=1 count=1 worker2 # [ 34.038768] nix-grpc-daemon[883]: time=2026-09-23T04:41:52.169Z level=INFO msg="Uploading 1 narinfos" lb # [ 34.090430] niks3-server[970]: 2026/09/23 04:41:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 34.096393] niks3-server[970]: 2026/09/23 04:41:52 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst lb # [ 34.101716] niks3-server[970]: 2026/09/23 04:41:52 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls lb # [ 34.103414] niks3-server[970]: 2026/09/23 04:41:52 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo lb # [ 34.109338] niks3-server[970]: 2026/09/23 04:41:52 INFO Completed upload id=1 worker2 # [ 34.074787] nix-grpc-daemon[883]: time=2026-09-23T04:41:52.206Z level=INFO msg="Upload complete. (234ms)" worker2 # [ 34.080254] nix-grpc-daemon[783]: ts=2026-09-23T04:41:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 34.086327] nix-grpc-daemon[783]: ts=2026-09-23T04:41:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 34.099103] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 34.110732] nix-daemon[869]: reaped child process 881, status = succeeded worker2 # [ 34.113667] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051... client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 34.954128] nix-daemon[869]: reaped child process 903, status = succeeded worker2 # [ 35.044646] nix-daemon[869]: accepted connection from pid 920, user nix-grpc-daemon (trusted) worker2 # [ 35.060411] nix-daemon[869]: reaped child process 923, status = succeeded lb # [ 35.104450] niks3-server[970]: 2026/09/23 04:41:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 35.078775] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.210Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 35.080492] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.212Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)" lb # [ 35.144634] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls lb # [ 35.158918] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 35.166367] niks3-server[970]: 2026/09/23 04:41:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 35.172740] niks3-server[970]: 2026/09/23 04:41:53 INFO Signed narinfos id=2 count=1 worker2 # [ 35.138308] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.269Z level=INFO msg="Uploading 1 narinfos" lb # [ 35.177852] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv lb # [ 35.186896] niks3-server[970]: 2026/09/23 04:41:53 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 35.191507] niks3-server[970]: 2026/09/23 04:41:53 INFO Completed upload id=2 worker2 # [ 35.156854] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.288Z level=INFO msg="Upload complete. (235ms)" lb # [ 35.194069] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo worker2 # [ 35.163579] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=1 worker2 # [ 35.177643] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=0 worker2 # [ 35.182126] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 35.190965] nix-daemon[869]: reaped child process 901, status = succeeded worker2 # [ 35.220925] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 35.234706] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 35.248271] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.380Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 35.256515] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'... worker2 # [ 35.274872] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 35.381432] nix-daemon[869]: accepted connection from pid 929, user nix-grpc-daemon (trusted) worker2 # [ 35.397526] nix-daemon[869]: reaped child process 932, status = succeeded lb # [ 35.437235] niks3-server[970]: 2026/09/23 04:41:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 35.638274] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.770Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 35.643121] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.774Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)" lb # [ 35.697868] niks3-server[970]: 2026/09/23 04:41:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 35.703419] niks3-server[970]: 2026/09/23 04:41:53 INFO Signed narinfos id=3 count=1 worker2 # [ 35.672156] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.803Z level=INFO msg="Uploading 1 narinfos" lb # [ 35.718590] niks3-server[970]: 2026/09/23 04:41:53 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 35.767224] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls lb # [ 35.827292] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst lb # [ 35.865845] niks3-server[970]: 2026/09/23 04:41:53 INFO Completed upload id=3 worker2 # [ 35.831439] nix-grpc-daemon[883]: time=2026-09-23T04:41:53.963Z level=INFO msg="Upload complete. (542ms)" worker2 # [ 35.835470] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 35.837916] nix-grpc-daemon[783]: ts=2026-09-23T04:41:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 lb # [ 35.876456] niks3-server[970]: 2026/09/23 04:41:53 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo worker2 # [ 35.842950] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 35.848888] nix-daemon[869]: reaped child process 927, status = succeeded worker2 # [ 35.853567] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051... client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv lb # [ 37.008335] envoy[927]: conn peer=[2001:db8:1::1]:56358 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=363 ms=623 lb # [ 37.011925] envoy[927]: conn peer=[2001:db8:1::1]:56368 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=365 ms=834 lb # [ 37.015233] envoy[927]: conn peer=[2001:db8:1::1]:45738 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=6 lb # [ 37.018917] envoy[927]: conn peer=[2001:db8:1::1]:45754 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=7 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 37.272215] nix-daemon[869]: reaped child process 936, status = succeeded worker2 # [ 37.358516] nix-daemon[869]: accepted connection from pid 952, user nix-grpc-daemon (trusted) worker2 # [ 37.372522] nix-daemon[869]: reaped child process 955, status = succeeded lb # [ 37.414338] niks3-server[970]: 2026/09/23 04:41:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 37.387412] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.519Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 37.389049] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.520Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)" lb # [ 37.449673] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls lb # [ 37.462645] niks3-server[970]: 2026/09/23 04:41:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker2 # [ 37.436052] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.567Z level=INFO msg="Uploading 1 narinfos" lb # [ 37.471570] niks3-server[970]: 2026/09/23 04:41:55 INFO Signed narinfos id=4 count=1 lb # [ 37.475259] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv lb # [ 37.479717] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst lb # [ 37.488348] niks3-server[970]: 2026/09/23 04:41:55 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 37.493740] niks3-server[970]: 2026/09/23 04:41:55 INFO Completed upload id=4 worker2 # [ 37.458971] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.590Z level=INFO msg="Upload complete. (198ms)" lb # [ 37.495739] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo worker2 # [ 37.468848] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=1 worker2 # [ 37.475364] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=0 worker2 # [ 37.479249] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 37.494410] nix-daemon[869]: reaped child process 934, status = succeeded worker2 # [ 37.517079] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 37.528247] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 37.537423] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.669Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 37.543136] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'... worker2 # [ 37.559370] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 37.641990] nix-daemon[869]: accepted connection from pid 962, user nix-grpc-daemon (trusted) worker2 # [ 37.656696] nix-daemon[869]: reaped child process 965, status = succeeded lb # [ 37.698599] niks3-server[970]: 2026/09/23 04:41:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 37.672187] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.804Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 37.673592] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.805Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)" lb # [ 37.729741] niks3-server[970]: 2026/09/23 04:41:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 37.739256] niks3-server[970]: 2026/09/23 04:41:55 INFO Signed narinfos id=5 count=1 worker2 # [ 37.707641] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.839Z level=INFO msg="Uploading 1 narinfos" lb # [ 37.745584] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst lb # [ 37.751909] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls lb # [ 37.764786] niks3-server[970]: 2026/09/23 04:41:55 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 37.770421] niks3-server[970]: 2026/09/23 04:41:55 INFO Completed upload id=5 worker2 # [ 37.735613] nix-grpc-daemon[883]: time=2026-09-23T04:41:55.867Z level=INFO msg="Upload complete. (171ms)" worker2 # [ 37.739666] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 37.745079] nix-grpc-daemon[783]: ts=2026-09-23T04:41:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 lb # [ 37.784550] niks3-server[970]: 2026/09/23 04:41:55 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo worker2 # [ 37.752686] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 37.759337] nix-daemon[869]: reaped child process 960, status = succeeded client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051... worker2 # [ 37.762176] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 38.377806] nix-daemon[869]: reaped child process 969, status = succeeded worker2 # [ 38.454476] nix-daemon[869]: accepted connection from pid 985, user nix-grpc-daemon (trusted) worker2 # [ 38.467815] nix-daemon[869]: reaped child process 988, status = succeeded lb # [ 38.509969] niks3-server[970]: 2026/09/23 04:41:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 38.483677] nix-grpc-daemon[883]: time=2026-09-23T04:41:56.615Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 38.485441] nix-grpc-daemon[883]: time=2026-09-23T04:41:56.617Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)" lb # [ 38.546532] niks3-server[970]: 2026/09/23 04:41:56 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv lb # [ 38.557687] niks3-server[970]: 2026/09/23 04:41:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 38.563117] niks3-server[970]: 2026/09/23 04:41:56 INFO Signed narinfos id=6 count=1 worker2 # [ 38.531290] nix-grpc-daemon[883]: time=2026-09-23T04:41:56.663Z level=INFO msg="Uploading 1 narinfos" lb # [ 38.571124] niks3-server[970]: 2026/09/23 04:41:56 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls lb # [ 38.579345] niks3-server[970]: 2026/09/23 04:41:56 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete worker2 # [ 38.552538] nix-grpc-daemon[883]: time=2026-09-23T04:41:56.684Z level=INFO msg="Upload complete. (181ms)" lb # [ 38.588227] niks3-server[970]: 2026/09/23 04:41:56 INFO Completed upload id=6 lb # [ 38.591200] niks3-server[970]: 2026/09/23 04:41:56 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst worker2 # [ 38.555671] nix-grpc-daemon[783]: ts=2026-09-23T04:41:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=1 lb # [ 38.593579] niks3-server[970]: 2026/09/23 04:41:56 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo worker2 # [ 38.562648] nix-daemon[869]: reaped child process 967, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.23 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds) client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client # copying 3 paths... client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.55 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.14 seconds) worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker1 # [ 39.390560] nix-grpc-daemon[781]: ts=2026-09-23T04:41:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 39.375738] nix-grpc-daemon[783]: ts=2026-09-23T04:41:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 39.386460] nix-grpc-daemon[883]: time=2026-09-23T04:41:57.518Z level=INFO msg="All 2 paths already cached" worker2 # [ 39.402065] nix-grpc-daemon[783]: ts=2026-09-23T04:41:57Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.19 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 0.75 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: worker2 # [ 39.924181] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker2 # [ 39.927065] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 39.934604] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.066Z level=INFO msg="All 2 paths already cached" worker2 # [ 39.945976] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=0 worker2 # [ 39.952593] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 39.957388] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 39.965329] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 39.978761] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.110Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'... worker2 # [ 39.987241] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 39.998383] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 40.102676] nix-daemon[869]: accepted connection from pid 1017, user nix-grpc-daemon (trusted) worker2 # [ 40.118974] nix-daemon[869]: reaped child process 1020, status = succeeded lb # [ 40.161884] niks3-server[970]: 2026/09/23 04:41:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 40.137134] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 40.139189] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.270Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)" lb # [ 40.200562] niks3-server[970]: 2026/09/23 04:41:58 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls lb # [ 40.208135] niks3-server[970]: 2026/09/23 04:41:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker2 # [ 40.181813] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.313Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.216980] niks3-server[970]: 2026/09/23 04:41:58 INFO Signed narinfos id=7 count=1 lb # [ 40.221893] niks3-server[970]: 2026/09/23 04:41:58 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst lb # [ 40.234996] niks3-server[970]: 2026/09/23 04:41:58 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 40.239812] niks3-server[970]: 2026/09/23 04:41:58 INFO Completed upload id=7 worker2 # [ 40.205124] nix-grpc-daemon[883]: time=2026-09-23T04:41:58.337Z level=INFO msg="Upload complete. (195ms)" lb # [ 40.242408] niks3-server[970]: 2026/09/23 04:41:58 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo worker2 # [ 40.210066] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 40.216391] nix-grpc-daemon[783]: ts=2026-09-23T04:41:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 40.227499] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 40.240509] nix-daemon[869]: reaped child process 1015, status = succeeded worker2 # [ 40.243434] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051... client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 41.283501] nix-daemon[869]: reaped child process 1024, status = succeeded worker2 # [ 41.356911] nix-daemon[869]: accepted connection from pid 1041, user nix-grpc-daemon (trusted) worker2 # [ 41.369385] nix-daemon[869]: reaped child process 1044, status = succeeded lb # [ 41.411450] niks3-server[970]: 2026/09/23 04:41:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 41.387114] nix-grpc-daemon[883]: time=2026-09-23T04:41:59.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 41.388657] nix-grpc-daemon[883]: time=2026-09-23T04:41:59.520Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)" lb # [ 41.455434] niks3-server[970]: 2026/09/23 04:41:59 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv lb # [ 41.469597] niks3-server[970]: 2026/09/23 04:41:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker2 # [ 41.439742] nix-grpc-daemon[883]: time=2026-09-23T04:41:59.571Z level=INFO msg="Uploading 1 narinfos" lb # [ 41.474739] niks3-server[970]: 2026/09/23 04:41:59 INFO Signed narinfos id=8 count=1 lb # [ 41.476843] niks3-server[970]: 2026/09/23 04:41:59 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls lb # [ 41.480997] niks3-server[970]: 2026/09/23 04:41:59 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst lb # [ 41.493563] niks3-server[970]: 2026/09/23 04:41:59 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 41.496850] niks3-server[970]: 2026/09/23 04:41:59 INFO Completed upload id=8 worker2 # [ 41.463267] nix-grpc-daemon[883]: time=2026-09-23T04:41:59.595Z level=INFO msg="Upload complete. (211ms)" lb # [ 41.498694] niks3-server[970]: 2026/09/23 04:41:59 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo worker2 # [ 41.469844] nix-grpc-daemon[783]: ts=2026-09-23T04:41:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=1 worker2 # [ 41.484628] nix-daemon[869]: reaped child process 1022, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.72 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.03 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker2 # [ 41.826392] nix-grpc-daemon[783]: ts=2026-09-23T04:41:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 41.831735] nix-grpc-daemon[783]: ts=2026-09-23T04:41:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 41.849747] nix-grpc-daemon[783]: ts=2026-09-23T04:41:59Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=0 worker2 # [ 41.854600] nix-grpc-daemon[783]: ts=2026-09-23T04:41:59Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 41.878950] nix-grpc-daemon[781]: ts=2026-09-23T04:41:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 41.887467] nix-grpc-daemon[781]: ts=2026-09-23T04:41:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'... worker1 # [ 41.899794] nix-grpc-daemon[781]: ts=2026-09-23T04:41:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 41.908692] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 41.930930] nix-grpc-daemon[781]: ts=2026-09-23T04:42:00Z level=info event=niks3_push_started pid=875 worker1 # [ 42.055546] nix-grpc-daemon[875]: time=2026-09-23T04:42:00.133Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 42.145890] nix-daemon[846]: accepted connection from pid 887, user nix-grpc-daemon (trusted) worker1 # [ 42.159320] nix-daemon[846]: reaped child process 890, status = succeeded lb # [ 42.181625] niks3-server[970]: 2026/09/23 04:42:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.173540] nix-grpc-daemon[875]: time=2026-09-23T04:42:00.251Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 42.175024] nix-grpc-daemon[875]: time=2026-09-23T04:42:00.253Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)" lb # [ 42.225877] niks3-server[970]: 2026/09/23 04:42:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 42.230515] niks3-server[970]: 2026/09/23 04:42:00 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst worker1 # [ 42.223730] nix-grpc-daemon[875]: time=2026-09-23T04:42:00.302Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.240531] niks3-server[970]: 2026/09/23 04:42:00 INFO Signed narinfos id=9 count=1 lb # [ 42.251922] niks3-server[970]: 2026/09/23 04:42:00 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls lb # [ 42.260388] niks3-server[970]: 2026/09/23 04:42:00 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 42.266860] niks3-server[970]: 2026/09/23 04:42:00 INFO Completed upload id=9 worker1 # [ 42.250632] nix-grpc-daemon[875]: time=2026-09-23T04:42:00.328Z level=INFO msg="Upload complete. (190ms)" worker1 # [ 42.252672] nix-grpc-daemon[781]: ts=2026-09-23T04:42:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=504 lb # [ 42.269499] niks3-server[970]: 2026/09/23 04:42:00 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo worker1 # [ 42.259383] nix-grpc-daemon[781]: ts=2026-09-23T04:42:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 42.263340] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 42.265707] nix-daemon[846]: reaped child process 873, status = succeeded worker1 # [ 42.270642] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051... client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker1 # [ 43.309546] nix-daemon[846]: reaped child process 894, status = succeeded worker1 # [ 43.389642] nix-daemon[846]: accepted connection from pid 910, user nix-grpc-daemon (trusted) worker1 # [ 43.401870] nix-daemon[846]: reaped child process 913, status = succeeded lb # [ 43.425138] niks3-server[970]: 2026/09/23 04:42:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.415675] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 43.417114] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.495Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)" lb # [ 43.460295] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls lb # [ 43.464597] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 43.468697] niks3-server[970]: 2026/09/23 04:42:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 43.472786] niks3-server[970]: 2026/09/23 04:42:01 INFO Signed narinfos id=10 count=1 worker1 # [ 43.459187] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.537Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.481435] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 43.485413] niks3-server[970]: 2026/09/23 04:42:01 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 43.489578] niks3-server[970]: 2026/09/23 04:42:01 INFO Completed upload id=10 worker1 # [ 43.473450] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.551Z level=INFO msg="Upload complete. (195ms)" lb # [ 43.491070] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo worker1 # [ 43.477353] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 worker2 # [ 43.465392] nix-grpc-daemon[783]: ts=2026-09-23T04:42:01Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=0 worker1 # [ 43.487504] nix-daemon[846]: reaped child process 892, status = succeeded worker2 # [ 43.469438] nix-grpc-daemon[783]: ts=2026-09-23T04:42:01Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 43.526963] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 43.537396] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 43.546591] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.624Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'... worker1 # [ 43.554453] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 43.564316] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 43.653577] nix-daemon[846]: accepted connection from pid 917, user nix-grpc-daemon (trusted) worker1 # [ 43.668209] nix-daemon[846]: reaped child process 920, status = succeeded lb # [ 43.689226] niks3-server[970]: 2026/09/23 04:42:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.680463] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 43.681960] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.760Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)" lb # [ 43.718220] niks3-server[970]: 2026/09/23 04:42:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 43.723563] niks3-server[970]: 2026/09/23 04:42:01 INFO Signed narinfos id=11 count=1 worker1 # [ 43.711084] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.789Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.734131] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst lb # [ 43.741148] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls lb # [ 43.751489] niks3-server[970]: 2026/09/23 04:42:01 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete worker1 # [ 43.741960] nix-grpc-daemon[875]: time=2026-09-23T04:42:01.820Z level=INFO msg="Upload complete. (168ms)" lb # [ 43.758120] niks3-server[970]: 2026/09/23 04:42:01 INFO Completed upload id=11 worker1 # [ 43.744294] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 43.747339] nix-grpc-daemon[781]: ts=2026-09-23T04:42:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 lb # [ 43.764284] niks3-server[970]: 2026/09/23 04:42:01 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo worker1 # [ 43.754522] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 43.756945] nix-daemon[846]: reaped child process 915, status = succeeded client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051... worker1 # [ 43.762177] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker1 # [ 45.181585] nix-daemon[846]: reaped child process 924, status = succeeded worker1 # [ 45.248978] nix-daemon[846]: accepted connection from pid 940, user nix-grpc-daemon (trusted) worker1 # [ 45.261625] nix-daemon[846]: reaped child process 943, status = succeeded lb # [ 45.283780] niks3-server[970]: 2026/09/23 04:42:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.275141] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.353Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 45.276862] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.354Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)" lb # [ 45.315582] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv lb # [ 45.331730] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst lb # [ 45.337890] niks3-server[970]: 2026/09/23 04:42:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker1 # [ 45.330957] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.409Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.346924] niks3-server[970]: 2026/09/23 04:42:03 INFO Signed narinfos id=12 count=1 lb # [ 45.351961] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls lb # [ 45.362187] niks3-server[970]: 2026/09/23 04:42:03 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 45.366264] niks3-server[970]: 2026/09/23 04:42:03 INFO Completed upload id=12 lb # [ 45.367941] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo worker1 # [ 45.350297] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.428Z level=INFO msg="Upload complete. (181ms)" worker1 # [ 45.356799] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 worker2 # [ 45.344433] nix-grpc-daemon[783]: ts=2026-09-23T04:42:03Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=0 worker1 # [ 45.366914] nix-daemon[846]: reaped child process 922, status = succeeded worker2 # [ 45.348859] nix-grpc-daemon[783]: ts=2026-09-23T04:42:03Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 45.405591] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 45.415913] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 45.426920] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.505Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'... worker1 # [ 45.433736] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 45.443159] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 45.534148] nix-daemon[846]: accepted connection from pid 947, user nix-grpc-daemon (trusted) worker1 # [ 45.547101] nix-daemon[846]: reaped child process 950, status = succeeded lb # [ 45.568830] niks3-server[970]: 2026/09/23 04:42:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.560387] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.638Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.561853] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.640Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)" lb # [ 45.600082] niks3-server[970]: 2026/09/23 04:42:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 45.604372] niks3-server[970]: 2026/09/23 04:42:03 INFO Signed narinfos id=13 count=1 worker1 # [ 45.590730] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.669Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.610756] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls lb # [ 45.614904] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst lb # [ 45.625577] niks3-server[970]: 2026/09/23 04:42:03 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 45.629485] niks3-server[970]: 2026/09/23 04:42:03 INFO Completed upload id=13 worker1 # [ 45.613254] nix-grpc-daemon[875]: time=2026-09-23T04:42:03.691Z level=INFO msg="Upload complete. (160ms)" lb # [ 45.631378] niks3-server[970]: 2026/09/23 04:42:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo worker1 # [ 45.617091] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 45.621756] nix-grpc-daemon[781]: ts=2026-09-23T04:42:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 45.631247] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 45.636126] nix-daemon[846]: reaped child process 945, status = succeeded client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051... worker1 # [ 45.639344] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker1 # [ 46.253482] nix-daemon[846]: reaped child process 954, status = succeeded worker1 # [ 46.324391] nix-daemon[846]: accepted connection from pid 970, user nix-grpc-daemon (trusted) worker1 # [ 46.338133] nix-daemon[846]: reaped child process 973, status = succeeded lb # [ 46.358593] niks3-server[970]: 2026/09/23 04:42:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 46.348311] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.426Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 46.350170] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.428Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)" lb # [ 46.382476] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls lb # [ 46.384384] niks3-server[970]: 2026/09/23 04:42:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 46.386411] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv lb # [ 46.389691] niks3-server[970]: 2026/09/23 04:42:04 INFO Signed narinfos id=14 count=1 worker1 # [ 46.373621] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.451Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.392587] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst lb # [ 46.397480] niks3-server[970]: 2026/09/23 04:42:04 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 46.400728] niks3-server[970]: 2026/09/23 04:42:04 INFO Completed upload id=14 worker1 # [ 46.384413] nix-grpc-daemon[875]: time=2026-09-23T04:42:04.462Z level=INFO msg="Upload complete. (139ms)" worker1 # [ 46.386964] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 lb # [ 46.405357] niks3-server[970]: 2026/09/23 04:42:04 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo worker1 # [ 46.394841] nix-daemon[846]: reaped child process 952, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.74 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 46.679367] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55646 worker1 # [ 46.685590] nix-grpc-daemon[781]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55646 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.16 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 46.818826] nix-grpc-daemon[783]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47758 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 46.840365] nix-grpc-daemon[783]: ts=2026-09-23T04:42:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47758 worker2 # [ 46.844115] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'... worker2 # [ 46.854159] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'... worker2 # [ 46.862449] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.16 seconds) client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top client # finding garbage collector roots... lb # [ 47.008901] envoy[927]: conn peer=[2001:db8:1::1]:45764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=4388 ms=5110 lb # [ 47.012679] envoy[927]: conn peer=[2001:db8:1::1]:45768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=44 lb # [ 47.016173] envoy[927]: conn peer=[2001:db8:1::1]:45782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1879 ms=1557 lb # [ 47.019603] envoy[927]: conn peer=[2001:db8:1::1]:45796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4351 ms=4595 client # removing stale temporary roots file "/nix/var/nix/temproots/825" client # removing stale temporary roots file "/nix/var/nix/temproots/808" client # removing stale temporary roots file "/nix/var/nix/temproots/888" client # removing stale temporary roots file "/nix/var/nix/temproots/846" client # removing stale temporary roots file "/nix/var/nix/temproots/858" client # removing stale temporary roots file "/nix/var/nix/temproots/903" client # removing stale temporary roots file "/nix/var/nix/temproots/866" client # removing stale temporary roots file "/nix/var/nix/temproots/880" client # deleting specified paths... client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 47.122672] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.130167] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.138643] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.270Z level=INFO msg="All 1 paths already cached" worker2 # [ 47.143542] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 47.167983] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 client # copying 3 paths... worker2 # [ 47.158759] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'... worker1 # [ 47.190646] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 47.175973] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 47.196435] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 47.183396] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 47.187733] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.191343] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.39 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 5.56 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 47.322869] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 64265fa6e39145dfb90f7da1db5a5f74 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 47.446894] nix[956]: don't know how to build these paths: client # [ 47.447768] nix[956]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker1 # [ 47.433729] nix-grpc-daemon[781]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 47.422137] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker2 # [ 47.427268] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 47.431828] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.441351] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 client # [ 47.482670] nix[956]: copying 1 paths to 'lb:50051'... client # [ 47.483553] nix[956]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'... worker2 # [ 47.452312] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.462674] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 47.567736] nix-daemon[869]: accepted connection from pid 1065, user nix-grpc-daemon (trusted) worker2 # [ 47.582075] nix-daemon[869]: reaped child process 1068, status = succeeded lb # [ 47.624529] niks3-server[970]: 2026/09/23 04:42:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.599691] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 47.601149] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.731Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)" lb # [ 47.656523] niks3-server[970]: 2026/09/23 04:42:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 47.665342] niks3-server[970]: 2026/09/23 04:42:05 INFO Signed narinfos id=15 count=1 worker2 # [ 47.635169] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.767Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.670333] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst lb # [ 47.682004] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls lb # [ 47.692374] niks3-server[970]: 2026/09/23 04:42:05 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 47.698586] niks3-server[970]: 2026/09/23 04:42:05 INFO Completed upload id=15 worker2 # [ 47.663875] nix-grpc-daemon[883]: time=2026-09-23T04:42:05.795Z level=INFO msg="Upload complete. (189ms)" lb # [ 47.702218] niks3-server[970]: 2026/09/23 04:42:05 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo worker2 # [ 47.668833] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 47.674377] nix-grpc-daemon[783]: ts=2026-09-23T04:42:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 47.686834] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 47.696138] nix-daemon[869]: reaped child process 1063, status = succeeded worker2 # [ 47.699291] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # [ 47.730665] nix[956]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client # [ 48.375834] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 8e28dfff5c94458582796e2bfa826dd9 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds) worker2 # [ 48.507601] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 client # [ 48.545180] nix[972]: this derivation will be built: worker1 # [ 48.534551] nix-grpc-daemon[781]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 48.542555] nix-grpc-daemon[781]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'... client # [ 48.568147] nix[972]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker2 # [ 48.540500] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker2 # [ 48.544375] nix-grpc-daemon[783]: ts=2026-09-23T04:42:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 client # [ 48.576443] nix[972]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.04 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 51.718840] nix-daemon[869]: reaped child process 1072, status = succeeded worker2 # [ 51.793364] nix-daemon[869]: accepted connection from pid 1104, user nix-grpc-daemon (trusted) worker2 # [ 51.807090] nix-daemon[869]: reaped child process 1107, status = succeeded lb # [ 51.850169] niks3-server[970]: 2026/09/23 04:42:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.824053] nix-grpc-daemon[883]: time=2026-09-23T04:42:09.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 51.827057] nix-grpc-daemon[883]: time=2026-09-23T04:42:09.957Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)" lb # [ 51.892092] niks3-server[970]: 2026/09/23 04:42:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 51.898455] niks3-server[970]: 2026/09/23 04:42:09 INFO Signed narinfos id=16 count=1 worker2 # [ 51.870472] nix-grpc-daemon[883]: time=2026-09-23T04:42:10.002Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.907089] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv lb # [ 51.912847] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls lb # [ 51.918763] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 51.930139] niks3-server[970]: 2026/09/23 04:42:09 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 51.937409] niks3-server[970]: 2026/09/23 04:42:09 INFO Completed upload id=16 worker2 # [ 51.903927] nix-grpc-daemon[883]: time=2026-09-23T04:42:10.035Z level=INFO msg="Upload complete. (195ms)" worker2 # [ 51.909995] nix-grpc-daemon[783]: ts=2026-09-23T04:42:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 51.918310] nix-grpc-daemon[783]: ts=2026-09-23T04:42:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 51.930577] nix-daemon[869]: reaped child process 1070, status = succeeded client # [ 51.972278] systemd[1]: dedup2.service: Deactivated successfully. client # [ 51.975286] systemd[1]: dedup2.service: Consumed 141ms CPU time over 3.596s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic. lb # [ 51.994844] niks3-server[970]: 2026/09/23 04:42:09 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo client # [ 52.011120] systemd[1]: dedup1.service: Deactivated successfully. client # [ 52.013898] systemd[1]: dedup1.service: Consumed 112ms CPU time over 4.690s wall clock time, 43.5M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.02 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) (finished: subtest: two clients wanting the same drv build it once, in 5.63 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 53.020214] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 53.030216] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=0 worker2 # [ 53.034818] nix-grpc-daemon[783]: ts=2026-09-23T04:42:11Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 53.056844] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 53.069935] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 53.081729] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 53.090986] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 53.195366] nix-daemon[846]: accepted connection from pid 999, user nix-grpc-daemon (trusted) worker1 # [ 53.209993] nix-daemon[846]: reaped child process 1002, status = succeeded lb # [ 53.232760] niks3-server[970]: 2026/09/23 04:42:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.224626] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.226787] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.304Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)" lb # [ 53.263355] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst lb # [ 53.272953] niks3-server[970]: 2026/09/23 04:42:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker1 # [ 53.262432] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.340Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.278668] niks3-server[970]: 2026/09/23 04:42:11 INFO Signed narinfos id=17 count=1 lb # [ 53.287545] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls lb # [ 53.298881] niks3-server[970]: 2026/09/23 04:42:11 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 53.306795] niks3-server[970]: 2026/09/23 04:42:11 INFO Completed upload id=17 worker1 # [ 53.290416] nix-grpc-daemon[875]: time=2026-09-23T04:42:11.368Z level=INFO msg="Upload complete. (185ms)" lb # [ 53.308787] niks3-server[970]: 2026/09/23 04:42:11 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo worker1 # [ 53.295268] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 53.300573] nix-grpc-daemon[781]: ts=2026-09-23T04:42:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 53.312693] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 53.318136] nix-daemon[846]: reaped child process 997, status = succeeded worker1 # [ 53.322046] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) lb # [ 57.008901] envoy[927]: conn peer=[2001:db8:1::1]:42652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=137 lb # [ 57.012735] envoy[927]: conn peer=[2001:db8:1::1]:42670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=78 lb # [ 57.016413] envoy[927]: conn peer=[2001:db8:1::1]:42664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=83 lb # [ 57.020073] envoy[927]: conn peer=[2001:db8:1::1]:42660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=88 lb # [ 57.023642] envoy[927]: conn peer=[2001:db8:1::1]:42682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1598 ms=3416 lb # [ 57.027776] envoy[927]: conn peer=[2001:db8:1::1]:42678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1545 ms=4548 worker1 # [ 58.933421] nix-grpc-daemon[781]: ts=2026-09-23T04:42:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 worker1 # [ 58.957444] nix-daemon[846]: reaped child process 1004, status = succeeded worker1 # [ 58.959427] nix-daemon[846]: reaped child process 1006, status = succeeded worker1 # [ 59.110139] nix-grpc-daemon[781]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 59.103557] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=0 worker2 # [ 59.108303] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.112721] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 59.123949] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 59.134691] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 59.145702] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.247031] nix-daemon[869]: accepted connection from pid 1124, user nix-grpc-daemon (trusted) worker2 # [ 59.261996] nix-daemon[869]: reaped child process 1127, status = succeeded lb # [ 59.305715] niks3-server[970]: 2026/09/23 04:42:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.276953] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.408Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 59.278444] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.408Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)" lb # [ 59.332180] niks3-server[970]: 2026/09/23 04:42:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker2 # [ 59.305740] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.437Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.340443] niks3-server[970]: 2026/09/23 04:42:17 INFO Signed narinfos id=18 count=1 lb # [ 59.347670] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst lb # [ 59.358617] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls lb # [ 59.370342] niks3-server[970]: 2026/09/23 04:42:17 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 59.375802] niks3-server[970]: 2026/09/23 04:42:17 INFO Completed upload id=18 worker2 # [ 59.340971] nix-grpc-daemon[883]: time=2026-09-23T04:42:17.472Z level=INFO msg="Upload complete. (184ms)" lb # [ 59.380587] niks3-server[970]: 2026/09/23 04:42:17 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo worker2 # [ 59.345122] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 59.351431] nix-grpc-daemon[783]: ts=2026-09-23T04:42:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 59.362110] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.366951] nix-daemon[869]: reaped child process 1122, status = succeeded worker2 # [ 59.371696] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 65.788140] nix-grpc-daemon[783]: ts=2026-09-23T04:42:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 worker2 # [ 65.812266] nix-daemon[869]: reaped child process 1129, status = succeeded worker2 # [ 65.815591] nix-daemon[869]: reaped child process 1131, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.02 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share client # [ 65.992545] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 32488c1e4eec489d8c15c6beda8987a3 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.05 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share worker1 # [ 66.114726] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 client # [ 66.133293] nix[1045]: don't know how to build these paths: client # [ 66.134175] nix[1045]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker2 # [ 66.107826] nix-grpc-daemon[783]: ts=2026-09-23T04:42:24Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker1 # [ 66.128803] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 66.112354] nix-grpc-daemon[783]: ts=2026-09-23T04:42:24Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 66.139248] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 client # [ 66.162024] nix[1045]: copying 1 paths to 'lb:50051'... client # [ 66.162895] nix[1045]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'... worker1 # [ 66.150125] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 66.160952] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 66.262924] nix-daemon[846]: accepted connection from pid 1029, user nix-grpc-daemon (trusted) worker1 # [ 66.277182] nix-daemon[846]: reaped child process 1032, status = succeeded lb # [ 66.298922] niks3-server[970]: 2026/09/23 04:42:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.287913] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.365Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.289215] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.365Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)" lb # [ 66.324142] niks3-server[970]: 2026/09/23 04:42:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 66.329262] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst worker1 # [ 66.320404] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.398Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.336142] niks3-server[970]: 2026/09/23 04:42:24 INFO Signed narinfos id=19 count=1 lb # [ 66.345847] niks3-server[970]: 2026/09/23 04:42:24 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 66.365187] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls lb # [ 66.370306] niks3-server[970]: 2026/09/23 04:42:24 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo lb # [ 66.373633] niks3-server[970]: 2026/09/23 04:42:24 INFO Completed upload id=19 worker1 # [ 66.357764] nix-grpc-daemon[875]: time=2026-09-23T04:42:24.435Z level=INFO msg="Upload complete. (185ms)" worker1 # [ 66.361857] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 66.367864] nix-grpc-daemon[781]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 66.378132] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 66.388909] nix-daemon[846]: reaped child process 1027, status = succeeded worker1 # [ 66.392402] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # [ 66.405131] nix[1045]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... lb # [ 67.009224] envoy[927]: conn peer=[2001:db8:1::1]:53586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=5947 lb # [ 67.012953] envoy[927]: conn peer=[2001:db8:1::1]:53590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=6707 client # [ 67.051745] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 0846223b70e64b71b43fd0f4b3d8bd6f client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds) client: must succeed: sleep 2; systemctl kill -s INT share1 worker1 # [ 67.195161] nix-grpc-daemon[781]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 client # [ 67.216114] nix[1061]: this derivation will be built: worker2 # [ 67.187211] nix-grpc-daemon[783]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 67.198349] nix-grpc-daemon[783]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'... client # [ 67.244158] nix[1061]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker2 # [ 67.219249] nix-grpc-daemon[783]: ts=2026-09-23T04:42:25Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker1 # [ 67.245142] nix-grpc-daemon[781]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 client # [ 67.260559] nix[1061]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... client # [ 69.201651] systemd[1]: share1.service: Sent signal SIGINT to main process 1045 (nix) on client request. client # [ 69.208654] nix[1045]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.07 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 69.215677] nix[1045]: error: interrupted by the user client # [ 69.257218] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 69.258846] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 69.260821] systemd[1]: share1.service: Consumed 115ms CPU time over 3.266s wall clock time, 43.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds) client: waiting for success: ! systemctl is-active share2 worker1 # [ 70.635420] nix-daemon[846]: reaped child process 1036, status = succeeded worker1 # [ 70.713712] nix-daemon[846]: accepted connection from pid 1066, user nix-grpc-daemon (trusted) worker1 # [ 70.725086] nix-daemon[846]: reaped child process 1069, status = succeeded lb # [ 70.748897] niks3-server[970]: 2026/09/23 04:42:28 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 70.742109] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 70.743979] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.821Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)" lb # [ 70.777824] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls lb # [ 70.779450] niks3-server[970]: 2026/09/23 04:42:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 70.781868] niks3-server[970]: 2026/09/23 04:42:28 INFO Signed narinfos id=20 count=1 worker1 # [ 70.765761] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.843Z level=INFO msg="Uploading 1 narinfos" lb # [ 70.785260] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 70.789822] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv lb # [ 70.793184] niks3-server[970]: 2026/09/23 04:42:28 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 70.795890] niks3-server[970]: 2026/09/23 04:42:28 INFO Completed upload id=20 worker1 # [ 70.781033] nix-grpc-daemon[875]: time=2026-09-23T04:42:28.859Z level=INFO msg="Upload complete. (178ms)" lb # [ 70.800907] niks3-server[970]: 2026/09/23 04:42:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo worker1 # [ 70.783762] nix-grpc-daemon[781]: ts=2026-09-23T04:42:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker1 # [ 70.787129] nix-grpc-daemon[781]: ts=2026-09-23T04:42:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker1 # [ 70.792787] nix-daemon[846]: reaped child process 1034, status = succeeded client # [ 70.867187] systemd[1]: share2.service: Deactivated successfully. client # [ 70.870749] systemd[1]: share2.service: Consumed 121ms CPU time over 3.818s wall clock time, 33.4M memory peak, 6.4K incoming IP traffic, 6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds) (finished: subtest: the first client leaving does not fail the second, in 5.63 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 71.669425] systemd[1]: Started Nix Daemon. client # [ 71.759812] nix-daemon[1112]: accepted connection from pid 1109, user root (trusted) worker2 # [ 71.842861] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 71.866550] nix-grpc-daemon[781]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker2 # [ 71.856535] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=0 worker2 # [ 71.858734] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 71.861208] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 71.869704] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 71.877143] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 71.880917] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 71.966518] nix-daemon[869]: accepted connection from pid 1173, user nix-grpc-daemon (trusted) worker2 # [ 71.979525] nix-daemon[869]: reaped child process 1176, status = succeeded lb # [ 72.022439] niks3-server[970]: 2026/09/23 04:42:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 71.997215] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 71.999240] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.130Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)" lb # [ 72.052772] niks3-server[970]: 2026/09/23 04:42:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker2 # [ 72.025596] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.157Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.060664] niks3-server[970]: 2026/09/23 04:42:30 INFO Signed narinfos id=21 count=1 lb # [ 72.066569] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls lb # [ 72.070352] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst lb # [ 72.079812] niks3-server[970]: 2026/09/23 04:42:30 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 72.083148] niks3-server[970]: 2026/09/23 04:42:30 INFO Completed upload id=21 worker2 # [ 72.048259] nix-grpc-daemon[883]: time=2026-09-23T04:42:30.180Z level=INFO msg="Upload complete. (160ms)" worker2 # [ 72.050302] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 72.086297] niks3-server[970]: 2026/09/23 04:42:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo worker2 # [ 72.053426] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 72.060721] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 72.063043] nix-daemon[869]: reaped child process 1171, status = succeeded worker2 # [ 72.068110] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 73.305696] nix-daemon[869]: reaped child process 1180, status = succeeded worker2 # [ 73.391089] nix-daemon[869]: accepted connection from pid 1196, user nix-grpc-daemon (trusted) worker2 # [ 73.405337] nix-daemon[869]: reaped child process 1199, status = succeeded lb # [ 73.448591] niks3-server[970]: 2026/09/23 04:42:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 73.617721] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.748Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 73.630706] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.748Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)" lb # [ 73.698979] niks3-server[970]: 2026/09/23 04:42:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 73.709076] niks3-server[970]: 2026/09/23 04:42:31 INFO Signed narinfos id=22 count=1 worker2 # [ 73.677665] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.809Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.718087] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv lb # [ 73.726504] niks3-server[970]: 2026/09/23 04:42:31 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 73.748138] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls lb # [ 73.757562] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 73.781221] niks3-server[970]: 2026/09/23 04:42:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo lb # [ 73.787219] niks3-server[970]: 2026/09/23 04:42:31 INFO Completed upload id=22 worker2 # [ 73.752493] nix-grpc-daemon[883]: time=2026-09-23T04:42:31.884Z level=INFO msg="Upload complete. (475ms)" worker2 # [ 73.760118] nix-grpc-daemon[783]: ts=2026-09-23T04:42:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=1 worker2 # [ 73.776193] nix-daemon[869]: reaped child process 1178, status = succeeded worker2 # [ 73.806086] nix-grpc-daemon[783]: ts=2026-09-23T04:42:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 73.846777] nix-grpc-daemon[781]: ts=2026-09-23T04:42:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 73.858106] nix-grpc-daemon[781]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'... worker1 # [ 73.872348] nix-grpc-daemon[781]: ts=2026-09-23T04:42:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 74.087875] nix-grpc-daemon[781]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 74.094111] nix-grpc-daemon[781]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 74.099704] nix-grpc-daemon[875]: time=2026-09-23T04:42:32.176Z level=INFO msg="All 1 paths already cached" worker2 # [ 74.093583] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=0 worker2 # [ 74.099225] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 74.104137] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 74.115444] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 74.127781] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.259Z level=INFO msg="All 1 paths already cached" worker2 # [ 74.135133] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 74.145267] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 74.254935] nix-daemon[869]: accepted connection from pid 1205, user nix-grpc-daemon (trusted) worker2 # [ 74.272075] nix-daemon[869]: reaped child process 1208, status = succeeded lb # [ 74.312145] niks3-server[970]: 2026/09/23 04:42:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 74.284944] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.417Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 74.286475] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.418Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)" lb # [ 74.346561] niks3-server[970]: 2026/09/23 04:42:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 74.351633] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst worker2 # [ 74.324487] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.456Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.359266] niks3-server[970]: 2026/09/23 04:42:32 INFO Signed narinfos id=23 count=1 lb # [ 74.365871] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls lb # [ 74.377134] niks3-server[970]: 2026/09/23 04:42:32 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 74.386324] niks3-server[970]: 2026/09/23 04:42:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo lb # [ 74.388928] niks3-server[970]: 2026/09/23 04:42:32 INFO Completed upload id=23 worker2 # [ 74.354378] nix-grpc-daemon[883]: time=2026-09-23T04:42:32.486Z level=INFO msg="Upload complete. (196ms)" worker2 # [ 74.359109] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 74.364944] nix-grpc-daemon[783]: ts=2026-09-23T04:42:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 74.378337] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 74.384544] nix-daemon[869]: reaped child process 1203, status = succeeded worker2 # [ 74.388911] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 75.839428] nix-daemon[869]: reaped child process 1212, status = succeeded worker2 # [ 75.932110] nix-daemon[869]: accepted connection from pid 1228, user nix-grpc-daemon (trusted) worker2 # [ 75.949236] nix-daemon[869]: reaped child process 1231, status = succeeded lb # [ 75.988698] niks3-server[970]: 2026/09/23 04:42:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 76.009441] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.141Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 76.012097] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.143Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)" lb # [ 76.080573] niks3-server[970]: 2026/09/23 04:42:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 76.085375] niks3-server[970]: 2026/09/23 04:42:34 INFO Signed narinfos id=24 count=1 worker2 # [ 76.053779] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.185Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.098805] niks3-server[970]: 2026/09/23 04:42:34 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 76.144328] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv lb # [ 76.150457] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst lb # [ 76.159356] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls lb # [ 76.165995] niks3-server[970]: 2026/09/23 04:42:34 INFO Completed upload id=24 worker2 # [ 76.134669] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.266Z level=INFO msg="Upload complete. (336ms)" lb # [ 76.171118] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo worker2 # [ 76.142532] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=1 worker2 # [ 76.158343] nix-daemon[869]: reaped child process 1210, status = succeeded worker1 # [ 76.207368] nix-grpc-daemon[781]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 76.218225] nix-grpc-daemon[781]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'... worker2 # [ 76.225946] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 76.231884] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 76.396412] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 76.418085] nix-grpc-daemon[781]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 76.427841] nix-grpc-daemon[875]: time=2026-09-23T04:42:34.505Z level=INFO msg="All 2 paths already cached" worker2 # [ 76.423256] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=0 worker2 # [ 76.427336] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 76.434793] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 76.446861] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 76.459464] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.591Z level=INFO msg="All 2 paths already cached" worker2 # [ 76.467601] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 76.478180] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 76.586089] nix-daemon[869]: accepted connection from pid 1237, user nix-grpc-daemon (trusted) worker2 # [ 76.602131] nix-daemon[869]: reaped child process 1240, status = succeeded lb # [ 76.644560] niks3-server[970]: 2026/09/23 04:42:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 76.617072] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.749Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 76.618598] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.750Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)" lb # [ 76.678604] niks3-server[970]: 2026/09/23 04:42:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker2 # [ 76.652650] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.784Z level=INFO msg="Uploading 1 narinfos" lb # [ 76.687670] niks3-server[970]: 2026/09/23 04:42:34 INFO Signed narinfos id=25 count=1 lb # [ 76.692605] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls lb # [ 76.699242] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst lb # [ 76.711081] niks3-server[970]: 2026/09/23 04:42:34 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 76.717945] niks3-server[970]: 2026/09/23 04:42:34 INFO Completed upload id=25 worker2 # [ 76.683235] nix-grpc-daemon[883]: time=2026-09-23T04:42:34.815Z level=INFO msg="Upload complete. (192ms)" lb # [ 76.724294] niks3-server[970]: 2026/09/23 04:42:34 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo worker2 # [ 76.687373] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 76.692618] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 76.705065] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 76.709587] nix-daemon[869]: reaped child process 1235, status = succeeded worker2 # [ 76.714678] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) lb # [ 77.010318] envoy[927]: conn peer=[2001:db8:1::1]:59662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=767 ms=3097 lb # [ 77.013917] envoy[927]: conn peer=[2001:db8:1::1]:59664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1602 tx=1598 ms=3642 lb # [ 77.017440] envoy[927]: conn peer=[2001:db8:1::1]:45004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=81 lb # [ 77.020973] envoy[927]: conn peer=[2001:db8:1::1]:59670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1962 ms=2064 lb # [ 77.024590] envoy[927]: conn peer=[2001:db8:1::1]:45016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=17 lb # [ 77.028155] envoy[927]: conn peer=[2001:db8:1::1]:45012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2141 ms=2174 worker2 # [ 77.755816] nix-daemon[869]: reaped child process 1244, status = succeeded worker2 # [ 77.844992] nix-daemon[869]: accepted connection from pid 1260, user nix-grpc-daemon (trusted) worker2 # [ 77.858855] nix-daemon[869]: reaped child process 1263, status = succeeded lb # [ 77.902570] niks3-server[970]: 2026/09/23 04:42:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 77.877051] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 77.878539] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.010Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)" lb # [ 77.949741] niks3-server[970]: 2026/09/23 04:42:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 77.958359] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst lb # [ 77.963697] niks3-server[970]: 2026/09/23 04:42:35 INFO Signed narinfos id=26 count=1 worker2 # [ 77.932128] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.063Z level=INFO msg="Uploading 1 narinfos" lb # [ 77.967576] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv lb # [ 77.972792] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls lb # [ 77.984901] niks3-server[970]: 2026/09/23 04:42:35 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 77.988992] niks3-server[970]: 2026/09/23 04:42:35 INFO Completed upload id=26 worker2 # [ 77.954459] nix-grpc-daemon[883]: time=2026-09-23T04:42:36.086Z level=INFO msg="Upload complete. (231ms)" lb # [ 77.991413] niks3-server[970]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo worker2 # [ 77.961466] nix-grpc-daemon[783]: ts=2026-09-23T04:42:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=1 worker2 # [ 77.970622] nix-grpc-daemon[783]: ts=2026-09-23T04:42:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 77.982707] nix-daemon[869]: reaped child process 1242, status = succeeded worker1 # [ 78.006286] nix-grpc-daemon[781]: ts=2026-09-23T04:42:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 78.015921] nix-grpc-daemon[781]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'... worker1 # [ 78.029173] nix-grpc-daemon[781]: ts=2026-09-23T04:42:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=232 client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.57 seconds) client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client # [ 78.192557] nix-daemon[1112]: reaped child process 1114, status = succeeded client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.03 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.65 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds) client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds) worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 78.823794] nix-daemon[1112]: accepted connection from pid 1195, user root (trusted) client # this derivation will be built: client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker1 # [ 78.946915] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:56694 client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 78.952933] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 79.055182] nix-daemon[846]: accepted connection from pid 1079, user nix-grpc-daemon (trusted) worker1 # [ 79.066513] nix-daemon[846]: reaped child process 1082, status = succeeded lb # [ 79.090281] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 79.082241] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 79.084275] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.162Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)" lb # [ 79.115841] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 79.110186] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.188Z level=INFO msg="Uploading 1 narinfos" lb # [ 79.126760] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=27 count=1 lb # [ 79.134112] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls lb # [ 79.140313] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 79.152686] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 79.163393] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=27 worker1 # [ 79.150134] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.228Z level=INFO msg="Upload complete. (198ms)" lb # [ 79.167130] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo worker2 # [ 79.151786] nix-grpc-daemon[783]: ts=2026-09-23T04:42:37Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=0 worker2 # [ 79.156265] nix-grpc-daemon[783]: ts=2026-09-23T04:42:37Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 79.214933] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 79.228212] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 79.245331] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 79.256628] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 79.362459] nix-daemon[846]: accepted connection from pid 1086, user nix-grpc-daemon (trusted) worker1 # [ 79.376978] nix-daemon[846]: reaped child process 1089, status = succeeded lb # [ 79.397424] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 79.407180] niks3-server[970]: 2026/09/23 04:42:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 79.399793] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.478Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 79.401303] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.479Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)" worker1 # [ 79.403087] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.481Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)" lb # [ 79.459125] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls lb # [ 79.461174] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst lb # [ 79.465967] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 79.470263] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=28 count=1 lb # [ 79.474394] niks3-server[970]: 2026/09/23 04:42:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 79.477077] niks3-server[970]: 2026/09/23 04:42:37 INFO Signed narinfos id=29 count=2 lb # [ 79.478927] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst worker1 # [ 79.462548] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.540Z level=INFO msg="Uploading 3 narinfos" lb # [ 79.481487] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls lb # [ 79.494478] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 79.497839] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=28 lb # [ 79.500062] niks3-server[970]: 2026/09/23 04:42:37 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 79.502659] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo lb # [ 79.505621] niks3-server[970]: 2026/09/23 04:42:37 INFO Completed upload id=29 lb # [ 79.507249] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo worker1 # [ 79.490947] nix-grpc-daemon[875]: time=2026-09-23T04:42:37.569Z level=INFO msg="Upload complete. (219ms)" lb # [ 79.508989] niks3-server[970]: 2026/09/23 04:42:37 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo worker1 # [ 79.496393] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=2 nar_bytes_in=976 worker1 # [ 79.502869] nix-grpc-daemon[781]: ts=2026-09-23T04:42:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 79.515107] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 79.525214] nix-daemon[846]: reaped child process 1084, status = succeeded client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051... worker1 # [ 79.531560] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 80.570281] nix-daemon[846]: reaped child process 1093, status = succeeded worker1 # [ 80.647385] nix-daemon[846]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker1 # [ 80.660214] nix-daemon[846]: reaped child process 1112, status = succeeded lb # [ 80.685360] niks3-server[970]: 2026/09/23 04:42:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.676160] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.754Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 80.713168] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls lb # [ 80.721566] niks3-server[970]: 2026/09/23 04:42:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 80.728865] niks3-server[970]: 2026/09/23 04:42:38 INFO Signed narinfos id=30 count=1 worker1 # [ 80.715751] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.793Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.736355] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv lb # [ 80.750337] niks3-server[970]: 2026/09/23 04:42:38 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 80.756623] niks3-server[970]: 2026/09/23 04:42:38 INFO Completed upload id=30 worker1 # [ 80.740720] nix-grpc-daemon[875]: time=2026-09-23T04:42:38.818Z level=INFO msg="Upload complete. (201ms)" lb # [ 80.761211] niks3-server[970]: 2026/09/23 04:42:38 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo client # copying 1 paths... worker1 # [ 80.746618] nix-grpc-daemon[781]: ts=2026-09-23T04:42:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 80.736308] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker1 # [ 80.761086] nix-daemon[846]: reaped child process 1091, status = succeeded worker2 # [ 80.744701] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 80.777472] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 80.783203] nix-grpc-daemon[783]: ts=2026-09-23T04:42:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=120 client # [ 80.923395] nix-daemon[1112]: reaped child process 1198, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.23 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 81.028095] nix-daemon[1112]: accepted connection from pid 1217, user root (trusted) client # this derivation will be built: client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker2 # [ 81.111235] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60040 client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 81.133100] nix-grpc-daemon[781]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 81.143260] nix-grpc-daemon[875]: time=2026-09-23T04:42:39.221Z level=INFO msg="All 1 paths already cached" worker2 # [ 81.139264] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=0 worker2 # [ 81.143636] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 81.193692] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 81.206575] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 81.225664] nix-grpc-daemon[783]: ts=2026-09-23T04:42:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 81.234977] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 81.254199] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5) worker2 # [ 81.305894] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 2/5) worker2 # [ 81.419385] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5) worker2 # [ 81.731691] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 4/5) worker2 # [ 81.939977] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 81.944361] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 81.949547] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 82.073407] nix-daemon[869]: accepted connection from pid 1280, user nix-grpc-daemon (trusted) worker2 # [ 82.093518] nix-daemon[869]: reaped child process 1283, status = succeeded lb # [ 82.134511] niks3-server[970]: 2026/09/23 04:42:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 82.108551] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.239Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 82.109999] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.239Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)" lb # [ 82.161933] niks3-server[970]: 2026/09/23 04:42:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker2 # [ 82.132806] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.264Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.168173] niks3-server[970]: 2026/09/23 04:42:40 INFO Signed narinfos id=31 count=1 lb # [ 82.173869] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls lb # [ 82.186019] niks3-server[970]: 2026/09/23 04:42:40 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 82.191182] niks3-server[970]: 2026/09/23 04:42:40 INFO Completed upload id=31 worker2 # [ 82.157507] nix-grpc-daemon[883]: time=2026-09-23T04:42:40.289Z level=INFO msg="Upload complete. (189ms)" lb # [ 82.193237] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst worker2 # [ 82.160461] nix-grpc-daemon[783]: ts=2026-09-23T04:42:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_in=560 lb # [ 82.196340] niks3-server[970]: 2026/09/23 04:42:40 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo worker2 # [ 82.164194] nix-grpc-daemon[783]: ts=2026-09-23T04:42:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 82.170472] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 82.177169] nix-daemon[869]: reaped child process 1273, status = succeeded worker2 # [ 82.183384] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'... worker2 # [ 82.190462] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 83.229718] nix-daemon[869]: reaped child process 1288, status = succeeded worker2 # [ 83.321826] nix-daemon[869]: accepted connection from pid 1304, user nix-grpc-daemon (trusted) worker2 # [ 83.337588] nix-daemon[869]: reaped child process 1307, status = succeeded lb # [ 83.379242] niks3-server[970]: 2026/09/23 04:42:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 83.359939] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.491Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 83.418426] niks3-server[970]: 2026/09/23 04:42:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker2 # [ 83.393062] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.524Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.428175] niks3-server[970]: 2026/09/23 04:42:41 INFO Signed narinfos id=32 count=1 lb # [ 83.433884] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls lb # [ 83.440699] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv lb # [ 83.449216] niks3-server[970]: 2026/09/23 04:42:41 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 83.471124] niks3-server[970]: 2026/09/23 04:42:41 INFO Completed upload id=32 worker2 # [ 83.436592] nix-grpc-daemon[883]: time=2026-09-23T04:42:41.568Z level=INFO msg="Upload complete. (236ms)" lb # [ 83.474253] niks3-server[970]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo worker2 # [ 83.444497] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker2 # [ 83.455313] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 83.465694] nix-daemon[869]: reaped child process 1285, status = succeeded worker1 # [ 83.497330] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 83.509387] nix-grpc-daemon[781]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'... worker1 # [ 83.523983] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_out=120 client # [ 83.644473] nix-daemon[1112]: reaped child process 1220, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.72 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 83.743628] nix-daemon[1112]: accepted connection from pid 1239, user root (trusted) client # this derivation will be built: client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker2 # [ 83.821616] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60040 client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 83.844070] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 83.852308] nix-grpc-daemon[875]: time=2026-09-23T04:42:41.929Z level=INFO msg="All 1 paths already cached" worker2 # [ 83.848168] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=0 worker2 # [ 83.852967] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 83.874492] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 83.886338] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 83.904411] nix-grpc-daemon[875]: time=2026-09-23T04:42:41.982Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 83.912674] nix-grpc-daemon[781]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 83.921695] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 84.022583] nix-daemon[846]: accepted connection from pid 1117, user nix-grpc-daemon (trusted) worker1 # [ 84.036425] nix-daemon[846]: reaped child process 1120, status = succeeded lb # [ 84.057461] niks3-server[970]: 2026/09/23 04:42:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 84.048816] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.127Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 84.050494] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.128Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)" lb # [ 84.087125] niks3-server[970]: 2026/09/23 04:42:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker1 # [ 84.079491] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.157Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.095295] niks3-server[970]: 2026/09/23 04:42:42 INFO Signed narinfos id=33 count=1 lb # [ 84.110202] niks3-server[970]: 2026/09/23 04:42:42 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 84.113910] niks3-server[970]: 2026/09/23 04:42:42 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst lb # [ 84.119091] niks3-server[970]: 2026/09/23 04:42:42 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls lb # [ 84.124252] niks3-server[970]: 2026/09/23 04:42:42 INFO Completed upload id=33 worker1 # [ 84.108475] nix-grpc-daemon[875]: time=2026-09-23T04:42:42.186Z level=INFO msg="Upload complete. (172ms)" worker1 # [ 84.112622] nix-grpc-daemon[781]: ts=2026-09-23T04:42:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=0 paths=1 nar_bytes_in=560 worker1 # [ 84.117439] nix-grpc-daemon[781]: ts=2026-09-23T04:42:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 84.127065] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) worker1 # [ 84.132253] nix-daemon[846]: reaped child process 1115, status = succeeded client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051... worker1 # [ 84.136777] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 85.176281] nix-daemon[846]: reaped child process 1124, status = succeeded worker1 # [ 85.252088] nix-daemon[846]: accepted connection from pid 1140, user nix-grpc-daemon (trusted) worker1 # [ 85.264246] nix-daemon[846]: reaped child process 1143, status = succeeded lb # [ 85.288534] niks3-server[970]: 2026/09/23 04:42:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 85.279267] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.357Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 85.314890] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls lb # [ 85.324387] niks3-server[970]: 2026/09/23 04:42:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 85.329487] niks3-server[970]: 2026/09/23 04:42:43 INFO Signed narinfos id=34 count=1 worker1 # [ 85.317927] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.396Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.339723] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv lb # [ 85.350118] niks3-server[970]: 2026/09/23 04:42:43 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 85.357925] niks3-server[970]: 2026/09/23 04:42:43 INFO Completed upload id=34 worker1 # [ 85.341644] nix-grpc-daemon[875]: time=2026-09-23T04:42:43.419Z level=INFO msg="Upload complete. (197ms)" lb # [ 85.362975] niks3-server[970]: 2026/09/23 04:42:43 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo worker1 # [ 85.346795] nix-grpc-daemon[781]: ts=2026-09-23T04:42:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56694 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker1 # [ 85.361531] nix-daemon[846]: reaped child process 1122, status = succeeded client # copying 1 paths... worker2 # [ 85.376499] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 85.388737] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 85.420789] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 worker2 # [ 85.426407] nix-grpc-daemon[783]: ts=2026-09-23T04:42:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60040 duration_s=0 paths=1 nar_bytes_out=120 client # [ 85.567379] nix-daemon[1112]: reaped child process 1242, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.92 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 7.35 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.23 seconds) client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds) client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds) worker2 # [ 86.274581] systemd[1]: Stopping Nix worker-protocol over gRPC... lb # [ 87.010427] envoy[927]: conn peer=[2001:db8:1::1]:45034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=75 lb # [ 87.014129] envoy[927]: conn peer=[2001:db8:1::1]:45026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2372 ms=1668 lb # [ 87.017710] envoy[927]: conn peer=[2001:db8:1::1]:45048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3849 tx=2066 ms=1906 lb # [ 87.021336] envoy[927]: conn peer=[2001:db8:1::1]:45054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=64 lb # [ 87.024954] envoy[927]: conn peer=[2001:db8:1::1]:45056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2123 ms=2447 lb # [ 87.028550] envoy[927]: conn peer=[2001:db8:1::1]:36148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=82 lb # [ 87.031994] envoy[927]: conn peer=[2001:db8:1::1]:36168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=59 lb # [ 87.036201] envoy[927]: conn peer=[2001:db8:1::1]:36162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=2108 ms=1657 worker2 # [ 87.171359] nix-grpc-daemon[783]: ts=2026-09-23T04:42:45Z level=info event=signal_exit lb # [ 87.212861] niks3-server[970]: 2026/09/23 04:42:45 INFO lead: released remote=192.168.1.4:35722 worker1 # [ 88.197569] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 88.182388] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting worker2 # [ 88.186518] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60028 worker2 # [ 88.190226] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 88.193626] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 88.196871] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60028 worker2 # [ 88.200660] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 88.222652] nix-daemon[869]: reaped child process 871, status = succeeded worker2 # [ 88.327804] nix-grpc-daemon[783]: E0000 00:00:1790138566.459686 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker2 # [ 88.329916] nix-grpc-daemon[783]: E0000 00:00:1790138566.459832 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 88.397941] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.402346] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 88.387555] nix-grpc-daemon[783]: ts=2026-09-23T04:42:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 88.603276] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.607583] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 88.808842] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 88.812903] nix-grpc-daemon[781]: ts=2026-09-23T04:42:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 88.961585] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 88.966361] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 88.969902] systemd[1]: nix-grpc-daemon.service: Consumed 2.554s CPU time over 1min 11.948s wall clock time, 68.4M memory peak, 110.4K incoming IP traffic, 275.9K outgoing IP traffic. worker2 # [ 88.982162] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 88.986309] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 88.990710] systemd[1]: nix-grpc-daemon.socket: Consumed 326us CPU time over 1min 21.309s wall clock time, 256K memory peak, 119.1K incoming IP traffic, 101.8K outgoing IP traffic. worker1 # [ 89.013725] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:50051 lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 89.017731] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2 worker1 # [ 89.189337] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_take_over lb # [ 89.205716] niks3-server[970]: 2026/09/23 04:42:47 INFO lead: acquired remote=192.168.1.3:52858 worker1 # [ 89.208155] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56694 client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker1 # [ 89.213283] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56694 worker1 # [ 89.218198] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.221904] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds) worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds) worker2 # [ 89.275218] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 89.340514] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 89.422907] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 89.434209] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:39268 worker1 # [ 89.438008] nix-grpc-daemon[781]: ts=2026-09-23T04:42:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 89.956863] systemd[1]: Starting scheduler-token.service... worker2 # [ 90.018092] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 90.020318] systemd[1]: Finished scheduler-token.service. worker2 # [ 90.023207] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 61ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 90.031363] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 90.116662] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 90.121162] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 90.124346] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 90.126300] nix-grpc-daemon[1334]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 90.127568] nix-grpc-daemon[1334]: E0000 00:00:1790138568.255814 1334 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 90.136770] nix-grpc-daemon[1334]: E0000 00:00:1790138568.264054 1334 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 90.139087] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 90.143887] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 90.146577] nix-grpc-daemon[1334]: E0000 00:00:1790138568.278588 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 90.159907] nix-grpc-daemon[1334]: E0000 00:00:1790138568.291925 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 90.164056] nix-grpc-daemon[1334]: E0000 00:00:1790138568.294198 1376 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 90.189435] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=signal_exit worker2 # [ 90.174384] nix-grpc-daemon[1334]: E0000 00:00:1790138568.306399 1376 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 90.211154] niks3-server[970]: 2026/09/23 04:42:48 INFO lead: released remote=192.168.1.3:52858 worker2 # [ 90.187072] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 90.207955] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:39268 worker1 # [ 90.213126] nix-grpc-daemon[781]: ts=2026-09-23T04:42:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 91.177536] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 91.194104] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting worker1 # [ 91.201368] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:39268 worker1 # [ 91.206392] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 91.210653] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 91.213871] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:39268 worker1 # [ 91.217290] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 91.237441] nix-daemon[846]: reaped child process 848, status = succeeded worker1 # [ 91.314638] nix-grpc-daemon[781]: E0000 00:00:1790138569.392717 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 91.319554] nix-grpc-daemon[781]: E0000 00:00:1790138569.392852 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 91.378196] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 91.382299] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 91.413399] nix-grpc-daemon[781]: ts=2026-09-23T04:42:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 91.583429] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 91.587524] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 91.788722] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 91.792721] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 91.993907] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 91.997957] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 92.200452] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 92.203589] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 92.401553] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 92.405440] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 92.606623] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 92.610561] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 92.811888] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 92.815864] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 92.970947] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 92.975298] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 92.978861] systemd[1]: nix-grpc-daemon.service: Consumed 1.894s CPU time over 1min 15.980s wall clock time, 60.7M memory peak, 83.5K incoming IP traffic, 153.4K outgoing IP traffic. worker1 # [ 92.991146] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 92.995403] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 92.998746] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 1min 25.313s wall clock time, 256K memory peak, 92.7K incoming IP traffic, 81.4K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 93.016907] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2 worker2 # [ 93.020845] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 93.162665] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 93.173767] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 93.189806] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5) worker2 # [ 93.222638] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 93.226455] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 93.232384] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 2/5) worker2 # [ 93.269800] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 3/5) worker2 # [ 93.410590] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 600 ms (attempt 4/5) worker2 # [ 93.427391] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 93.429601] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 93.630541] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 93.632541] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 93.833422] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 93.835524] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 94.012576] nix-grpc-daemon[1334]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 94.014110] nix-grpc-daemon[1334]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... worker2 # [ 94.031072] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'... client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 94.035843] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 94.037707] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 94.041548] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 94.054286] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=niks3_push_started pid=1395 worker2 # [ 94.153531] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.285Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 94.239729] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 94.242659] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 94.247066] nix-daemon[869]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 94.259509] nix-daemon[869]: reaped child process 1410, status = succeeded lb # [ 94.302886] niks3-server[970]: 2026/09/23 04:42:52 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 94.314564] niks3-server[970]: 2026/09/23 04:42:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.288892] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.420Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 94.292053] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.422Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)" worker2 # [ 94.293656] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.422Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)" lb # [ 94.369839] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls lb # [ 94.376118] niks3-server[970]: 2026/09/23 04:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 94.388489] niks3-server[970]: 2026/09/23 04:42:52 INFO Signed narinfos id=36 count=2 lb # [ 94.390262] niks3-server[970]: 2026/09/23 04:42:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 94.392532] niks3-server[970]: 2026/09/23 04:42:52 INFO Signed narinfos id=35 count=1 worker2 # [ 94.360092] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.491Z level=INFO msg="Uploading 3 narinfos" lb # [ 94.396581] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 94.401867] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls lb # [ 94.409908] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst lb # [ 94.419879] niks3-server[970]: 2026/09/23 04:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 94.424795] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo lb # [ 94.431084] niks3-server[970]: 2026/09/23 04:42:52 INFO Completed upload id=35 lb # [ 94.432715] niks3-server[970]: 2026/09/23 04:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 94.436278] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo lb # [ 94.439028] niks3-server[970]: 2026/09/23 04:42:52 INFO Completed upload id=36 worker2 # [ 94.404572] nix-grpc-daemon[1395]: time=2026-09-23T04:42:52.536Z level=INFO msg="Upload complete. (245ms)" lb # [ 94.442629] niks3-server[970]: 2026/09/23 04:42:52 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo worker2 # [ 94.409956] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 94.424220] nix-daemon[869]: reaped child process 1393, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.42 seconds) worker2 # [ 94.442367] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache worker2 # [ 94.446101] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds) worker1 # [ 94.533666] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 94.543270] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 94.632394] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 94.636928] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 94.640110] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 94.642088] nix-grpc-daemon[1166]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 94.643390] nix-grpc-daemon[1166]: E0000 00:00:1790138572.715129 1166 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 94.651849] nix-grpc-daemon[1166]: E0000 00:00:1790138572.724144 1166 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 94.669348] niks3-server[970]: 2026/09/23 04:42:52 INFO lead: acquired remote=192.168.1.3:57370 worker1 # [ 94.656100] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 94.659121] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 94.661055] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_take_over worker1 # [ 94.665140] nix-grpc-daemon[1166]: E0000 00:00:1790138572.733226 1207 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 94.647261] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 94.651129] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 94.673207] nix-grpc-daemon[1166]: E0000 00:00:1790138572.750321 1207 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 94.676183] nix-grpc-daemon[1166]: E0000 00:00:1790138572.750628 1207 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly no endpoints of x86_64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 94.687045] nix-grpc-daemon[1166]: E0000 00:00:1790138572.765266 1207 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 94.698908] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 94.700470] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 94.852393] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 94.856417] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 94.901512] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 94.905554] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 95.057642] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 95.088171] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 95.091905] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:39766 worker1 # [ 95.095828] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1 # [ 95.106111] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 95.110446] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:39766 worker1 # [ 95.113972] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.22 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb # [ 95.870370] niks3-server[970]: 2026/09/23 04:42:53 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 95.874350] systemd[1]: Stopping niks3 server... worker2 # [ 95.837243] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:53Z level=info event=scheduler_lock_lost status=200 lb # [ 95.877431] niks3-server[970]: 2026/09/23 04:42:53 INFO lead: released remote=192.168.1.3:57370 worker1 # [ 95.862876] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:53Z level=info event=scheduler_lock_lost status=200 lb # [ 95.888833] systemd[1]: niks3.service: Deactivated successfully. lb # [ 95.890445] systemd[1]: Stopped niks3 server. lb # [ 95.892080] systemd[1]: niks3.service: Consumed 750ms CPU time over 1min 16.921s wall clock time, 29M memory peak, 100.7K incoming IP traffic, 143.9K outgoing IP traffic. lb # [ 95.898440] systemd[1]: Starting niks3 server... lb # [ 95.948352] niks3-server[1112]: 2026/09/23 04:42:53 goose: no migrations to run. current version: 20260920000000 lb # [ 95.953495] niks3-server[1112]: 2026/09/23 04:42:53 OK 1_commit_pending_closure.sql (5.98ms) lb # [ 95.956993] niks3-server[1112]: 2026/09/23 04:42:53 OK 2_object_stats_trigger.sql (3.13ms) lb # [ 95.958073] niks3-server[1112]: 2026/09/23 04:42:53 goose: up to current file version: 2 lb # [ 95.959393] niks3-server[1112]: 2026/09/23 04:42:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 95.970268] niks3-server[1112]: 2026/09/23 04:42:53 INFO Read proxy enabled — serving cache objects from S3 lb # [ 95.971626] niks3-server[1112]: 2026/09/23 04:42:53 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 95.973080] niks3-server[1112]: 2026/09/23 04:42:53 INFO systemd watchdog enabled interval=15s lb # [ 95.974686] systemd[1]: Started niks3 server. lb # [ 95.975766] niks3-server[1112]: 2026/09/23 04:42:53 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 96.284155] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39774 worker1 # [ 96.296224] nix-daemon[846]: accepted connection from pid 1166, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker2 # [ 96.293180] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker1 # [ 96.336754] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=0 worker1 # [ 96.341150] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:54Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 96.371926] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 96.384736] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'... worker2 # [ 96.397232] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 96.408516] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 96.515471] nix-daemon[869]: accepted connection from pid 1417, user nix-grpc-daemon (trusted) worker2 # [ 96.529402] nix-daemon[869]: reaped child process 1420, status = succeeded lb # [ 96.571310] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 96.547781] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.679Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 96.550275] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.681Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)" lb # [ 96.610797] niks3-server[1112]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst lb # [ 96.616617] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 96.635994] niks3-server[1112]: 2026/09/23 04:42:54 INFO Signed narinfos id=37 count=1 worker2 # [ 96.604811] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.736Z level=INFO msg="Uploading 1 narinfos" lb # [ 96.652861] niks3-server[1112]: 2026/09/23 04:42:54 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 96.657775] niks3-server[1112]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls lb # [ 96.660974] niks3-server[1112]: 2026/09/23 04:42:54 INFO Completed upload id=37 worker2 # [ 96.626374] nix-grpc-daemon[1395]: time=2026-09-23T04:42:54.758Z level=INFO msg="Upload complete. (205ms)" worker2 # [ 96.631585] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=536 worker2 # [ 96.638450] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 96.650141] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 96.662177] nix-daemon[869]: reaped child process 1415, status = succeeded worker2 # [ 96.664932] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051... client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 96.883161] niks3-server[1112]: 2026/09/23 04:42:54 INFO lead: acquired remote=192.168.1.3:57386 lb # [ 97.010604] envoy[927]: conn peer=[2001:db8:1::4]:50354 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=51856 tx=3112 ms=71181 lb # [ 97.014441] envoy[927]: conn peer=[2001:db8:1::1]:36178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=15 lb # [ 97.017874] envoy[927]: conn peer=[2001:db8:1::3]:59812 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15015 tx=3132 ms=74187 lb # [ 97.021544] envoy[927]: conn peer=[2001:db8:1::1]:45052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=305 ms=1261 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 97.897460] nix-daemon[869]: reaped child process 1424, status = succeeded worker2 # [ 97.990820] nix-daemon[869]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) worker2 # [ 98.003701] nix-daemon[869]: reaped child process 1445, status = succeeded lb # [ 98.045853] niks3-server[1112]: 2026/09/23 04:42:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 98.019773] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 98.021470] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.153Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)" lb # [ 98.084427] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 98.113520] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls lb # [ 98.117304] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 98.123530] niks3-server[1112]: 2026/09/23 04:42:56 INFO Signed narinfos id=38 count=1 lb # [ 98.126259] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst worker2 # [ 98.091807] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.223Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.132551] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 98.137464] niks3-server[1112]: 2026/09/23 04:42:56 INFO Completed upload id=38 worker2 # [ 98.102839] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.234Z level=INFO msg="Upload complete. (228ms)" lb # [ 98.142405] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo worker2 # [ 98.109895] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 worker1 # [ 98.137973] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:56Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=0 worker2 # [ 98.125715] nix-daemon[869]: reaped child process 1422, status = succeeded worker1 # [ 98.142155] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:56Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 98.163256] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 98.178176] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 98.191760] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.323Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'... worker2 # [ 98.200653] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 98.211084] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 98.308799] nix-daemon[869]: accepted connection from pid 1449, user nix-grpc-daemon (trusted) worker2 # [ 98.321684] nix-daemon[869]: reaped child process 1452, status = succeeded lb # [ 98.363977] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 98.339416] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.471Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 98.341097] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.473Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)" lb # [ 98.397360] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 98.367082] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.499Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.403227] niks3-server[1112]: 2026/09/23 04:42:56 INFO Signed narinfos id=39 count=1 lb # [ 98.413712] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls lb # [ 98.419260] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst lb # [ 98.430996] niks3-server[1112]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 98.438775] niks3-server[1112]: 2026/09/23 04:42:56 INFO Completed upload id=39 worker2 # [ 98.403649] nix-grpc-daemon[1395]: time=2026-09-23T04:42:56.535Z level=INFO msg="Upload complete. (180ms)" lb # [ 98.442325] niks3-server[1112]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo worker2 # [ 98.408405] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 98.414461] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 98.426109] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 98.433679] nix-daemon[869]: reaped child process 1447, status = succeeded worker2 # [ 98.438503] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 99.677422] nix-daemon[869]: reaped child process 1457, status = succeeded worker2 # [ 99.758919] nix-daemon[869]: accepted connection from pid 1473, user nix-grpc-daemon (trusted) worker2 # [ 99.771607] nix-daemon[869]: reaped child process 1476, status = succeeded lb # [ 99.814795] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 99.789246] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.921Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 99.791035] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.922Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)" lb # [ 99.857617] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst lb # [ 99.866331] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 99.874555] niks3-server[1112]: 2026/09/23 04:42:57 INFO Signed narinfos id=40 count=1 worker2 # [ 99.843573] nix-grpc-daemon[1395]: time=2026-09-23T04:42:57.975Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.883624] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv lb # [ 99.892243] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls lb # [ 99.899124] niks3-server[1112]: 2026/09/23 04:42:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 99.907963] niks3-server[1112]: 2026/09/23 04:42:57 INFO Completed upload id=40 worker2 # [ 99.873505] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.005Z level=INFO msg="Upload complete. (227ms)" lb # [ 99.910495] niks3-server[1112]: 2026/09/23 04:42:57 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo worker2 # [ 99.879793] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 worker1 # [ 99.905560] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:57Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=0 worker1 # [ 99.909634] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:57Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 99.896318] nix-daemon[869]: reaped child process 1455, status = succeeded worker2 # [ 99.930764] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 99.942927] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 99.956092] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.087Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'... worker2 # [ 99.965438] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 99.975346] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 100.073050] nix-daemon[869]: accepted connection from pid 1480, user nix-grpc-daemon (trusted) worker2 # [ 100.086289] nix-daemon[869]: reaped child process 1483, status = succeeded lb # [ 100.127826] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.110391] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.242Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 100.111880] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.243Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)" lb # [ 100.167935] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 100.176453] niks3-server[1112]: 2026/09/23 04:42:58 INFO Signed narinfos id=41 count=1 worker2 # [ 100.145495] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.277Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.189902] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst lb # [ 100.196147] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls lb # [ 100.206317] niks3-server[1112]: 2026/09/23 04:42:58 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 100.215441] niks3-server[1112]: 2026/09/23 04:42:58 INFO Completed upload id=41 worker2 # [ 100.180372] nix-grpc-daemon[1395]: time=2026-09-23T04:42:58.312Z level=INFO msg="Upload complete. (192ms)" lb # [ 100.219129] niks3-server[1112]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo worker2 # [ 100.185575] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 100.191296] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 100.203111] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 100.210934] nix-daemon[869]: reaped child process 1478, status = succeeded worker2 # [ 100.218834] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051... client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 101.054898] nix-daemon[869]: reaped child process 1487, status = succeeded worker2 # [ 101.125967] nix-daemon[869]: accepted connection from pid 1503, user nix-grpc-daemon (trusted) worker2 # [ 101.139814] nix-daemon[869]: reaped child process 1506, status = succeeded lb # [ 101.182389] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.161244] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.293Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 101.162934] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.294Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)" lb # [ 101.227503] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 101.233094] niks3-server[1112]: 2026/09/23 04:42:59 INFO Signed narinfos id=42 count=1 worker2 # [ 101.200661] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.332Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.245274] niks3-server[1112]: 2026/09/23 04:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 101.283507] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv lb # [ 101.286795] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls lb # [ 101.290990] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst lb # [ 101.298176] niks3-server[1112]: 2026/09/23 04:42:59 INFO Completed upload id=42 worker2 # [ 101.263222] nix-grpc-daemon[1395]: time=2026-09-23T04:42:59.395Z level=INFO msg="Upload complete. (237ms)" lb # [ 101.301647] niks3-server[1112]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo worker2 # [ 101.268917] nix-grpc-daemon[1334]: ts=2026-09-23T04:42:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 worker2 # [ 101.284399] nix-daemon[869]: reaped child process 1485, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.19 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 5.66 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1 # [ 101.598873] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 101.649259] nix-grpc-daemon[1166]: ts=2026-09-23T04:42:59Z level=info event=signal_exit lb # [ 101.885938] niks3-server[1112]: 2026/09/23 04:42:59 INFO lead: released remote=192.168.1.3:57386 worker2 # [ 102.853791] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 102.875567] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting worker1 # [ 102.878956] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 102.883257] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:39766 worker1 # [ 102.886631] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 102.890086] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:39766 worker1 # [ 102.894230] nix-grpc-daemon[1166]: ts=2026-09-23T04:43:00Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 102.904312] nix-daemon[846]: reaped child process 1227, status = succeeded worker1 # [ 102.989863] nix-grpc-daemon[1166]: E0000 00:00:1790138581.067863 1212 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true worker1 # [ 102.995446] nix-grpc-daemon[1166]: E0000 00:00:1790138581.073441 1212 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 103.054143] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.058415] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.259299] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.263315] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.464589] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.468576] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.669867] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.673855] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 103.875790] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 103.879590] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 104.079410] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 104.083421] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 104.087199] systemd[1]: nix-grpc-daemon.service: Consumed 214ms CPU time over 9.541s wall clock time, 14.1M memory peak, 10.4K incoming IP traffic, 15.1K outgoing IP traffic. worker1 # [ 104.097864] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 104.080798] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 104.102357] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 104.084797] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 104.105677] systemd[1]: nix-grpc-daemon.socket: Consumed 521us CPU time over 9.573s wall clock time, 256K memory peak, 21.9K incoming IP traffic, 16.4K outgoing IP traffic. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker2 # [ 104.285911] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.289864] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.490772] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.494947] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.696164] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.700135] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.902483] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.905835] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.106816] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.110789] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2 # [ 105.311910] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.315875] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.517192] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.521173] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.722203] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.726281] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.928213] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.931840] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.132882] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.137106] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 106.338141] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.342116] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.543385] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.547322] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.748442] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.752397] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.842351] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:04Z level=info event=scheduler_take_over lb # [ 106.877612] niks3-server[1112]: 2026/09/23 04:43:04 INFO lead: acquired remote=192.168.1.4:60490 worker2 # [ 106.954310] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.958196] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 107.010662] envoy[927]: conn peer=[2001:db8:1::1]:45064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9662 tx=4387 ms=5029 lb # [ 107.014407] envoy[927]: conn peer=[2001:db8:1::3]:48558 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=950 tx=293 ms=8383 worker2 # [ 107.159208] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.163342] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 107.364436] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.382143] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 107.386096] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:36288 worker2 # [ 107.389959] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker2 # [ 108.494783] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 108.501075] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 108.519679] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=0 worker2 # [ 108.524794] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 108.564152] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 108.576647] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'... worker2 # [ 108.589236] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 108.600783] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 108.700216] nix-daemon[869]: accepted connection from pid 1533, user nix-grpc-daemon (trusted) worker2 # [ 108.713655] nix-daemon[869]: reaped child process 1536, status = succeeded lb # [ 108.755635] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.729179] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.860Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 108.731171] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.862Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)" lb # [ 108.787647] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker2 # [ 108.762720] nix-grpc-daemon[1395]: time=2026-09-23T04:43:06.894Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.797678] niks3-server[1112]: 2026/09/23 04:43:06 INFO Signed narinfos id=43 count=1 lb # [ 108.809923] niks3-server[1112]: 2026/09/23 04:43:06 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 109.104429] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst lb # [ 109.111617] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls lb # [ 109.130742] niks3-server[1112]: 2026/09/23 04:43:07 INFO Completed upload id=43 worker2 # [ 109.096088] nix-grpc-daemon[1395]: time=2026-09-23T04:43:07.227Z level=INFO msg="Upload complete. (483ms)" worker2 # [ 109.101134] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=520 lb # [ 109.139155] niks3-server[1112]: 2026/09/23 04:43:07 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo worker2 # [ 109.109365] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 109.121642] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 109.129554] nix-daemon[869]: reaped child process 1531, status = succeeded client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051... client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker2 # [ 109.142825] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 110.765972] nix-daemon[869]: reaped child process 1540, status = succeeded worker2 # [ 110.841471] nix-daemon[869]: accepted connection from pid 1556, user nix-grpc-daemon (trusted) worker2 # [ 110.853852] nix-daemon[869]: reaped child process 1559, status = succeeded lb # [ 110.895488] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.910553] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.042Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 110.912374] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.044Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)" lb # [ 110.983248] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 110.991890] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 110.997802] niks3-server[1112]: 2026/09/23 04:43:08 INFO Signed narinfos id=44 count=1 worker2 # [ 110.966528] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.098Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.003767] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv lb # [ 111.010225] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls lb # [ 111.019279] niks3-server[1112]: 2026/09/23 04:43:08 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 111.029336] niks3-server[1112]: 2026/09/23 04:43:08 INFO Completed upload id=44 worker2 # [ 110.994700] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.126Z level=INFO msg="Upload complete. (238ms)" lb # [ 111.035571] niks3-server[1112]: 2026/09/23 04:43:08 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo worker2 # [ 111.001621] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 worker2 # [ 111.016559] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=0 worker2 # [ 111.021236] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 111.033162] nix-daemon[869]: reaped child process 1538, status = succeeded worker2 # [ 111.061263] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 111.074141] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 111.087918] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.219Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'... worker2 # [ 111.097126] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 111.105227] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 111.204246] nix-daemon[869]: accepted connection from pid 1565, user nix-grpc-daemon (trusted) worker2 # [ 111.217775] nix-daemon[869]: reaped child process 1568, status = succeeded lb # [ 111.259356] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.232436] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.364Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 111.234308] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.366Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)" lb # [ 111.288364] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker2 # [ 111.264195] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.396Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.298744] niks3-server[1112]: 2026/09/23 04:43:09 INFO Signed narinfos id=45 count=1 lb # [ 111.308439] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls lb # [ 111.313971] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst lb # [ 111.324901] niks3-server[1112]: 2026/09/23 04:43:09 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 111.330469] niks3-server[1112]: 2026/09/23 04:43:09 INFO Completed upload id=45 worker2 # [ 111.297398] nix-grpc-daemon[1395]: time=2026-09-23T04:43:09.429Z level=INFO msg="Upload complete. (178ms)" lb # [ 111.334029] niks3-server[1112]: 2026/09/23 04:43:09 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo worker2 # [ 111.301477] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 111.308352] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 111.317496] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 111.326469] nix-daemon[869]: reaped child process 1563, status = succeeded client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051... worker2 # [ 111.332370] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 112.569510] nix-daemon[869]: reaped child process 1572, status = succeeded worker2 # [ 112.652946] nix-daemon[869]: accepted connection from pid 1588, user nix-grpc-daemon (trusted) worker2 # [ 112.667477] nix-daemon[869]: reaped child process 1591, status = succeeded lb # [ 112.707473] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 112.681557] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.813Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 112.683098] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.815Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)" lb # [ 112.745185] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv lb # [ 112.759913] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker2 # [ 112.735809] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.867Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.770193] niks3-server[1112]: 2026/09/23 04:43:10 INFO Signed narinfos id=46 count=1 lb # [ 112.773229] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls lb # [ 112.778569] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst lb # [ 112.788924] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 112.791702] niks3-server[1112]: 2026/09/23 04:43:10 INFO Completed upload id=46 worker2 # [ 112.757087] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.888Z level=INFO msg="Upload complete. (217ms)" lb # [ 112.794095] niks3-server[1112]: 2026/09/23 04:43:10 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo worker2 # [ 112.765702] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 worker2 # [ 112.776276] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=0 worker2 # [ 112.780196] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 112.789910] nix-daemon[869]: reaped child process 1570, status = succeeded worker2 # [ 112.819904] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 112.832277] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 112.846288] nix-grpc-daemon[1395]: time=2026-09-23T04:43:10.978Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'... worker2 # [ 112.855299] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 112.865165] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 112.960590] nix-daemon[869]: accepted connection from pid 1597, user nix-grpc-daemon (trusted) worker2 # [ 112.975761] nix-daemon[869]: reaped child process 1600, status = succeeded lb # [ 113.016985] niks3-server[1112]: 2026/09/23 04:43:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 113.002338] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.134Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 113.003870] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.135Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)" lb # [ 113.063452] niks3-server[1112]: 2026/09/23 04:43:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 113.072995] niks3-server[1112]: 2026/09/23 04:43:11 INFO Signed narinfos id=47 count=1 worker2 # [ 113.041865] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.173Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.079092] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls lb # [ 113.082673] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst lb # [ 113.096245] niks3-server[1112]: 2026/09/23 04:43:11 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 113.104138] niks3-server[1112]: 2026/09/23 04:43:11 INFO Completed upload id=47 worker2 # [ 113.069366] nix-grpc-daemon[1395]: time=2026-09-23T04:43:11.201Z level=INFO msg="Upload complete. (193ms)" lb # [ 113.109526] niks3-server[1112]: 2026/09/23 04:43:11 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo worker2 # [ 113.074170] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 113.081111] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 113.093547] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 113.102696] nix-daemon[869]: reaped child process 1595, status = succeeded worker2 # [ 113.111072] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051... client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 113.955854] nix-daemon[869]: reaped child process 1604, status = succeeded worker2 # [ 114.030914] nix-daemon[869]: accepted connection from pid 1620, user nix-grpc-daemon (trusted) worker2 # [ 114.044893] nix-daemon[869]: reaped child process 1623, status = succeeded lb # [ 114.086521] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 114.060514] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.192Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 114.062309] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.194Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)" lb # [ 114.136305] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 114.141182] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls lb # [ 114.149180] niks3-server[1112]: 2026/09/23 04:43:12 INFO Signed narinfos id=48 count=1 worker2 # [ 114.117576] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.248Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.152848] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv lb # [ 114.158151] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst lb # [ 114.167714] niks3-server[1112]: 2026/09/23 04:43:12 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 114.173639] niks3-server[1112]: 2026/09/23 04:43:12 INFO Completed upload id=48 lb # [ 114.174970] niks3-server[1112]: 2026/09/23 04:43:12 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo worker2 # [ 114.139094] nix-grpc-daemon[1395]: time=2026-09-23T04:43:12.270Z level=INFO msg="Upload complete. (220ms)" worker2 # [ 114.147777] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 worker2 # [ 114.161590] nix-daemon[869]: reaped child process 1602, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.86 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds) worker1 # [ 114.399604] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 114.406597] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 114.497106] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 114.499212] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 114.502514] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 114.506798] nix-grpc-daemon[1253]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 114.508141] nix-grpc-daemon[1253]: E0000 00:00:1790138592.584743 1253 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 114.514275] nix-grpc-daemon[1253]: E0000 00:00:1790138592.590140 1253 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 114.516397] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 114.517927] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 114.525060] nix-grpc-daemon[1253]: E0000 00:00:1790138592.601583 1295 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 114.536747] nix-grpc-daemon[1253]: E0000 00:00:1790138592.614993 1295 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 114.540143] nix-grpc-daemon[1253]: E0000 00:00:1790138592.617178 1295 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 114.551045] nix-grpc-daemon[1253]: E0000 00:00:1790138592.629278 1295 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 114.560993] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 114.545077] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:36288 worker2 # [ 114.548807] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.30 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs client # [ 114.798794] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 4747fc9c4a364f099d01790495a7f9a7 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds) worker2 # [ 114.905505] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36798 client # [ 114.943613] nix[1365]: don't know how to build these paths: client # [ 114.944650] nix[1365]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker2 # [ 114.918882] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=0 worker2 # [ 114.923994] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 114.962796] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 114.975510] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36798 client # [ 115.018227] nix[1365]: copying 1 paths to 'lb:50051'... worker2 # [ 114.987723] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 client # [ 115.020367] nix[1365]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'... worker2 # [ 115.000069] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 115.097461] nix-daemon[869]: accepted connection from pid 1636, user nix-grpc-daemon (trusted) worker2 # [ 115.110355] nix-daemon[869]: reaped child process 1639, status = succeeded lb # [ 115.152785] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 115.126335] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 115.128294] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.260Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)" lb # [ 115.185494] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker2 # [ 115.159689] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.291Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.194423] niks3-server[1112]: 2026/09/23 04:43:13 INFO Signed narinfos id=49 count=1 lb # [ 115.198395] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst lb # [ 115.206090] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls lb # [ 115.223112] niks3-server[1112]: 2026/09/23 04:43:13 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 115.228180] niks3-server[1112]: 2026/09/23 04:43:13 INFO Completed upload id=49 worker2 # [ 115.193505] nix-grpc-daemon[1395]: time=2026-09-23T04:43:13.325Z level=INFO msg="Upload complete. (182ms)" lb # [ 115.230076] niks3-server[1112]: 2026/09/23 04:43:13 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo worker2 # [ 115.199428] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 115.206377] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 worker2 # [ 115.215262] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 115.225291] nix-daemon[869]: reaped child process 1634, status = succeeded worker2 # [ 115.229897] nix-daemon[869]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) client # [ 115.261279] nix[1365]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051... lb # [ 117.011386] envoy[927]: conn peer=[2001:db8:1::1]:52648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9590 tx=4414 ms=5704 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 117.459874] systemd-journald[283]: Received client request to rotate journal, rotating. worker1 # [ 117.479843] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/055d72bd56914de896f84ccd0e161ef1. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 117.368280] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds) client: must succeed: sleep 5.0 worker2 # [ 118.145392] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:16Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.03 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 122.653490] nix-daemon[869]: reaped child process 1643, status = succeeded worker2 # [ 122.740869] nix-daemon[869]: accepted connection from pid 1684, user nix-grpc-daemon (trusted) worker2 # [ 122.754760] nix-daemon[869]: reaped child process 1687, status = succeeded lb # [ 122.797742] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 122.774176] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 122.775720] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.907Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)" lb # [ 122.841680] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 122.847268] niks3-server[1112]: 2026/09/23 04:43:20 INFO Signed narinfos id=50 count=1 worker2 # [ 122.817040] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.948Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.856591] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv lb # [ 122.862371] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls lb # [ 122.866792] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 122.875981] niks3-server[1112]: 2026/09/23 04:43:20 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 122.879603] niks3-server[1112]: 2026/09/23 04:43:20 INFO Completed upload id=50 worker2 # [ 122.844933] nix-grpc-daemon[1395]: time=2026-09-23T04:43:20.976Z level=INFO msg="Upload complete. (201ms)" lb # [ 122.881922] niks3-server[1112]: 2026/09/23 04:43:20 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo worker2 # [ 122.850683] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36798 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 122.864880] nix-daemon[869]: reaped child process 1641, status = succeeded client # [ 122.905511] systemd[1]: rs.service: Deactivated successfully. client # [ 122.907071] systemd[1]: rs.service: Consumed 125ms CPU time over 8.108s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic. worker2 # [ 123.147610] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:21Z level=info event=signal_exit lb # [ 123.882877] niks3-server[1112]: 2026/09/23 04:43:21 INFO lead: released remote=192.168.1.4:60490 lb # [ 124.539718] niks3-server[1112]: 2026/09/23 04:43:22 INFO lead: acquired remote=192.168.1.3:49876 worker1 # [ 124.527805] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:22Z level=info event=scheduler_take_over worker1 # [ 124.867379] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 124.851713] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting worker2 # [ 124.857764] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36288 worker2 # [ 124.862264] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 124.866228] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 124.869793] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36288 worker2 # [ 124.873791] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:22Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 124.894273] nix-daemon[869]: reaped child process 1387, status = succeeded worker1 # [ 125.067965] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 125.072162] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.067552] nix-grpc-daemon[1334]: ts=2026-09-23T04:43:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 125.213489] nix-grpc-daemon[1334]: E0000 00:00:1790138603.345089 1380 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 13, started true, shutdown true worker2 # [ 125.220232] nix-grpc-daemon[1334]: E0000 00:00:1790138603.352194 1380 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 125.272842] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 125.280173] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53326 worker1 # [ 125.283832] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.18 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 126.650360] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 126.653720] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 126.657544] systemd[1]: nix-grpc-daemon.service: Consumed 1.778s CPU time over 36.618s wall clock time, 45.4M memory peak, 89.7K incoming IP traffic, 243.8K outgoing IP traffic. worker2 # [ 126.674180] systemd[1]: Starting scheduler-token.service... worker2 # [ 126.722389] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 126.724076] systemd[1]: Finished scheduler-token.service. worker2 # [ 126.725311] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 50ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 126.729936] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 126.818228] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 126.823070] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 126.827907] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 126.832133] nix-grpc-daemon[1697]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 126.835220] nix-grpc-daemon[1697]: E0000 00:00:1790138604.958089 1697 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 126.838481] nix-grpc-daemon[1697]: E0000 00:00:1790138604.959817 1697 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 126.841180] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 126.843367] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 126.846099] nix-grpc-daemon[1697]: E0000 00:00:1790138604.976671 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 126.856696] nix-grpc-daemon[1697]: E0000 00:00:1790138604.988704 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 126.861054] nix-grpc-daemon[1697]: E0000 00:00:1790138604.991047 1739 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 126.870915] nix-grpc-daemon[1697]: E0000 00:00:1790138605.002922 1739 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 126.880828] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 126.907806] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 126.912149] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53326 worker1 # [ 126.915979] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb # [ 127.012791] envoy[927]: conn peer=[2001:db8:1::1]:57328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2999 tx=1540 ms=7958 lb # [ 127.017550] envoy[927]: conn peer=[2001:db8:1::4]:44742 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36575 tx=2345 ms=34870 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.33 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid client # [ 127.129684] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 681202239daa43c4a761e48aa40e0803 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds) worker1 # [ 127.254255] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53342 worker1 # [ 127.266264] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) client # [ 127.291353] nix[1401]: don't know how to build these paths: client # [ 127.292302] nix[1401]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker1 # [ 127.287772] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker1 # [ 127.292648] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 127.336718] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53342 worker1 # [ 127.347604] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5) worker1 # [ 127.363195] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 2/5) worker1 # [ 127.372183] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 3/5) worker1 # [ 127.494756] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 632 ms (attempt 4/5) worker1 # [ 128.137556] nix-grpc-daemon[1253]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 128.141559] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 128.158955] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53342 client # [ 128.182200] nix[1401]: copying 1 paths to 'lb:50051'... client # [ 128.186301] nix[1401]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'... worker1 # [ 128.174403] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53342 worker1 # [ 128.182946] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) worker1 # [ 128.204402] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=info event=niks3_push_started pid=1382 worker1 # [ 128.317657] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.394Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 128.402899] nix-daemon[846]: accepted connection from pid 1399, user nix-grpc-daemon (trusted) worker1 # [ 128.415072] nix-daemon[846]: reaped child process 1402, status = succeeded lb # [ 128.438545] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 128.431658] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 128.433535] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.511Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)" lb # [ 128.475188] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 128.484629] niks3-server[1112]: 2026/09/23 04:43:26 INFO Signed narinfos id=51 count=1 worker1 # [ 128.473457] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.551Z level=INFO msg="Uploading 1 narinfos" lb # [ 128.491097] niks3-server[1112]: 2026/09/23 04:43:26 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst lb # [ 128.498296] niks3-server[1112]: 2026/09/23 04:43:26 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls lb # [ 128.508462] niks3-server[1112]: 2026/09/23 04:43:26 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker1 # [ 128.497138] nix-grpc-daemon[1382]: time=2026-09-23T04:43:26.575Z level=INFO msg="Upload complete. (177ms)" lb # [ 128.513505] niks3-server[1112]: 2026/09/23 04:43:26 INFO Completed upload id=51 worker1 # [ 128.500659] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53342 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 128.505332] nix-grpc-daemon[1253]: ts=2026-09-23T04:43:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53342 worker1 # [ 128.512878] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) worker1 # [ 128.517645] nix-daemon[846]: reaped child process 1380, status = succeeded worker1 # [ 128.523412] nix-daemon[846]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) client # [ 128.535562] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker1 # [ 130.748416] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1253 (nix-grpc-daemon) on client request. worker1 # [ 130.751316] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 130.774868] niks3-server[1112]: 2026/09/23 04:43:28 INFO lead: released remote=192.168.1.3:49876 worker1 # [ 130.764113] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 130.766179] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. client # [ 130.777233] nix[1401]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker1 # [ 130.767397] systemd[1]: nix-grpc-daemon.service: Consumed 382ms CPU time over 16.358s wall clock time, 35.6M memory peak, 15.4K incoming IP traffic, 27.4K outgoing IP traffic. worker2 # [ 130.745849] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" client # [ 130.781961] nix[1401]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker1 # [ 130.782071] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker1 # [ 130.783584] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 130.814349] nix-daemon[846]: reaped child process 1372, status = succeeded worker1 # [ 130.815666] nix-daemon[846]: reaped child process 1404, status = succeeded worker1 # [ 130.823090] nix-daemon[1406]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 130.829752] nix-daemon[846]: reaped child process 1406, status = failed with exit code 1 worker1 # [ 130.874795] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 130.876755] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 130.879877] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 130.885057] nix-grpc-daemon[1442]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 130.887189] nix-grpc-daemon[1442]: E0000 00:00:1790138608.962340 1442 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 130.893968] nix-grpc-daemon[1442]: E0000 00:00:1790138608.967888 1442 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 130.897845] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 130.901137] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 130.915710] niks3-server[1112]: 2026/09/23 04:43:28 INFO lead: acquired remote=192.168.1.3:38674 worker1 # [ 130.901987] nix-grpc-daemon[1442]: E0000 00:00:1790138608.978214 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 130.908123] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:28Z level=info event=scheduler_take_over worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.19 seconds) client: must succeed: sleep 5.0 worker1 # [ 130.915163] nix-grpc-daemon[1442]: E0000 00:00:1790138608.993411 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 130.919139] nix-grpc-daemon[1442]: E0000 00:00:1790138608.995642 1485 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 130.929228] nix-grpc-daemon[1442]: E0000 00:00:1790138609.007529 1485 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 130.938614] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 130.945187] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53348 worker1 # [ 130.946889] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 130.946918] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 130.970347] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 130.971673] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53348 worker1 # [ 130.973212] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1 # [ 131.278615] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker1 # [ 131.284092] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 131.326938] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 worker1 # [ 131.337271] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) worker1 # [ 131.350950] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) client # [ 131.376884] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... worker1 # [ 131.365561] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) client: (finished: must succeed: sleep 5.0, in 5.03 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 136.184943] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:34Z level=info event=niks3_push_started pid=1527 worker1 # [ 136.217344] nix-daemon[846]: reaped child process 1502, status = succeeded worker1 # [ 136.296751] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.374Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 136.381751] nix-daemon[846]: accepted connection from pid 1539, user nix-grpc-daemon (trusted) worker1 # [ 136.394464] nix-daemon[846]: reaped child process 1542, status = succeeded lb # [ 136.420369] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 136.412189] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.490Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 136.413706] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.492Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)" lb # [ 136.464265] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 136.474226] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 136.480849] niks3-server[1112]: 2026/09/23 04:43:34 INFO Signed narinfos id=52 count=1 worker1 # [ 136.468255] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.546Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.491859] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv lb # [ 136.501735] niks3-server[1112]: 2026/09/23 04:43:34 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 136.509623] niks3-server[1112]: 2026/09/23 04:43:34 INFO Completed upload id=52 worker1 # [ 136.493525] nix-grpc-daemon[1527]: time=2026-09-23T04:43:34.571Z level=INFO msg="Upload complete. (193ms)" worker1 # [ 136.500119] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 lb # [ 136.522439] niks3-server[1112]: 2026/09/23 04:43:34 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls worker1 # [ 136.514070] nix-daemon[846]: reaped child process 1500, status = succeeded client # [ 136.549623] systemd[1]: mid.service: Deactivated successfully. client # [ 136.554255] systemd[1]: mid.service: Consumed 142ms CPU time over 9.423s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic. lb # [ 137.012137] envoy[927]: conn peer=[2001:db8:1::3]:47004 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1036 tx=393 ms=16207 lb # [ 137.015843] envoy[927]: conn peer=[2001:db8:1::1]:46668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=1922 ms=9265 client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.16 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr client # [ 137.270133] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 26cfbde22445449b8102d8e307875673 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds) worker2 # [ 137.363202] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 137.374638] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 137.416377] nix[1437]: don't know how to build these paths: client # [ 137.417237] nix[1437]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker1 # [ 137.416947] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:35Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=0 worker1 # [ 137.421980] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:35Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 137.447623] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 137.459315] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5) worker2 # [ 137.482066] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 2/5) worker2 # [ 137.493832] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 3/5) worker2 # [ 137.666122] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 4/5) worker2 # [ 138.007770] nix-grpc-daemon[1697]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 138.011642] nix-grpc-daemon[1697]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 138.028670] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 client # [ 138.071171] nix[1437]: copying 1 paths to 'lb:50051'... worker2 # [ 138.040572] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # [ 138.073800] nix[1437]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'... worker2 # [ 138.050664] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 138.073491] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=info event=niks3_push_started pid=1790 worker2 # [ 138.175495] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.307Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 138.262140] nix-daemon[869]: accepted connection from pid 1803, user nix-grpc-daemon (trusted) worker2 # [ 138.273714] nix-daemon[869]: reaped child process 1806, status = succeeded lb # [ 138.319378] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.294418] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.426Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 138.295958] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.428Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)" lb # [ 138.360648] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 138.368160] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst worker2 # [ 138.340707] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.472Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.375694] niks3-server[1112]: 2026/09/23 04:43:36 INFO Signed narinfos id=53 count=1 lb # [ 138.387222] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls lb # [ 138.396606] niks3-server[1112]: 2026/09/23 04:43:36 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 138.403246] niks3-server[1112]: 2026/09/23 04:43:36 INFO Completed upload id=53 lb # [ 138.405068] niks3-server[1112]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo worker2 # [ 138.368252] nix-grpc-daemon[1790]: time=2026-09-23T04:43:36.500Z level=INFO msg="Upload complete. (189ms)" worker2 # [ 138.372902] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 138.379313] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 138.392577] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 138.403771] nix-daemon[869]: reaped child process 1788, status = succeeded worker2 # [ 138.408173] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 138.439813] nix[1437]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 140.692979] systemd[1]: intr.service: Sent signal SIGINT to main process 1437 (nix) on client request. worker2 # [ 140.664345] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:38Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv client # [ 140.698893] nix[1437]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds) client # [ 140.705558] nix[1437]: error: interrupted by the user client # [ 140.732126] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 140.734891] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 140.738400] systemd[1]: intr.service: Consumed 117ms CPU time over 3.464s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 140.832161] nix-daemon[869]: reaped child process 1811, status = succeeded worker2 # [ 140.833381] nix-daemon[1814]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 140.837883] nix-daemon[869]: reaped child process 1814, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.64 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw client # [ 141.936537] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: f194202b1b6b47beb3caf1134456e725 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds) client # [ 142.073164] nix[1458]: don't know how to build these paths: client # [ 142.074035] nix[1458]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker2 # [ 142.040382] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44890 worker1 # [ 142.070886] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=0 worker1 # [ 142.074948] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 142.114210] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 worker1 # [ 142.125093] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5) worker1 # [ 142.213636] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5) worker1 # [ 142.357468] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 3/5) worker1 # [ 142.471480] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 4/5) worker1 # [ 142.663740] nix-grpc-daemon[1442]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 142.667262] nix-grpc-daemon[1442]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 142.684552] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53380 client # [ 142.707168] nix[1458]: copying 1 paths to 'lb:50051'... client # [ 142.710812] nix[1458]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'... worker1 # [ 142.698433] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53380 worker1 # [ 142.709792] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) worker1 # [ 142.808464] nix-daemon[846]: accepted connection from pid 1583, user nix-grpc-daemon (trusted) worker1 # [ 142.820993] nix-daemon[846]: reaped child process 1586, status = succeeded lb # [ 142.845447] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 142.841323] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 142.843667] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.921Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)" lb # [ 142.887699] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst lb # [ 142.898680] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 142.908657] niks3-server[1112]: 2026/09/23 04:43:40 INFO Signed narinfos id=54 count=1 worker1 # [ 142.893571] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.970Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.919541] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls lb # [ 142.928395] niks3-server[1112]: 2026/09/23 04:43:40 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 142.931473] niks3-server[1112]: 2026/09/23 04:43:40 INFO Completed upload id=54 lb # [ 142.934914] niks3-server[1112]: 2026/09/23 04:43:40 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo worker1 # [ 142.917890] nix-grpc-daemon[1527]: time=2026-09-23T04:43:40.993Z level=INFO msg="Upload complete. (194ms)" worker1 # [ 142.922572] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 142.928281] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 worker1 # [ 142.938058] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) worker1 # [ 142.948830] nix-daemon[846]: reaped child process 1581, status = succeeded worker1 # [ 142.953125] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted) client # [ 142.965978] nix[1458]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051... worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 144.338318] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 144.366529] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2 worker2 # [ 144.534534] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44890 client # don't know how to build these paths: client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker1 # [ 144.565302] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53380 worker1 # [ 144.579907] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=0 worker1 # [ 144.581769] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 144.605071] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 144.617349] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 client # copying 1 paths to 'lb:50051'... worker2 # [ 144.629626] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'... worker2 # [ 144.650267] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 144.754942] nix-daemon[869]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker2 # [ 144.767742] nix-daemon[869]: reaped child process 1868, status = succeeded lb # [ 144.810529] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.783861] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 144.786296] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.917Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)" lb # [ 144.841555] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker2 # [ 144.817356] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.949Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.852392] niks3-server[1112]: 2026/09/23 04:43:42 INFO Signed narinfos id=55 count=1 lb # [ 144.857681] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls lb # [ 144.863119] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst lb # [ 144.876425] niks3-server[1112]: 2026/09/23 04:43:42 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 144.883522] niks3-server[1112]: 2026/09/23 04:43:42 INFO Completed upload id=55 lb # [ 144.885925] niks3-server[1112]: 2026/09/23 04:43:42 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo worker2 # [ 144.850612] nix-grpc-daemon[1790]: time=2026-09-23T04:43:42.982Z level=INFO msg="Upload complete. (188ms)" worker2 # [ 144.855667] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 144.862491] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 144.873109] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 144.896933] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:42Z level=info event=draining worker2 # [ 144.881630] nix-daemon[869]: reaped child process 1863, status = succeeded worker2 # [ 144.885709] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051... client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 146.321705] nix-daemon[869]: reaped child process 1872, status = succeeded worker2 # [ 146.406091] nix-daemon[869]: accepted connection from pid 1888, user nix-grpc-daemon (trusted) worker2 # [ 146.418542] nix-daemon[869]: reaped child process 1891, status = succeeded lb # [ 146.461527] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 146.435141] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.566Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 146.436677] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.568Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)" lb # [ 146.507128] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv lb # [ 146.513907] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker2 # [ 146.488305] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.620Z level=INFO msg="Uploading 1 narinfos" lb # [ 146.523122] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls lb # [ 146.525941] niks3-server[1112]: 2026/09/23 04:43:44 INFO Signed narinfos id=56 count=1 lb # [ 146.532942] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 146.541452] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 146.545943] niks3-server[1112]: 2026/09/23 04:43:44 INFO Completed upload id=56 lb # [ 146.547332] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo worker2 # [ 146.511240] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.643Z level=INFO msg="Upload complete. (215ms)" worker2 # [ 146.517722] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=1 worker1 # [ 146.540856] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:44Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=0 worker1 # [ 146.543207] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:44Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 146.533115] nix-daemon[869]: reaped child process 1870, status = succeeded worker2 # [ 146.571751] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 146.583751] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 146.597267] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.729Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 146.605920] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'... worker2 # [ 146.624163] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 146.725524] nix-daemon[869]: accepted connection from pid 1895, user nix-grpc-daemon (trusted) worker2 # [ 146.739803] nix-daemon[869]: reaped child process 1898, status = succeeded lb # [ 146.780491] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 146.753238] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.885Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 146.754701] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.886Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)" lb # [ 146.814842] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst lb # [ 146.818951] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 146.824139] niks3-server[1112]: 2026/09/23 04:43:44 INFO Signed narinfos id=57 count=1 worker2 # [ 146.792616] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.924Z level=INFO msg="Uploading 1 narinfos" lb # [ 146.836142] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls lb # [ 146.843428] niks3-server[1112]: 2026/09/23 04:43:44 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 146.854717] niks3-server[1112]: 2026/09/23 04:43:44 INFO Completed upload id=57 worker2 # [ 146.819933] nix-grpc-daemon[1790]: time=2026-09-23T04:43:44.951Z level=INFO msg="Upload complete. (182ms)" lb # [ 146.856420] niks3-server[1112]: 2026/09/23 04:43:44 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo worker2 # [ 146.824521] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 146.831400] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 146.841313] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 146.854525] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051... worker2 # [ 146.856696] nix-daemon[869]: reaped child process 1893, status = succeeded client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv lb # [ 147.013300] envoy[927]: conn peer=[2001:db8:1::1]:37824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=3327 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 147.670059] nix-daemon[869]: reaped child process 1902, status = succeeded worker2 # [ 147.745638] nix-daemon[869]: accepted connection from pid 1918, user nix-grpc-daemon (trusted) worker2 # [ 147.758411] nix-daemon[869]: reaped child process 1921, status = succeeded lb # [ 147.801333] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 147.775792] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.907Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 147.777341] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.909Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)" lb # [ 147.842986] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv lb # [ 147.848100] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker2 # [ 147.819361] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.951Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.854565] niks3-server[1112]: 2026/09/23 04:43:45 INFO Signed narinfos id=58 count=1 lb # [ 147.863443] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst lb # [ 147.872974] niks3-server[1112]: 2026/09/23 04:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 147.879242] niks3-server[1112]: 2026/09/23 04:43:45 INFO Completed upload id=58 worker2 # [ 147.843959] nix-grpc-daemon[1790]: time=2026-09-23T04:43:45.975Z level=INFO msg="Upload complete. (182ms)" lb # [ 147.884791] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo lb # [ 147.887902] niks3-server[1112]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls worker2 # [ 147.849681] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=1 worker1 # [ 147.875282] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:45Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=0 worker1 # [ 147.877225] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:45Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 147.865339] nix-daemon[869]: reaped child process 1900, status = succeeded worker2 # [ 147.900670] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 147.912934] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 147.925504] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.057Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 147.934436] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'... worker2 # [ 147.951802] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 148.052886] nix-daemon[869]: accepted connection from pid 1925, user nix-grpc-daemon (trusted) worker2 # [ 148.066864] nix-daemon[869]: reaped child process 1928, status = succeeded lb # [ 148.107603] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 148.081026] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.213Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 148.082518] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.214Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)" lb # [ 148.137908] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign worker2 # [ 148.108474] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.240Z level=INFO msg="Uploading 1 narinfos" lb # [ 148.143338] niks3-server[1112]: 2026/09/23 04:43:46 INFO Signed narinfos id=59 count=1 lb # [ 148.153500] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls lb # [ 148.159092] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst lb # [ 148.169714] niks3-server[1112]: 2026/09/23 04:43:46 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete worker2 # [ 148.139937] nix-grpc-daemon[1790]: time=2026-09-23T04:43:46.271Z level=INFO msg="Upload complete. (173ms)" lb # [ 148.174732] niks3-server[1112]: 2026/09/23 04:43:46 INFO Completed upload id=59 worker2 # [ 148.142510] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 148.181705] niks3-server[1112]: 2026/09/23 04:43:46 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo worker2 # [ 148.147629] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 148.156218] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 148.162456] nix-daemon[869]: reaped child process 1923, status = succeeded worker2 # [ 148.167612] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051... client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 149.221141] nix-daemon[869]: reaped child process 1932, status = succeeded worker2 # [ 149.279157] nix-daemon[869]: accepted connection from pid 1948, user nix-grpc-daemon (trusted) worker2 # [ 149.293045] nix-daemon[869]: reaped child process 1951, status = succeeded lb # [ 149.334374] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 149.308166] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.439Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 149.309756] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.441Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)" lb # [ 149.368261] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst lb # [ 149.380491] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv lb # [ 149.383463] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker2 # [ 149.353700] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.485Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.387986] niks3-server[1112]: 2026/09/23 04:43:47 INFO Signed narinfos id=60 count=1 lb # [ 149.397152] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls lb # [ 149.400584] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 149.403854] niks3-server[1112]: 2026/09/23 04:43:47 INFO Completed upload id=60 worker2 # [ 149.369150] nix-grpc-daemon[1790]: time=2026-09-23T04:43:47.501Z level=INFO msg="Upload complete. (194ms)" lb # [ 149.406418] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo worker2 # [ 149.374762] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=1 worker2 # [ 149.386937] nix-daemon[869]: reaped child process 1930, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.02 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.03 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 149.818259] nix-daemon[846]: reaped child process 1590, status = succeeded worker1 # [ 149.868669] nix-daemon[846]: accepted connection from pid 1638, user nix-grpc-daemon (trusted) worker1 # [ 149.882201] nix-daemon[846]: reaped child process 1641, status = succeeded lb # [ 149.902651] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.890945] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 149.892407] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.970Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)" lb # [ 149.923272] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 149.926485] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls lb # [ 149.927894] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 149.930417] niks3-server[1112]: 2026/09/23 04:43:47 INFO Signed narinfos id=61 count=1 worker1 # [ 149.915964] nix-grpc-daemon[1527]: time=2026-09-23T04:43:47.994Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.933907] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv lb # [ 149.939415] niks3-server[1112]: 2026/09/23 04:43:47 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 149.942334] niks3-server[1112]: 2026/09/23 04:43:47 INFO Completed upload id=61 worker1 # [ 149.926283] nix-grpc-daemon[1527]: time=2026-09-23T04:43:48.004Z level=INFO msg="Upload complete. (160ms)" lb # [ 149.945559] niks3-server[1112]: 2026/09/23 04:43:47 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo worker1 # [ 149.928847] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53380 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 149.936718] nix-daemon[846]: reaped child process 1588, status = succeeded client # [ 149.973766] systemd[1]: sw.service: Deactivated successfully. client # [ 149.978450] systemd[1]: sw.service: Consumed 130ms CPU time over 8.040s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds) lb: waiting until exactly no endpoints of x86_64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 150.899092] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:48Z level=info event=signal_exit lb # [ 150.920985] niks3-server[1112]: 2026/09/23 04:43:48 INFO lead: released remote=192.168.1.3:38674 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb # [ 151.875800] niks3-server[1112]: 2026/09/23 04:43:49 INFO lead: acquired remote=192.168.1.4:42496 worker2 # [ 151.842125] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:49Z level=info event=scheduler_take_over worker2 # [ 151.887310] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 151.907614] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=scheduler_restarting worker1 # [ 151.914212] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53348 worker1 # [ 151.918257] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 151.922312] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 151.925355] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53348 worker1 # [ 151.928891] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:49Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 151.945770] nix-daemon[846]: reaped child process 1498, status = succeeded worker2 # [ 152.088385] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.092613] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 152.122934] nix-grpc-daemon[1442]: ts=2026-09-23T04:43:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 152.182295] nix-grpc-daemon[1442]: E0000 00:00:1790138630.260198 1489 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 152.187150] nix-grpc-daemon[1442]: E0000 00:00:1790138630.260424 1489 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 152.293742] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.297738] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 152.498998] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.502907] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 152.703991] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.709291] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1 # [ 152.917451] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 152.922414] systemd[1]: nix-grpc-daemon.service: Consumed 586ms CPU time over 22.156s wall clock time, 36M memory peak, 25K incoming IP traffic, 32.6K outgoing IP traffic. worker2 # [ 152.911459] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.925139] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 152.929363] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53656 worker2 # [ 152.933356] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 153.913162] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 154.017344] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 154.019613] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 154.022792] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 154.027057] nix-grpc-daemon[1653]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 154.029158] nix-grpc-daemon[1653]: E0000 00:00:1790138632.105159 1653 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 154.034379] nix-grpc-daemon[1653]: E0000 00:00:1790138632.110570 1653 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 154.038105] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 154.039992] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 154.046065] nix-grpc-daemon[1653]: E0000 00:00:1790138632.123773 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 154.059440] nix-grpc-daemon[1653]: E0000 00:00:1790138632.137697 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 154.064052] nix-grpc-daemon[1653]: E0000 00:00:1790138632.139960 1695 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 154.073427] nix-grpc-daemon[1653]: E0000 00:00:1790138632.151683 1695 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 154.082309] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 154.065718] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53656 worker2 # [ 154.070191] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.21 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.11 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 155.143071] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 9afc2ac7392b4900bba2e7ee1025d0dd client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 155.267567] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 155.279067] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) client # [ 155.304359] nix[1514]: don't know how to build these paths: client # [ 155.305286] nix[1514]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv client # [ 155.306308] nix[1514]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv client # [ 155.307463] nix[1514]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv client # [ 155.308640] nix[1514]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker2 # [ 155.287428] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=0 worker2 # [ 155.291440] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 155.295808] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=0 worker2 # [ 155.299687] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 155.304150] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=0 worker2 # [ 155.307927] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=0 worker1 # [ 155.349348] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 155.330887] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker1 # [ 155.360238] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5) worker2 # [ 155.343220] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 client # [ 155.385171] nix[1514]: copying 1 paths to 'lb:50051'... worker2 # [ 155.354394] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # [ 155.387194] nix[1514]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'... worker2 # [ 155.366740] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 155.467926] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5) worker2 # [ 155.461937] nix-daemon[869]: accepted connection from pid 1966, user nix-grpc-daemon (trusted) worker2 # [ 155.474647] nix-daemon[869]: reaped child process 1969, status = succeeded lb # [ 155.517887] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 155.493190] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.624Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 155.494645] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.624Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)" lb # [ 155.547707] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 155.558721] niks3-server[1112]: 2026/09/23 04:43:53 INFO Signed narinfos id=62 count=1 worker2 # [ 155.526824] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.658Z level=INFO msg="Uploading 1 narinfos" lb # [ 155.576712] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst lb # [ 155.585683] niks3-server[1112]: 2026/09/23 04:43:53 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 155.587902] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls lb # [ 155.592254] niks3-server[1112]: 2026/09/23 04:43:53 INFO Completed upload id=62 worker2 # [ 155.557647] nix-grpc-daemon[1790]: time=2026-09-23T04:43:53.689Z level=INFO msg="Upload complete. (180ms)" lb # [ 155.594385] niks3-server[1112]: 2026/09/23 04:43:53 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo worker2 # [ 155.562578] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 155.568552] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 155.579512] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 155.589583] nix-daemon[869]: reaped child process 1964, status = succeeded worker2 # [ 155.593787] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # [ 155.625271] nix[1514]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051... worker1 # [ 155.663128] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 3/5) worker1 # [ 155.770256] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 4/5) worker1 # [ 156.024344] nix-grpc-daemon[1653]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 156.027364] nix-grpc-daemon[1653]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 156.045711] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 client # [ 156.071190] nix[1514]: copying 1 paths to 'lb:50051'... worker1 # [ 156.059290] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 client # [ 156.073557] nix[1514]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'... worker1 # [ 156.068784] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 156.092092] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=info event=niks3_push_started pid=1714 worker1 # [ 156.194146] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.272Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 156.281815] nix-daemon[846]: accepted connection from pid 1727, user nix-grpc-daemon (trusted) worker1 # [ 156.294307] nix-daemon[846]: reaped child process 1730, status = succeeded lb # [ 156.318150] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 156.312748] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.391Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 156.315326] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.392Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb # [ 156.358087] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 156.360758] niks3-server[1112]: 2026/09/23 04:43:54 INFO Signed narinfos id=63 count=1 worker1 # [ 156.346916] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.424Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.366274] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst lb # [ 156.374100] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls lb # [ 156.378483] niks3-server[1112]: 2026/09/23 04:43:54 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 156.384187] niks3-server[1112]: 2026/09/23 04:43:54 INFO Completed upload id=63 worker1 # [ 156.368381] nix-grpc-daemon[1714]: time=2026-09-23T04:43:54.446Z level=INFO msg="Upload complete. (170ms)" worker1 # [ 156.370527] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 156.387471] niks3-server[1112]: 2026/09/23 04:43:54 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo worker1 # [ 156.373832] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 156.380351] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 156.382904] nix-daemon[846]: reaped child process 1712, status = succeeded worker1 # [ 156.387524] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) client # [ 156.400861] nix[1514]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051... worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb # [ 157.013502] envoy[927]: conn peer=[2001:db8:1::1]:46424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4384 ms=4857 lb # [ 157.017423] envoy[927]: conn peer=[2001:db8:1::1]:37832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1557 ms=7883 lb # [ 157.020954] envoy[927]: conn peer=[2001:db8:1::3]:55412 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1018 tx=442 ms=21179 worker1 # [ 157.601305] nix-daemon[846]: reaped child process 1734, status = succeeded worker2 # [ 157.610951] nix-daemon[869]: reaped child process 1973, status = succeeded worker1 # [ 157.676354] nix-daemon[846]: accepted connection from pid 1759, user nix-grpc-daemon (trusted) worker1 # [ 157.688621] nix-daemon[846]: reaped child process 1762, status = succeeded lb # [ 157.713389] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 157.685304] nix-daemon[869]: accepted connection from pid 2001, user nix-grpc-daemon (trusted) worker1 # [ 157.706445] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.784Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 157.707873] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.786Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)" worker2 # [ 157.697265] nix-daemon[869]: reaped child process 2004, status = succeeded lb # [ 157.738769] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 157.714336] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.846Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 157.715971] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.847Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)" lb # [ 157.772329] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv lb # [ 157.784070] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 157.786661] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 157.789707] niks3-server[1112]: 2026/09/23 04:43:55 INFO Signed narinfos id=64 count=1 worker1 # [ 157.775480] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.853Z level=INFO msg="Uploading 1 narinfos" lb # [ 157.794728] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 157.797226] niks3-server[1112]: 2026/09/23 04:43:55 INFO Signed narinfos id=65 count=1 worker2 # [ 157.761931] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.893Z level=INFO msg="Uploading 1 narinfos" lb # [ 157.803177] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv lb # [ 157.806253] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls lb # [ 157.808828] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst lb # [ 157.814039] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls lb # [ 157.817682] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 157.821856] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 157.825836] niks3-server[1112]: 2026/09/23 04:43:55 INFO Completed upload id=65 worker2 # [ 157.791161] nix-grpc-daemon[1790]: time=2026-09-23T04:43:55.923Z level=INFO msg="Upload complete. (188ms)" lb # [ 157.827847] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo lb # [ 157.831817] niks3-server[1112]: 2026/09/23 04:43:55 INFO Completed upload id=64 lb # [ 157.833776] niks3-server[1112]: 2026/09/23 04:43:55 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo worker1 # [ 157.815959] nix-grpc-daemon[1714]: time=2026-09-23T04:43:55.894Z level=INFO msg="Upload complete. (222ms)" worker2 # [ 157.797346] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=1 worker2 # [ 157.804912] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 157.823312] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=1 worker2 # [ 157.809182] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 157.817212] nix-daemon[869]: reaped child process 1971, status = succeeded worker1 # [ 157.836795] nix-daemon[846]: reaped child process 1732, status = succeeded worker2 # [ 157.847681] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker1 # [ 157.866703] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 157.860233] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker1 # [ 157.878948] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 client # [ 157.902807] nix[1514]: copying 1 paths to 'lb:50051'... worker1 # [ 157.890106] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 client # [ 157.904725] nix[1514]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'... client # [ 157.909207] nix[1514]: copying 1 paths to 'lb:50051'... worker2 # [ 157.879128] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 client # [ 157.912275] nix[1514]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'... worker1 # [ 157.905212] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker2 # [ 157.888844] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 157.984068] nix-daemon[869]: accepted connection from pid 2008, user nix-grpc-daemon (trusted) worker1 # [ 158.004753] nix-daemon[846]: accepted connection from pid 1766, user nix-grpc-daemon (trusted) worker2 # [ 157.996398] nix-daemon[869]: reaped child process 2011, status = succeeded worker1 # [ 158.017457] nix-daemon[846]: reaped child process 1769, status = succeeded lb # [ 158.039343] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 158.043939] niks3-server[1112]: 2026/09/23 04:43:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 158.037672] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.115Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 158.019801] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 158.039268] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.117Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)" worker2 # [ 158.021328] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.153Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)" lb # [ 158.085622] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls lb # [ 158.087722] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign worker1 # [ 158.076395] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.154Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.092241] niks3-server[1112]: 2026/09/23 04:43:56 INFO Signed narinfos id=66 count=1 lb # [ 158.102856] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst lb # [ 158.108083] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 158.111903] niks3-server[1112]: 2026/09/23 04:43:56 INFO Completed upload id=66 worker1 # [ 158.096062] nix-grpc-daemon[1714]: time=2026-09-23T04:43:56.174Z level=INFO msg="Upload complete. (178ms)" lb # [ 158.114573] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst lb # [ 158.117176] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 158.100647] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 158.120707] niks3-server[1112]: 2026/09/23 04:43:56 INFO Signed narinfos id=67 count=1 worker2 # [ 158.086099] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.218Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.123749] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo worker1 # [ 158.106143] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 lb # [ 158.127820] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls lb # [ 158.131740] niks3-server[1112]: 2026/09/23 04:43:56 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete worker1 # [ 158.115580] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) lb # [ 158.134284] niks3-server[1112]: 2026/09/23 04:43:56 INFO Completed upload id=67 worker1 # [ 158.121213] nix-daemon[846]: reaped child process 1764, status = succeeded lb # [ 158.137282] niks3-server[1112]: 2026/09/23 04:43:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo worker2 # [ 158.100947] nix-grpc-daemon[1790]: time=2026-09-23T04:43:56.232Z level=INFO msg="Upload complete. (199ms)" worker1 # [ 158.125327] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) client # [ 158.137983] nix[1514]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051... worker2 # [ 158.105826] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 158.112273] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 158.125109] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 158.132858] nix-daemon[869]: reaped child process 2006, status = succeeded client # [ 158.172107] nix[1514]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051... worker2 # [ 158.141820] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 159.961385] nix-daemon[846]: reaped child process 1773, status = succeeded worker2 # [ 159.984115] nix-daemon[869]: reaped child process 2015, status = succeeded worker1 # [ 160.044605] nix-daemon[846]: accepted connection from pid 1798, user nix-grpc-daemon (trusted) worker1 # [ 160.057233] nix-daemon[846]: reaped child process 1801, status = succeeded lb # [ 160.079965] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.073116] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.074714] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.153Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)" worker2 # [ 160.058974] nix-daemon[869]: accepted connection from pid 2040, user nix-grpc-daemon (trusted) worker2 # [ 160.073092] nix-daemon[869]: reaped child process 2043, status = succeeded lb # [ 160.113579] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 160.086275] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 160.087764] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.219Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)" lb # [ 160.126540] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv lb # [ 160.131665] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst lb # [ 160.148885] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker2 # [ 160.117611] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.249Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.152355] niks3-server[1112]: 2026/09/23 04:43:58 INFO Signed narinfos id=69 count=1 lb # [ 160.155273] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 160.157206] niks3-server[1112]: 2026/09/23 04:43:58 INFO Signed narinfos id=68 count=1 lb # [ 160.159707] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker1 # [ 160.142926] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.220Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.161567] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls lb # [ 160.167248] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls lb # [ 160.173066] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 160.174760] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst lb # [ 160.178748] niks3-server[1112]: 2026/09/23 04:43:58 INFO Completed upload id=69 worker2 # [ 160.143552] nix-grpc-daemon[1790]: time=2026-09-23T04:43:58.275Z level=INFO msg="Upload complete. (191ms)" lb # [ 160.183077] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo lb # [ 160.185890] niks3-server[1112]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker2 # [ 160.148489] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=1 lb # [ 160.189686] niks3-server[1112]: 2026/09/23 04:43:58 INFO Completed upload id=68 worker1 # [ 160.173556] nix-grpc-daemon[1714]: time=2026-09-23T04:43:58.251Z level=INFO msg="Upload complete. (240ms)" lb # [ 160.192604] niks3-server[1112]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo worker2 # [ 160.158995] nix-daemon[869]: reaped child process 2013, status = succeeded worker1 # [ 160.179562] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=1 worker1 # [ 160.194635] nix-daemon[846]: reaped child process 1771, status = succeeded client # [ 160.224770] systemd[1]: burst.service: Deactivated successfully. client # [ 160.228831] systemd[1]: burst.service: Consumed 153ms CPU time over 5.084s wall clock time, 55.7M memory peak, 14K incoming IP traffic, 18.2K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.20 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 161.315291] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44890 worker1 # [ 161.344584] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 161.373158] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=0 worker2 # [ 161.377200] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 161.381691] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=0 worker2 # [ 161.386490] nix-grpc-daemon[1697]: ts=2026-09-23T04:43:59Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=0 worker1 # [ 161.450675] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 161.464573] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 161.476406] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 161.484961] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 161.590615] nix-daemon[846]: accepted connection from pid 1809, user nix-grpc-daemon (trusted) worker1 # [ 161.603183] nix-daemon[846]: reaped child process 1812, status = succeeded lb # [ 161.627450] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 161.620743] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.698Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 161.622784] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.700Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)" lb # [ 161.661392] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst lb # [ 161.665605] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 161.671756] niks3-server[1112]: 2026/09/23 04:43:59 INFO Signed narinfos id=70 count=1 worker1 # [ 161.659026] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.737Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.681404] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls lb # [ 161.692777] niks3-server[1112]: 2026/09/23 04:43:59 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 161.699453] niks3-server[1112]: 2026/09/23 04:43:59 INFO Completed upload id=70 lb # [ 161.702259] niks3-server[1112]: 2026/09/23 04:43:59 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo worker1 # [ 161.685147] nix-grpc-daemon[1714]: time=2026-09-23T04:43:59.763Z level=INFO msg="Upload complete. (186ms)" worker1 # [ 161.689918] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 161.695710] nix-grpc-daemon[1653]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 161.706570] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 161.715400] nix-daemon[846]: reaped child process 1807, status = succeeded worker1 # [ 161.720421] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 162.534422] nix-daemon[846]: reaped child process 1816, status = succeeded worker1 # [ 162.609200] nix-daemon[846]: accepted connection from pid 1832, user nix-grpc-daemon (trusted) worker1 # [ 162.621861] nix-daemon[846]: reaped child process 1835, status = succeeded lb # [ 162.645868] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 162.641271] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.719Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 162.642804] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.721Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)" lb # [ 162.688704] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 162.696890] niks3-server[1112]: 2026/09/23 04:44:00 INFO Signed narinfos id=71 count=1 worker1 # [ 162.683893] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.762Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.707506] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls lb # [ 162.712065] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv lb # [ 162.717750] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 162.727662] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 162.730885] niks3-server[1112]: 2026/09/23 04:44:00 INFO Completed upload id=71 worker1 # [ 162.714898] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.793Z level=INFO msg="Upload complete. (188ms)" lb # [ 162.733636] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo worker1 # [ 162.720899] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=1 worker2 # [ 162.703702] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:00Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 162.729953] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 162.713720] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:00Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=0 worker1 # [ 162.740139] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 162.748078] nix-daemon[846]: reaped child process 1814, status = succeeded worker1 # [ 162.749497] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 162.753987] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 162.836593] nix-daemon[846]: accepted connection from pid 1839, user nix-grpc-daemon (trusted) worker1 # [ 162.850420] nix-daemon[846]: reaped child process 1842, status = succeeded lb # [ 162.871875] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 162.865361] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 162.867372] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.945Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)" lb # [ 162.903313] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 162.891914] nix-grpc-daemon[1714]: time=2026-09-23T04:44:00.970Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.909177] niks3-server[1112]: 2026/09/23 04:44:00 INFO Signed narinfos id=72 count=1 lb # [ 162.919568] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst lb # [ 162.929677] niks3-server[1112]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 162.932753] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls lb # [ 162.942088] niks3-server[1112]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo lb # [ 162.945827] niks3-server[1112]: 2026/09/23 04:44:00 INFO Completed upload id=72 worker1 # [ 162.929976] nix-grpc-daemon[1714]: time=2026-09-23T04:44:01.008Z level=INFO msg="Upload complete. (171ms)" worker1 # [ 162.934320] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 162.940326] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 162.949231] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 162.963213] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 162.965415] nix-daemon[846]: reaped child process 1837, status = succeeded worker1 # [ 164.201585] nix-daemon[846]: reaped child process 1846, status = succeeded worker1 # [ 164.266522] nix-daemon[846]: accepted connection from pid 1862, user nix-grpc-daemon (trusted) worker1 # [ 164.278655] nix-daemon[846]: reaped child process 1865, status = succeeded lb # [ 164.303382] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.295782] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.373Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.297236] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.375Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)" lb # [ 164.348516] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls lb # [ 164.352672] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv lb # [ 164.358543] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 164.346955] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.425Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.362908] niks3-server[1112]: 2026/09/23 04:44:02 INFO Signed narinfos id=73 count=1 lb # [ 164.373065] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 164.383492] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 164.390209] niks3-server[1112]: 2026/09/23 04:44:02 INFO Completed upload id=73 worker1 # [ 164.374205] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.452Z level=INFO msg="Upload complete. (202ms)" lb # [ 164.392214] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo worker2 # [ 164.362696] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:02Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 164.380794] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=1 worker2 # [ 164.372446] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:02Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=0 worker1 # [ 164.395484] nix-daemon[846]: reaped child process 1844, status = succeeded worker1 # [ 164.432162] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 164.443932] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 164.455133] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 164.464334] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 164.562376] nix-daemon[846]: accepted connection from pid 1869, user nix-grpc-daemon (trusted) worker1 # [ 164.577205] nix-daemon[846]: reaped child process 1872, status = succeeded lb # [ 164.598660] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.590319] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.591780] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.670Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)" lb # [ 164.631418] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 164.638909] niks3-server[1112]: 2026/09/23 04:44:02 INFO Signed narinfos id=74 count=1 worker1 # [ 164.626154] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.704Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.643910] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls lb # [ 164.648590] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst lb # [ 164.663817] niks3-server[1112]: 2026/09/23 04:44:02 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete worker1 # [ 164.653417] nix-grpc-daemon[1714]: time=2026-09-23T04:44:02.731Z level=INFO msg="Upload complete. (177ms)" lb # [ 164.669409] niks3-server[1112]: 2026/09/23 04:44:02 INFO Completed upload id=74 worker1 # [ 164.656093] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=504 lb # [ 164.675360] niks3-server[1112]: 2026/09/23 04:44:02 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo worker1 # [ 164.661079] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 164.671246] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 164.680133] nix-daemon[846]: reaped child process 1867, status = succeeded worker1 # [ 164.683128] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 166.119921] nix-daemon[846]: reaped child process 1876, status = succeeded worker1 # [ 166.203089] nix-daemon[846]: accepted connection from pid 1892, user nix-grpc-daemon (trusted) worker1 # [ 166.215625] nix-daemon[846]: reaped child process 1895, status = succeeded lb # [ 166.237630] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.231805] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.309Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 166.234244] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.311Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)" lb # [ 166.283413] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker1 # [ 166.274610] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.352Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.290581] niks3-server[1112]: 2026/09/23 04:44:04 INFO Signed narinfos id=75 count=1 lb # [ 166.297969] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls lb # [ 166.302422] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 166.311063] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv lb # [ 166.319467] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 166.324118] niks3-server[1112]: 2026/09/23 04:44:04 INFO Completed upload id=75 worker1 # [ 166.307833] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.386Z level=INFO msg="Upload complete. (216ms)" lb # [ 166.328553] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo worker1 # [ 166.313580] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=1 worker2 # [ 166.297809] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:04Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 166.323478] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 166.307407] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:04Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=0 worker1 # [ 166.335540] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 166.341141] nix-daemon[846]: reaped child process 1874, status = succeeded worker1 # [ 166.346903] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.425Z level=INFO msg="All 1 paths already cached" worker1 # [ 166.352880] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 166.359431] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 166.446252] nix-daemon[846]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 166.459710] nix-daemon[846]: reaped child process 1902, status = succeeded lb # [ 166.481656] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.474061] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.552Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 166.475946] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.553Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)" lb # [ 166.514426] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 166.522686] niks3-server[1112]: 2026/09/23 04:44:04 INFO Signed narinfos id=76 count=1 worker1 # [ 166.510478] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.588Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.526846] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls lb # [ 166.534942] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst lb # [ 166.544258] niks3-server[1112]: 2026/09/23 04:44:04 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 166.551157] niks3-server[1112]: 2026/09/23 04:44:04 INFO Completed upload id=76 worker1 # [ 166.535350] nix-grpc-daemon[1714]: time=2026-09-23T04:44:04.613Z level=INFO msg="Upload complete. (168ms)" lb # [ 166.554336] niks3-server[1112]: 2026/09/23 04:44:04 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo worker1 # [ 166.540079] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 166.545433] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 166.554463] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 166.564591] nix-daemon[846]: reaped child process 1897, status = succeeded worker1 # [ 166.568210] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) lb # [ 167.013668] envoy[927]: conn peer=[2001:db8:1::1]:52430 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=5163 ms=4926 worker1 # [ 167.783200] nix-daemon[846]: reaped child process 1906, status = succeeded worker1 # [ 167.858087] nix-daemon[846]: accepted connection from pid 1922, user nix-grpc-daemon (trusted) worker1 # [ 167.869372] nix-daemon[846]: reaped child process 1925, status = succeeded lb # [ 167.893748] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 167.887850] nix-grpc-daemon[1714]: time=2026-09-23T04:44:05.965Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 167.889365] nix-grpc-daemon[1714]: time=2026-09-23T04:44:05.967Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)" lb # [ 167.934326] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 167.946982] niks3-server[1112]: 2026/09/23 04:44:05 INFO Signed narinfos id=77 count=1 worker1 # [ 167.934461] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.012Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.953701] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls lb # [ 167.960222] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst lb # [ 167.965478] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv lb # [ 167.975280] niks3-server[1112]: 2026/09/23 04:44:05 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 167.981050] niks3-server[1112]: 2026/09/23 04:44:05 INFO Completed upload id=77 lb # [ 167.983689] niks3-server[1112]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo worker1 # [ 167.966628] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.044Z level=INFO msg="Upload complete. (192ms)" worker1 # [ 167.972580] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=1 worker2 # [ 167.955616] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:06Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 167.965470] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:06Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=0 worker1 # [ 167.988313] nix-daemon[846]: reaped child process 1904, status = succeeded worker1 # [ 168.017402] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 168.030098] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 168.043131] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.121Z level=INFO msg="All 1 paths already cached" worker1 # [ 168.050870] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 168.058785] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 168.156894] nix-daemon[846]: accepted connection from pid 1929, user nix-grpc-daemon (trusted) worker1 # [ 168.169376] nix-daemon[846]: reaped child process 1932, status = succeeded lb # [ 168.193686] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.186212] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 168.187814] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.266Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)" lb # [ 168.227636] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 168.234907] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls lb # [ 168.239981] niks3-server[1112]: 2026/09/23 04:44:06 INFO Signed narinfos id=78 count=1 worker1 # [ 168.227871] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.306Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.247518] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst lb # [ 168.261248] niks3-server[1112]: 2026/09/23 04:44:06 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 168.268402] niks3-server[1112]: 2026/09/23 04:44:06 INFO Completed upload id=78 lb # [ 168.270376] niks3-server[1112]: 2026/09/23 04:44:06 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo worker1 # [ 168.252357] nix-grpc-daemon[1714]: time=2026-09-23T04:44:06.330Z level=INFO msg="Upload complete. (181ms)" worker1 # [ 168.256722] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 168.262758] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 168.273304] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 168.283498] nix-daemon[846]: reaped child process 1927, status = succeeded worker1 # [ 168.287476] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 169.113719] nix-daemon[846]: reaped child process 1936, status = succeeded worker1 # [ 169.185087] nix-daemon[846]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker1 # [ 169.196516] nix-daemon[846]: reaped child process 1955, status = succeeded lb # [ 169.220682] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.213636] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 169.215125] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.293Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)" lb # [ 169.267719] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv lb # [ 169.273115] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 169.278132] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst worker1 # [ 169.268034] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.346Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.283969] niks3-server[1112]: 2026/09/23 04:44:07 INFO Signed narinfos id=79 count=1 lb # [ 169.289821] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls lb # [ 169.298879] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 169.303289] niks3-server[1112]: 2026/09/23 04:44:07 INFO Completed upload id=79 worker1 # [ 169.287587] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.365Z level=INFO msg="Upload complete. (194ms)" lb # [ 169.304804] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo worker1 # [ 169.292636] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=1 worker2 # [ 169.275745] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:07Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 169.299360] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 169.285709] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:07Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=0 worker1 # [ 169.308555] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 169.316520] nix-daemon[846]: reaped child process 1934, status = succeeded worker1 # [ 169.318562] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.396Z level=INFO msg="All 1 paths already cached" worker1 # [ 169.322497] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 169.326148] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 169.412249] nix-daemon[846]: accepted connection from pid 1959, user nix-grpc-daemon (trusted) worker1 # [ 169.427361] nix-daemon[846]: reaped child process 1962, status = succeeded lb # [ 169.447750] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.440544] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.518Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 169.442502] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.520Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)" lb # [ 169.478385] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst lb # [ 169.485524] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign worker1 # [ 169.475629] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.553Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.491472] niks3-server[1112]: 2026/09/23 04:44:07 INFO Signed narinfos id=80 count=1 lb # [ 169.502847] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls lb # [ 169.512367] niks3-server[1112]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 169.517892] niks3-server[1112]: 2026/09/23 04:44:07 INFO Completed upload id=80 worker1 # [ 169.501518] nix-grpc-daemon[1714]: time=2026-09-23T04:44:07.579Z level=INFO msg="Upload complete. (169ms)" worker1 # [ 169.503676] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 169.521346] niks3-server[1112]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo worker1 # [ 169.508271] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 169.514143] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 169.516513] nix-daemon[846]: reaped child process 1957, status = succeeded worker1 # [ 169.521680] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 170.534985] nix-daemon[846]: reaped child process 1966, status = succeeded worker1 # [ 170.608892] nix-daemon[846]: accepted connection from pid 1982, user nix-grpc-daemon (trusted) worker1 # [ 170.621206] nix-daemon[846]: reaped child process 1985, status = succeeded lb # [ 170.646842] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.637352] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 170.639731] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.717Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)" lb # [ 170.686548] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv lb # [ 170.694801] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 170.701167] niks3-server[1112]: 2026/09/23 04:44:08 INFO Signed narinfos id=81 count=1 worker1 # [ 170.688095] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.766Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.706435] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls lb # [ 170.711706] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst lb # [ 170.723142] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 170.726377] niks3-server[1112]: 2026/09/23 04:44:08 INFO Completed upload id=81 worker1 # [ 170.710418] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.788Z level=INFO msg="Upload complete. (182ms)" lb # [ 170.728406] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo worker1 # [ 170.715894] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=1 worker2 # [ 170.698383] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:08Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 170.724296] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 170.708440] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:08Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=0 worker1 # [ 170.734363] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 170.743694] nix-daemon[846]: reaped child process 1964, status = succeeded worker1 # [ 170.745849] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.823Z level=INFO msg="All 2 paths already cached" worker1 # [ 170.747951] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 170.751724] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 170.833913] nix-daemon[846]: accepted connection from pid 1989, user nix-grpc-daemon (trusted) worker1 # [ 170.846866] nix-daemon[846]: reaped child process 1992, status = succeeded lb # [ 170.869455] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.861298] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.939Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 170.863373] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.941Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)" lb # [ 170.898565] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 170.905317] niks3-server[1112]: 2026/09/23 04:44:08 INFO Signed narinfos id=82 count=1 worker1 # [ 170.892761] nix-grpc-daemon[1714]: time=2026-09-23T04:44:08.970Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.914431] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst lb # [ 170.928143] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls lb # [ 170.930962] niks3-server[1112]: 2026/09/23 04:44:08 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 170.937971] niks3-server[1112]: 2026/09/23 04:44:08 INFO Completed upload id=82 worker1 # [ 170.921981] nix-grpc-daemon[1714]: time=2026-09-23T04:44:09.000Z level=INFO msg="Upload complete. (164ms)" lb # [ 170.941229] niks3-server[1112]: 2026/09/23 04:44:08 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo worker1 # [ 170.925425] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 170.930416] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 170.939108] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 170.946280] nix-daemon[846]: reaped child process 1987, status = succeeded worker1 # [ 170.950053] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 172.390574] nix-daemon[846]: reaped child process 1996, status = succeeded worker1 # [ 172.470496] nix-daemon[846]: accepted connection from pid 2012, user nix-grpc-daemon (trusted) worker1 # [ 172.483784] nix-daemon[846]: reaped child process 2015, status = succeeded lb # [ 172.506637] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.499182] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.577Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 172.500732] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.579Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)" lb # [ 172.546701] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst lb # [ 172.553250] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls lb # [ 172.562518] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign worker1 # [ 172.550678] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.628Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.566841] niks3-server[1112]: 2026/09/23 04:44:10 INFO Signed narinfos id=83 count=1 lb # [ 172.578477] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv lb # [ 172.586907] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 172.592136] niks3-server[1112]: 2026/09/23 04:44:10 INFO Completed upload id=83 worker1 # [ 172.575825] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.654Z level=INFO msg="Upload complete. (217ms)" lb # [ 172.595372] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo worker1 # [ 172.581752] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=1 worker2 # [ 172.564527] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:10Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.596972] nix-daemon[846]: reaped child process 1994, status = succeeded worker1 # [ 172.633734] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 172.646133] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 172.659250] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.737Z level=INFO msg="All 2 paths already cached" worker1 # [ 172.667184] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 172.675496] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 172.769465] nix-daemon[846]: accepted connection from pid 2019, user nix-grpc-daemon (trusted) worker1 # [ 172.781458] nix-daemon[846]: reaped child process 2022, status = succeeded lb # [ 172.805568] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.797206] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.875Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 172.798734] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.877Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)" lb # [ 172.838781] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign worker1 # [ 172.827696] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.906Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.845179] niks3-server[1112]: 2026/09/23 04:44:10 INFO Signed narinfos id=84 count=1 lb # [ 172.850629] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls lb # [ 172.855242] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst lb # [ 172.867754] niks3-server[1112]: 2026/09/23 04:44:10 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 172.875696] niks3-server[1112]: 2026/09/23 04:44:10 INFO Completed upload id=84 worker1 # [ 172.859538] nix-grpc-daemon[1714]: time=2026-09-23T04:44:10.937Z level=INFO msg="Upload complete. (171ms)" lb # [ 172.878273] niks3-server[1112]: 2026/09/23 04:44:10 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo worker1 # [ 172.863488] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 172.868238] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 172.877449] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 172.885386] nix-daemon[846]: reaped child process 2017, status = succeeded worker1 # [ 172.890190] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 174.325402] nix-daemon[846]: reaped child process 2026, status = succeeded worker1 # [ 174.401885] nix-daemon[846]: accepted connection from pid 2042, user nix-grpc-daemon (trusted) worker1 # [ 174.414439] nix-daemon[846]: reaped child process 2045, status = succeeded lb # [ 174.436442] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.428959] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.507Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 174.430446] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.508Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)" lb # [ 174.477283] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 174.483124] niks3-server[1112]: 2026/09/23 04:44:12 INFO Signed narinfos id=85 count=1 worker1 # [ 174.470773] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.548Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.494391] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv lb # [ 174.498248] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls lb # [ 174.502594] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst lb # [ 174.514265] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 174.517631] niks3-server[1112]: 2026/09/23 04:44:12 INFO Completed upload id=85 worker1 # [ 174.501897] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.580Z level=INFO msg="Upload complete. (203ms)" lb # [ 174.520516] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo worker1 # [ 174.507947] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=1 worker2 # [ 174.490811] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:12Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 174.523199] nix-daemon[846]: reaped child process 2024, status = succeeded worker1 # [ 174.552977] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 174.565478] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 174.578498] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.656Z level=INFO msg="All 2 paths already cached" worker1 # [ 174.586295] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 174.596155] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 174.688866] nix-daemon[846]: accepted connection from pid 2049, user nix-grpc-daemon (trusted) worker1 # [ 174.701196] nix-daemon[846]: reaped child process 2052, status = succeeded lb # [ 174.724988] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.717112] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.795Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 174.718672] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.796Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)" lb # [ 174.755225] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign lb # [ 174.760856] niks3-server[1112]: 2026/09/23 04:44:12 INFO Signed narinfos id=86 count=1 worker1 # [ 174.748043] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.826Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.769128] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst lb # [ 174.782154] niks3-server[1112]: 2026/09/23 04:44:12 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete worker1 # [ 174.773778] nix-grpc-daemon[1714]: time=2026-09-23T04:44:12.852Z level=INFO msg="Upload complete. (166ms)" lb # [ 174.790127] niks3-server[1112]: 2026/09/23 04:44:12 INFO Completed upload id=86 worker1 # [ 174.776080] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 174.794502] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls lb # [ 174.799849] niks3-server[1112]: 2026/09/23 04:44:12 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo worker1 # [ 174.784060] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 worker1 # [ 174.792914] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 174.795247] nix-daemon[846]: reaped child process 2047, status = succeeded worker1 # [ 174.800549] nix-daemon[846]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker1 # [ 176.230249] nix-daemon[846]: reaped child process 2056, status = succeeded worker1 # [ 176.300226] nix-daemon[846]: accepted connection from pid 2072, user nix-grpc-daemon (trusted) worker1 # [ 176.312025] nix-daemon[846]: reaped child process 2075, status = succeeded lb # [ 176.334886] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.328236] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.406Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.331036] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.408Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)" lb # [ 176.382328] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign lb # [ 176.385440] niks3-server[1112]: 2026/09/23 04:44:14 INFO Signed narinfos id=87 count=1 worker1 # [ 176.371721] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.449Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.391745] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls lb # [ 176.397451] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst lb # [ 176.403928] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv lb # [ 176.412981] niks3-server[1112]: 2026/09/23 04:44:14 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 176.416294] niks3-server[1112]: 2026/09/23 04:44:14 INFO Completed upload id=87 worker1 # [ 176.401611] nix-grpc-daemon[1714]: time=2026-09-23T04:44:14.479Z level=INFO msg="Upload complete. (192ms)" lb # [ 176.420094] niks3-server[1112]: 2026/09/23 04:44:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo worker1 # [ 176.406604] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35524 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=1 worker1 # [ 176.418928] nix-daemon[846]: reaped child process 2054, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.26 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 176.549849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: ebf89567922a4b9eb0a10950b2f27ab4 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 176.682834] nix[1584]: don't know how to build these paths: client # [ 176.683685] nix[1584]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv worker1 # [ 176.668742] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 176.673979] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35524 client # [ 176.707251] nix[1584]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker worker2 # [ 176.675730] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:14Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=0 lb # [ 177.014173] envoy[927]: conn peer=[2001:db8:1::1]:52446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27189 tx=12286 ms=15092 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code client # [ 206.708637] nix[1584]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip} client # [ 206.718388] nix[1584]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed client # [ 206.751119] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 206.754924] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 206.759243] systemd[1]: gpu.service: Consumed 153ms CPU time over 30.205s wall clock time, 37.6M memory peak, 5.5K incoming IP traffic, 5.2K outgoing IP traffic. lb # [ 207.015374] envoy[927]: conn peer=[2001:db8:1::1]:35228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=30046 client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.44 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.19 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 209.055431] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:47Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker1 # [ 209.566994] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35524 worker2 # [ 209.554842] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 209.575163] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=0 worker2 # [ 209.580296] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 209.620078] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 209.632795] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'... worker2 # [ 209.645386] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 209.656303] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 209.758937] nix-daemon[869]: accepted connection from pid 2083, user nix-grpc-daemon (trusted) worker2 # [ 209.773224] nix-daemon[869]: reaped child process 2086, status = succeeded lb # [ 209.814827] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 209.787949] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 209.790066] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.921Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)" lb # [ 209.857238] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign lb # [ 209.865089] niks3-server[1112]: 2026/09/23 04:44:47 INFO Signed narinfos id=88 count=1 worker2 # [ 209.833718] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.965Z level=INFO msg="Uploading 1 narinfos" lb # [ 209.874525] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls lb # [ 209.886067] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst lb # [ 209.891904] niks3-server[1112]: 2026/09/23 04:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 209.894869] niks3-server[1112]: 2026/09/23 04:44:47 INFO Completed upload id=88 lb # [ 209.897175] niks3-server[1112]: 2026/09/23 04:44:47 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo worker2 # [ 209.861612] nix-grpc-daemon[1790]: time=2026-09-23T04:44:47.993Z level=INFO msg="Upload complete. (193ms)" worker2 # [ 209.866609] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 209.872742] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 209.883503] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 209.893908] nix-daemon[869]: reaped child process 2081, status = succeeded client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051... worker2 # [ 209.901332] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 210.949395] nix-daemon[869]: reaped child process 2090, status = succeeded worker2 # [ 211.028895] nix-daemon[869]: accepted connection from pid 2106, user nix-grpc-daemon (trusted) worker2 # [ 211.042512] nix-daemon[869]: reaped child process 2109, status = succeeded lb # [ 211.084852] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 211.059213] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.191Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 211.060802] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.192Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)" lb # [ 211.126989] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv lb # [ 211.134992] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign lb # [ 211.137878] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls worker2 # [ 211.109660] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.241Z level=INFO msg="Uploading 1 narinfos" lb # [ 211.144344] niks3-server[1112]: 2026/09/23 04:44:49 INFO Signed narinfos id=89 count=1 lb # [ 211.149413] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 211.155066] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete lb # [ 211.158924] niks3-server[1112]: 2026/09/23 04:44:49 INFO Completed upload id=89 worker2 # [ 211.124308] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.256Z level=INFO msg="Upload complete. (215ms)" lb # [ 211.161931] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo worker2 # [ 211.132134] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=1 worker2 # [ 211.144358] nix-daemon[869]: reaped child process 2088, status = succeeded worker2 # [ 211.178243] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=0 worker2 # [ 211.182355] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 211.232224] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 211.245190] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 211.258499] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.390Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'... worker2 # [ 211.267475] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 211.276451] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 211.379332] nix-daemon[869]: accepted connection from pid 2115, user nix-grpc-daemon (trusted) worker2 # [ 211.392840] nix-daemon[869]: reaped child process 2118, status = succeeded lb # [ 211.434488] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 211.408880] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.540Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 211.410485] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.542Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)" lb # [ 211.468998] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign lb # [ 211.474760] niks3-server[1112]: 2026/09/23 04:44:49 INFO Signed narinfos id=90 count=1 worker2 # [ 211.446671] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.578Z level=INFO msg="Uploading 1 narinfos" lb # [ 211.483960] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst lb # [ 211.489299] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls lb # [ 211.498935] niks3-server[1112]: 2026/09/23 04:44:49 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 211.502611] niks3-server[1112]: 2026/09/23 04:44:49 INFO Completed upload id=90 worker2 # [ 211.467810] nix-grpc-daemon[1790]: time=2026-09-23T04:44:49.599Z level=INFO msg="Upload complete. (176ms)" lb # [ 211.504990] niks3-server[1112]: 2026/09/23 04:44:49 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo worker2 # [ 211.472160] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 211.477136] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 211.486811] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 211.492669] nix-daemon[869]: reaped child process 2113, status = succeeded worker2 # [ 211.497480] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051... client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 212.134477] nix-daemon[869]: reaped child process 2122, status = succeeded worker2 # [ 212.215431] nix-daemon[869]: accepted connection from pid 2138, user nix-grpc-daemon (trusted) worker2 # [ 212.228413] nix-daemon[869]: reaped child process 2141, status = succeeded lb # [ 212.271400] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 212.246610] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 212.248265] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.380Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)" lb # [ 212.321120] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv lb # [ 212.326432] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign lb # [ 212.332484] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls worker2 # [ 212.301094] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.432Z level=INFO msg="Uploading 1 narinfos" lb # [ 212.335897] niks3-server[1112]: 2026/09/23 04:44:50 INFO Signed narinfos id=91 count=1 lb # [ 212.341345] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst lb # [ 212.350497] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 212.353496] niks3-server[1112]: 2026/09/23 04:44:50 INFO Completed upload id=91 lb # [ 212.355714] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo worker2 # [ 212.320281] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.452Z level=INFO msg="Upload complete. (215ms)" worker2 # [ 212.325891] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=1 worker2 # [ 212.340124] nix-daemon[869]: reaped child process 2120, status = succeeded worker2 # [ 212.373672] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=0 worker2 # [ 212.380093] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 212.422816] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 212.435684] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 212.449545] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.581Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'... worker2 # [ 212.458670] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 212.469179] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 212.570592] nix-daemon[869]: accepted connection from pid 2147, user nix-grpc-daemon (trusted) worker2 # [ 212.585601] nix-daemon[869]: reaped child process 2150, status = succeeded lb # [ 212.625859] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 212.599912] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.731Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 212.601500] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.733Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)" lb # [ 212.661473] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign lb # [ 212.669662] niks3-server[1112]: 2026/09/23 04:44:50 INFO Signed narinfos id=92 count=1 worker2 # [ 212.638835] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.770Z level=INFO msg="Uploading 1 narinfos" lb # [ 212.677590] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls lb # [ 212.682014] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst lb # [ 212.696199] niks3-server[1112]: 2026/09/23 04:44:50 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 212.703041] niks3-server[1112]: 2026/09/23 04:44:50 INFO Completed upload id=92 worker2 # [ 212.668330] nix-grpc-daemon[1790]: time=2026-09-23T04:44:50.800Z level=INFO msg="Upload complete. (186ms)" lb # [ 212.705417] niks3-server[1112]: 2026/09/23 04:44:50 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo worker2 # [ 212.672477] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 212.677384] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 worker2 # [ 212.687208] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker2 # [ 212.696239] nix-daemon[869]: reaped child process 2145, status = succeeded worker2 # [ 212.700925] nix-daemon[869]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051... client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 213.537825] nix-daemon[869]: reaped child process 2154, status = succeeded worker2 # [ 213.622525] nix-daemon[869]: accepted connection from pid 2170, user nix-grpc-daemon (trusted) worker2 # [ 213.635376] nix-daemon[869]: reaped child process 2173, status = succeeded lb # [ 213.678799] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 213.652889] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 213.654472] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.786Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)" lb # [ 213.721203] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 213.694792] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.826Z level=INFO msg="Uploading 1 narinfos" lb # [ 213.730350] niks3-server[1112]: 2026/09/23 04:44:51 INFO Signed narinfos id=93 count=1 lb # [ 213.735595] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv lb # [ 213.740203] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls lb # [ 213.744917] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst lb # [ 213.755696] niks3-server[1112]: 2026/09/23 04:44:51 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 213.760169] niks3-server[1112]: 2026/09/23 04:44:51 INFO Completed upload id=93 lb # [ 213.763319] niks3-server[1112]: 2026/09/23 04:44:51 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo worker2 # [ 213.726906] nix-grpc-daemon[1790]: time=2026-09-23T04:44:51.858Z level=INFO msg="Upload complete. (218ms)" worker2 # [ 213.735119] nix-grpc-daemon[1697]: ts=2026-09-23T04:44:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44890 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=1 worker2 # [ 213.748802] nix-daemon[869]: reaped child process 2152, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.36 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 215.057383] nix-grpc-daemon[1653]: ts=2026-09-23T04:44:53Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.06 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 7.78 seconds) (finished: run the VM test script, in 216.05 seconds) test script finished in 216.22s cleanup kill QemuMachine (pid 50) client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T04:44:53Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 49) lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T04:44:54Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.10 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh