tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.01 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client: QEMU running (pid 49) worker2: QEMU running (pid 52) worker2 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... worker2 # Formatting '/build/vm-state-worker2/tmp.cMiNlg0aK1', fmt=raw size=1073741824 client # Disk image does not exist, creating the virtualisation disk image... worker1: QEMU running (pid 51) lb: QEMU running (pid 50) worker1 # Disk image does not exist, creating the virtualisation disk image... lb # Formatting '/build/vm-state-lb/tmp.ayn2XUb5BH', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.bdTzhpTQ9m', fmt=raw size=1073741824 worker2 # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.21 seconds) client # mke2fs 1.47.4 (6-Mar-2025) worker2 # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service client # Discarding device blocks: 0/262144 done worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client: waiting for the VM to finish booting client # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Filesystem UUID: 0df8a1e4-2b77-4449-8a89-10da3c9f98ba lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # Formatting '/build/vm-state-worker1/tmp.5zp5DwjaEV', fmt=raw size=1073741824 client # Filesystem UUID: b0304305-2254-42bb-91b1-71de4a72a3ed worker2 # Superblock backups stored on blocks: lb # Discarding device blocks: 0/262144 done worker1 # mke2fs 1.47.4 (6-Mar-2025) client # Superblock backups stored on blocks: worker2 # 32768, 98304, 163840, 229376 lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Discarding device blocks: 0/262144 done client # 32768, 98304, 163840, 229376 worker2 # lb # Filesystem UUID: f865ffbd-45df-49ba-a98a-fa2b73b6401c worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes client # worker2 # Allocating group tables: 0/8 done lb # Superblock backups stored on blocks: worker1 # Filesystem UUID: b6ead55f-799f-489d-8de3-4e78c873c878 client # Allocating group tables: 0/8 done worker2 # Writing inode tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker1 # Superblock backups stored on blocks: client # Writing inode tables: 0/8 done worker2 # Creating journal (8192 blocks): done lb # worker1 # 32768, 98304, 163840, 229376 client # Creating journal (8192 blocks): done worker2 # Writing superblocks and filesystem accounting information: 0/8 done lb # Allocating group tables: 0/8 done worker1 # client # Writing superblocks and filesystem accounting information: 0/8 done worker2 # lb # Writing inode tables: 0/8 done worker1 # Allocating group tables: 0/8 done client # worker2 # Virtualisation disk image created. lb # Creating journal (8192 blocks): done worker1 # Writing inode tables: 0/8 done client # Virtualisation disk image created. worker2 # Starting virtiofs daemons... lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Creating journal (8192 blocks): done client # Starting virtiofs daemons... worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # worker1 # Writing superblocks and filesystem accounting information: 0/8 done client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Virtualisation disk image created. worker1 # client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Starting virtiofs daemons... worker1 # Virtualisation disk image created. client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # Starting virtiofs daemons... client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) client # client # client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 client # Press Ctrl-B to configure iPXE (PCI 00:02.0)... client # client # client # client # client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # worker2 # worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker1 # worker1 # client # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 client # client # worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker2 # worker2 # worker2 # worker2 # lb # lb # worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker1 # worker1 # worker1 # worker1 # lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)... lb # lb # lb # lb # worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker2 # worker2 # lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # worker1 # lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)... lb # lb # client # Booting from ROM... worker2 # Booting from ROM... worker1 # Booting from ROM... lb # Booting from ROM... client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test/init regInfo=/nix/store/99yzymliyji70vhz8x5j5c4ckg24296s-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] NX (Execute Disable) protection: active client # [ 0.000000] APIC: Static calls initialized client # [ 0.000000] SMBIOS 2.8 present. worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 client # [ 0.000000] DMI: Memory slots populated: 1/1 lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5f2pyghh7bdk9s7schxljxgh1845dmij-nixos-system-lb-test/init regInfo=/nix/store/b17iww818kslhkx3acpxq2j13rz8h7sy-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7ivsh1xpnbmbn73qmia5fjss23hffswz-nixos-system-worker1-test/init regInfo=/nix/store/0m6yj9v4lz5rh7gnbwj9ql3hx52rgzbd-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.000000] BIOS-provided physical RAM map: worker1 # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] kvm-clock: using sched offset of 486801697 cycles worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xis419qg6ssac1mxl23xpjki6jlxkhwd-nixos-system-worker2-test/init regInfo=/nix/store/5fllks9l0kij5jdjhq2g604s52nb8f4c-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved worker2 # [ 0.000000] BIOS-provided physical RAM map: lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable client # [ 0.000005] tsc: Detected 2400.012 MHz processor worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.002731] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.002742] Using GB pages for direct mapping lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved client # [ 0.002884] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved client # [ 0.002893] ACPI: Early table checksum verification disabled worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.000000] NX (Execute Disable) protection: active worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.002896] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) lb # [ 0.000000] APIC: Static calls initialized worker1 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.000000] SMBIOS 2.8 present. worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker1 # [ 0.000000] APIC: Static calls initialized worker1 # [ 0.000000] SMBIOS 2.8 present. client # [ 0.002900] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker2 # [ 0.000000] NX (Execute Disable) protection: active client # [ 0.002903] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 lb # [ 0.000000] DMI: Memory slots populated: 1/1 worker2 # [ 0.000000] APIC: Static calls initialized lb # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] DMI: Memory slots populated: 1/1 worker2 # [ 0.000000] SMBIOS 2.8 present. worker1 # [ 0.000000] Hypervisor detected: KVM client # [ 0.002910] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002912] ACPI: FACS 0x000000005FFE0000 000040 worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker2 # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.000001] kvm-clock: using sched offset of 494704706 cycles worker2 # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] kvm-clock: using sched offset of 487491623 cycles worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002916] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000005] tsc: Detected 2400.012 MHz processor client # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor worker2 # [ 0.000000] kvm-clock: using sched offset of 490031191 cycles lb # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002919] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs client # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker1 # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker2 # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT lb # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.002719] found SMP MP-table at [mem 0x000f5450-0x000f545f] lb # [ 0.002744] Using GB pages for direct mapping client # [ 0.002922] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker2 # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker1 # [ 0.002731] Using GB pages for direct mapping lb # [ 0.002862] RAMDISK: [mem 0x5e36e000-0x5ffcffff] client # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker2 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # [ 0.002877] RAMDISK: [mem 0x5e370000-0x5ffcffff] lb # [ 0.002870] ACPI: Early table checksum verification disabled worker2 # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f] client # [ 0.002923] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker1 # [ 0.002892] ACPI: Early table checksum verification disabled lb # [ 0.002873] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker2 # [ 0.002735] Using GB pages for direct mapping worker1 # [ 0.002895] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) client # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker2 # [ 0.002851] RAMDISK: [mem 0x5e36f000-0x5ffcffff] lb # [ 0.002877] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002899] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker2 # [ 0.002858] ACPI: Early table checksum verification disabled lb # [ 0.002881] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.003149] No NUMA configuration found worker2 # [ 0.002861] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker1 # [ 0.002903] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.003150] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] lb # [ 0.002888] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002865] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002910] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.003152] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.002891] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.005406] Zone ranges: worker1 # [ 0.002912] ACPI: FACS 0x000000005FFE0000 000040 worker2 # [ 0.002869] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005407] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.002892] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002875] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005409] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.005410] Normal empty lb # [ 0.002894] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002877] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.005411] Device empty worker1 # [ 0.002915] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002895] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005412] Movable zone start for each node worker2 # [ 0.002879] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005413] Early memory node ranges worker1 # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002897] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005413] node 0: [mem 0x0000000000001000-0x000000000009efff] worker2 # [ 0.002881] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002918] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005414] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] lb # [ 0.002898] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker2 # [ 0.002882] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] lb # [ 0.002899] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] client # [ 0.005415] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker2 # [ 0.002884] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005434] On node 0, zone DMA: 1 pages in unavailable ranges lb # [ 0.002900] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.002921] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker2 # [ 0.002885] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.005701] On node 0, zone DMA: 97 pages in unavailable ranges lb # [ 0.002900] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.031973] On node 0, zone DMA32: 40 pages in unavailable ranges worker1 # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker2 # [ 0.002886] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] lb # [ 0.002900] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.032429] ACPI: PM-Timer IO Port: 0x608 worker1 # [ 0.002922] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker2 # [ 0.002887] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] client # [ 0.032441] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) lb # [ 0.002901] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker1 # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker2 # [ 0.002887] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.032470] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.002901] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] lb # [ 0.003126] No NUMA configuration found worker1 # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker2 # [ 0.002888] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.032473] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.003150] No NUMA configuration found worker2 # [ 0.002888] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] lb # [ 0.003127] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] client # [ 0.032474] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.003151] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] lb # [ 0.003129] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker2 # [ 0.002888] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.032476] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker1 # [ 0.003153] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.005386] Zone ranges: worker2 # [ 0.003112] No NUMA configuration found worker1 # [ 0.005413] Zone ranges: lb # [ 0.005387] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.032477] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.003113] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker1 # [ 0.005414] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.005388] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.032477] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) lb # [ 0.005389] Normal empty worker1 # [ 0.005415] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] worker2 # [ 0.003115] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker2 # [ 0.005403] Zone ranges: lb # [ 0.005390] Device empty worker1 # [ 0.005416] Normal empty client # [ 0.032480] ACPI: Using ACPI (MADT) for SMP configuration information worker1 # [ 0.005417] Device empty lb # [ 0.005391] Movable zone start for each node worker2 # [ 0.005403] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.032481] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker1 # [ 0.005417] Movable zone start for each node lb # [ 0.005392] Early memory node ranges client # [ 0.032485] TSC deadline timer available worker1 # [ 0.005418] Early memory node ranges worker2 # [ 0.005405] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] lb # [ 0.005392] node 0: [mem 0x0000000000001000-0x000000000009efff] client # [ 0.032489] CPU topo: Max. logical packages: 1 worker2 # [ 0.005406] Normal empty worker1 # [ 0.005418] node 0: [mem 0x0000000000001000-0x000000000009efff] worker2 # [ 0.005407] Device empty lb # [ 0.005393] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] client # [ 0.032489] CPU topo: Max. logical dies: 1 worker2 # [ 0.005407] Movable zone start for each node worker1 # [ 0.005420] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] client # [ 0.032490] CPU topo: Max. dies per package: 1 worker2 # [ 0.005408] Early memory node ranges lb # [ 0.005394] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] client # [ 0.032493] CPU topo: Max. threads per core: 1 worker1 # [ 0.005421] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker2 # [ 0.005408] node 0: [mem 0x0000000000001000-0x000000000009efff] lb # [ 0.005416] On node 0, zone DMA: 1 pages in unavailable ranges client # [ 0.032493] CPU topo: Num. cores per package: 1 worker1 # [ 0.005440] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.005409] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] client # [ 0.032494] CPU topo: Num. threads per package: 1 lb # [ 0.005676] On node 0, zone DMA: 97 pages in unavailable ranges worker1 # [ 0.005700] On node 0, zone DMA: 97 pages in unavailable ranges worker2 # [ 0.005410] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] lb # [ 0.032031] On node 0, zone DMA32: 40 pages in unavailable ranges client # [ 0.032494] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker1 # [ 0.031916] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.032496] ACPI: PM-Timer IO Port: 0x608 worker2 # [ 0.005429] On node 0, zone DMA: 1 pages in unavailable ranges client # [ 0.032511] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker1 # [ 0.032376] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.032508] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.005692] On node 0, zone DMA: 97 pages in unavailable ranges worker1 # [ 0.032388] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) client # [ 0.032541] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] lb # [ 0.032538] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 worker2 # [ 0.032846] On node 0, zone DMA32: 40 pages in unavailable ranges worker1 # [ 0.032418] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 worker2 # [ 0.033310] ACPI: PM-Timer IO Port: 0x608 client # [ 0.032542] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.032540] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.032421] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker2 # [ 0.033322] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) client # [ 0.032544] [mem 0x60000000-0xafffffff] available for PCI devices lb # [ 0.032542] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.032422] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) client # [ 0.032545] Booting paravirtualized kernel on KVM worker2 # [ 0.033351] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.032543] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker1 # [ 0.032423] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker2 # [ 0.033353] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) lb # [ 0.032544] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) client # [ 0.032548] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.032424] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker2 # [ 0.033355] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) lb # [ 0.032545] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) client # [ 0.037048] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.032425] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker2 # [ 0.033356] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) client # [ 0.039184] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 lb # [ 0.032547] ACPI: Using ACPI (MADT) for SMP configuration information worker1 # [ 0.032428] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.033357] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) lb # [ 0.032548] ACPI: HPET id: 0x8086a201 base: 0xfed00000 client # [ 0.039229] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.032429] ACPI: HPET id: 0x8086a201 base: 0xfed00000 lb # [ 0.032551] TSC deadline timer available worker2 # [ 0.033358] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker1 # [ 0.032432] TSC deadline timer available lb # [ 0.032555] CPU topo: Max. logical packages: 1 worker2 # [ 0.033360] ACPI: Using ACPI (MADT) for SMP configuration information worker1 # [ 0.032436] CPU topo: Max. logical packages: 1 lb # [ 0.032555] CPU topo: Max. logical dies: 1 worker1 # [ 0.032437] CPU topo: Max. logical dies: 1 worker2 # [ 0.033361] ACPI: HPET id: 0x8086a201 base: 0xfed00000 lb # [ 0.032556] CPU topo: Max. dies per package: 1 worker2 # [ 0.033364] TSC deadline timer available worker1 # [ 0.032437] CPU topo: Max. dies per package: 1 lb # [ 0.032559] CPU topo: Max. threads per core: 1 worker1 # [ 0.032441] CPU topo: Max. threads per core: 1 worker2 # [ 0.033368] CPU topo: Max. logical packages: 1 lb # [ 0.032560] CPU topo: Num. cores per package: 1 worker1 # [ 0.032441] CPU topo: Num. cores per package: 1 worker2 # [ 0.033369] CPU topo: Max. logical dies: 1 lb # [ 0.032560] CPU topo: Num. threads per package: 1 client # [ 0.039231] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test/init regInfo=/nix/store/99yzymliyji70vhz8x5j5c4ckg24296s-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.033369] CPU topo: Max. dies per package: 1 worker1 # [ 0.032442] CPU topo: Num. threads per package: 1 lb # [ 0.032560] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.033373] CPU topo: Max. threads per core: 1 worker1 # [ 0.032442] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.033373] CPU topo: Num. cores per package: 1 lb # [ 0.032579] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.039325] Unknown kernel command line parameters "regInfo=/nix/store/99yzymliyji70vhz8x5j5c4ckg24296s-closure-info/registration", will be passed to user space. worker1 # [ 0.032462] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker2 # [ 0.033373] CPU topo: Num. threads per package: 1 client # [ 0.039337] random: crng init done lb # [ 0.032610] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker2 # [ 0.033374] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker1 # [ 0.032493] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] client # [ 0.039338] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes lb # [ 0.032611] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker2 # [ 0.033391] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker1 # [ 0.032495] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] client # [ 0.041429] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.032613] [mem 0x60000000-0xafffffff] available for PCI devices worker2 # [ 0.033420] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.032496] [mem 0x60000000-0xafffffff] available for PCI devices lb # [ 0.032614] Booting paravirtualized kernel on KVM client # [ 0.042486] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.032497] Booting paravirtualized kernel on KVM client # [ 0.042518] Fallback order for Node 0: 0 worker2 # [ 0.033422] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] lb # [ 0.032617] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker2 # [ 0.033423] [mem 0x60000000-0xafffffff] available for PCI devices worker1 # [ 0.032500] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns client # [ 0.042521] Built 1 zonelists, mobility grouping on. Total pages: 393078 client # [ 0.042522] Policy zone: DMA32 worker2 # [ 0.033425] Booting paravirtualized kernel on KVM lb # [ 0.037077] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.036975] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 client # [ 0.045043] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.037202] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker2 # [ 0.033428] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.039313] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 client # [ 0.048149] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.037248] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.039359] kvm-guest: PV spinlocks disabled, single CPU client # [ 0.051629] allocated 3145728 bytes of page_ext worker2 # [ 0.037896] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 client # [ 0.061666] ftrace: allocating 48787 entries in 192 pages worker2 # [ 0.040390] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 client # [ 0.061668] ftrace: allocated 192 pages with 2 groups worker2 # [ 0.040434] kvm-guest: PV spinlocks disabled, single CPU client # [ 0.062487] Dynamic Preempt: lazy client # [ 0.062604] rcu: Preemptible hierarchical RCU implementation. client # [ 0.062605] rcu: RCU event tracing is enabled. lb # [ 0.037250] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5f2pyghh7bdk9s7schxljxgh1845dmij-nixos-system-lb-test/init regInfo=/nix/store/b17iww818kslhkx3acpxq2j13rz8h7sy-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.039361] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7ivsh1xpnbmbn73qmia5fjss23hffswz-nixos-system-worker1-test/init regInfo=/nix/store/0m6yj9v4lz5rh7gnbwj9ql3hx52rgzbd-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.062606] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.062607] Trampoline variant of Tasks RCU enabled. client # [ 0.062607] Rude variant of Tasks RCU enabled. lb # [ 0.037346] Unknown kernel command line parameters "regInfo=/nix/store/b17iww818kslhkx3acpxq2j13rz8h7sy-closure-info/registration", will be passed to user space. worker1 # [ 0.039455] Unknown kernel command line parameters "regInfo=/nix/store/0m6yj9v4lz5rh7gnbwj9ql3hx52rgzbd-closure-info/registration", will be passed to user space. worker2 # [ 0.040435] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xis419qg6ssac1mxl23xpjki6jlxkhwd-nixos-system-worker2-test/init regInfo=/nix/store/5fllks9l0kij5jdjhq2g604s52nb8f4c-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.037358] random: crng init done client # [ 0.062608] Tracing variant of Tasks RCU enabled. worker1 # [ 0.039467] random: crng init done client # [ 0.062608] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.037359] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker1 # [ 0.039468] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker2 # [ 0.040526] Unknown kernel command line parameters "regInfo=/nix/store/5fllks9l0kij5jdjhq2g604s52nb8f4c-closure-info/registration", will be passed to user space. client # [ 0.062609] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.040538] random: crng init done lb # [ 0.037682] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.041562] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.040539] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes client # [ 0.062673] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.037707] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.042606] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.037731] Fallback order for Node 0: 0 worker1 # [ 0.042636] Fallback order for Node 0: 0 worker2 # [ 0.042692] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.062674] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.037734] Built 1 zonelists, mobility grouping on. Total pages: 393078 worker1 # [ 0.042639] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.037735] Policy zone: DMA32 worker2 # [ 0.043777] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.042640] Policy zone: DMA32 client # [ 0.062675] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.043808] Fallback order for Node 0: 0 lb # [ 0.037839] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.045113] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.067045] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker2 # [ 0.043811] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.040870] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.043812] Policy zone: DMA32 worker1 # [ 0.048271] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.067334] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.044179] allocated 3145728 bytes of page_ext worker1 # [ 0.051769] allocated 3145728 bytes of page_ext worker2 # [ 0.046324] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.054038] ftrace: allocating 48787 entries in 192 pages worker1 # [ 0.061687] ftrace: allocating 48787 entries in 192 pages client # [ 0.067340] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.049446] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.054040] ftrace: allocated 192 pages with 2 groups worker1 # [ 0.061689] ftrace: allocated 192 pages with 2 groups lb # [ 0.054913] Dynamic Preempt: lazy worker2 # [ 0.053086] allocated 3145728 bytes of page_ext worker1 # [ 0.062511] Dynamic Preempt: lazy client # [ 0.067443] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.055104] rcu: Preemptible hierarchical RCU implementation. worker2 # [ 0.062878] ftrace: allocating 48787 entries in 192 pages worker1 # [ 0.062627] rcu: Preemptible hierarchical RCU implementation. client # [ 0.070952] Console: colour VGA+ 80x25 lb # [ 0.055105] rcu: RCU event tracing is enabled. worker2 # [ 0.062880] ftrace: allocated 192 pages with 2 groups worker1 # [ 0.062628] rcu: RCU event tracing is enabled. client # [ 0.070955] printk: legacy console [tty0] enabled worker2 # [ 0.063716] Dynamic Preempt: lazy lb # [ 0.055105] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.062629] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.100361] printk: legacy console [ttyS0] enabled worker2 # [ 0.063885] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.055107] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.062630] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.063885] rcu: RCU event tracing is enabled. client # [ 0.196947] ACPI: Core revision 20250807 lb # [ 0.055107] Rude variant of Tasks RCU enabled. worker1 # [ 0.062631] Rude variant of Tasks RCU enabled. lb # [ 0.055107] Tracing variant of Tasks RCU enabled. worker2 # [ 0.063886] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.062631] Tracing variant of Tasks RCU enabled. worker2 # [ 0.063888] Trampoline variant of Tasks RCU enabled. lb # [ 0.055108] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. client # [ 0.197856] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker1 # [ 0.062632] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker2 # [ 0.063888] Rude variant of Tasks RCU enabled. worker2 # [ 0.063888] Tracing variant of Tasks RCU enabled. lb # [ 0.055109] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.062633] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.199378] APIC: Switch to symmetric I/O mode setup client # [ 0.200355] x2apic enabled worker2 # [ 0.063889] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.055119] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.062683] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.063890] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.201132] APIC: Switched APIC routing to: physical x2apic lb # [ 0.055120] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.062685] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.063906] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.055121] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.062685] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.063907] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.059758] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.067040] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 client # [ 0.202934] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 lb # [ 0.060064] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.067312] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.063908] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.203906] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.068278] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 lb # [ 0.060072] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.067320] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.068564] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.205544] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) lb # [ 0.060181] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.067422] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.063714] Console: colour VGA+ 80x25 worker2 # [ 0.068571] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.070985] Console: colour VGA+ 80x25 client # [ 0.206628] x86/cpu: User Mode Instruction Prevention (UMIP) activated lb # [ 0.063717] printk: legacy console [tty0] enabled worker1 # [ 0.070987] printk: legacy console [tty0] enabled lb # [ 0.093225] printk: legacy console [ttyS0] enabled worker2 # [ 0.068674] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.100398] printk: legacy console [ttyS0] enabled worker2 # [ 0.072175] Console: colour VGA+ 80x25 client # [ 0.207694] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 lb # [ 0.189095] ACPI: Core revision 20250807 worker1 # [ 0.195974] ACPI: Core revision 20250807 worker2 # [ 0.072177] printk: legacy console [tty0] enabled client # [ 0.208541] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker2 # [ 0.101682] printk: legacy console [ttyS0] enabled worker2 # [ 0.196780] ACPI: Core revision 20250807 worker1 # [ 0.196854] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns lb # [ 0.189987] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns client # [ 0.209544] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto lb # [ 0.191553] APIC: Switch to symmetric I/O mode setup worker1 # [ 0.198324] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.197655] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker1 # [ 0.199292] x2apic enabled lb # [ 0.192534] x2apic enabled client # [ 0.211540] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.199156] APIC: Switch to symmetric I/O mode setup lb # [ 0.193310] APIC: Switched APIC routing to: physical x2apic worker1 # [ 0.200026] APIC: Switched APIC routing to: physical x2apic client # [ 0.212540] Transient Scheduler Attacks: Vulnerable: No microcode worker2 # [ 0.200119] x2apic enabled client # [ 0.213540] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker2 # [ 0.200856] APIC: Switched APIC routing to: physical x2apic lb # [ 0.195102] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker1 # [ 0.201831] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.214541] Speculative Return Stack Overflow: Mitigation: Safe RET worker2 # [ 0.202658] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 lb # [ 0.196075] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker1 # [ 0.202788] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker1 # [ 0.204497] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) lb # [ 0.197774] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker2 # [ 0.203617] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.198864] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.205586] x86/cpu: User Mode Instruction Prevention (UMIP) activated client # [ 0.215540] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker2 # [ 0.205295] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) client # [ 0.217548] Spectre V2 : Enabling IBPB for BPF worker1 # [ 0.206651] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 lb # [ 0.199914] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker2 # [ 0.206379] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.207493] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 lb # [ 0.201447] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 client # [ 0.218541] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.207462] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 client # [ 0.219541] active return thunk: srso_alias_return_thunk worker2 # [ 0.208292] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker1 # [ 0.208497] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto lb # [ 0.201775] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto client # [ 0.220562] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker1 # [ 0.210493] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl lb # [ 0.203771] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.209296] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto client # [ 0.221540] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.211493] Transient Scheduler Attacks: Vulnerable: No microcode lb # [ 0.204771] Transient Scheduler Attacks: Vulnerable: No microcode client # [ 0.222540] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker2 # [ 0.211292] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker1 # [ 0.212493] Spectre V2 : Mitigation: Enhanced / Automatic IBRS lb # [ 0.205771] Spectre V2 : Mitigation: Enhanced / Automatic IBRS client # [ 0.223540] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker2 # [ 0.212292] Transient Scheduler Attacks: Vulnerable: No microcode worker1 # [ 0.213493] Speculative Return Stack Overflow: Mitigation: Safe RET lb # [ 0.206771] Speculative Return Stack Overflow: Mitigation: Safe RET client # [ 0.224540] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker2 # [ 0.213292] Spectre V2 : Mitigation: Enhanced / Automatic IBRS client # [ 0.225540] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.214292] Speculative Return Stack Overflow: Mitigation: Safe RET client # [ 0.226540] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' client # [ 0.227541] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker1 # [ 0.214493] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization lb # [ 0.207771] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker1 # [ 0.216500] Spectre V2 : Enabling IBPB for BPF client # [ 0.229249] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 lb # [ 0.209778] Spectre V2 : Enabling IBPB for BPF worker2 # [ 0.215292] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization client # [ 0.230252] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.217207] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.217300] Spectre V2 : Enabling IBPB for BPF lb # [ 0.210771] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier client # [ 0.230541] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker1 # [ 0.218493] active return thunk: srso_alias_return_thunk lb # [ 0.211771] active return thunk: srso_alias_return_thunk client # [ 0.231540] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.218293] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker1 # [ 0.219515] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' lb # [ 0.212793] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.219293] active return thunk: srso_alias_return_thunk worker1 # [ 0.220493] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' client # [ 0.232541] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. lb # [ 0.213770] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker2 # [ 0.220313] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker1 # [ 0.221493] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' lb # [ 0.215502] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker2 # [ 0.221292] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.222493] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' lb # [ 0.216524] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker2 # [ 0.222292] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker1 # [ 0.223493] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' lb # [ 0.217527] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker2 # [ 0.223292] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker1 # [ 0.224493] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' lb # [ 0.218517] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.224292] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker1 # [ 0.225493] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.225292] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' lb # [ 0.219554] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker1 # [ 0.226494] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 lb # [ 0.220771] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker2 # [ 0.226292] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker1 # [ 0.227493] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 lb # [ 0.221771] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.227293] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker1 # [ 0.228493] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 lb # [ 0.222770] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker2 # [ 0.228292] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker1 # [ 0.229493] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.223770] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker2 # [ 0.229292] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.230493] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.224771] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.230292] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker1 # [ 0.231493] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.231292] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.225720] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.232292] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. client # [ 0.266474] Freeing SMP alternatives memory: 44K client # [ 0.267107] pid_max: default: 32768 minimum: 301 client # [ 0.267642] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.268644] landlock: Up and running. client # [ 0.269541] Yama: becoming mindful. client # [ 0.270738] LSM support for eBPF active client # [ 0.271482] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.272577] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.275011] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.276133] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker1 # [ 0.265748] Freeing SMP alternatives memory: 44K client # [ 0.276545] ... version: 2 lb # [ 0.259760] Freeing SMP alternatives memory: 44K worker1 # [ 0.266496] pid_max: default: 32768 minimum: 301 client # [ 0.277254] ... bit width: 48 lb # [ 0.260554] pid_max: default: 32768 minimum: 301 worker1 # [ 0.267317] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.277543] ... generic counters: 6 worker2 # [ 0.267103] Freeing SMP alternatives memory: 44K worker1 # [ 0.268281] landlock: Up and running. lb # [ 0.260892] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.267836] pid_max: default: 32768 minimum: 301 client # [ 0.278215] ... generic bitmap: 000000000000003f worker1 # [ 0.268907] Yama: becoming mindful. lb # [ 0.261868] landlock: Up and running. client # [ 0.278554] ... fixed-purpose counters: 0 worker1 # [ 0.269706] LSM support for eBPF active worker2 # [ 0.268388] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.262771] Yama: becoming mindful. client # [ 0.279265] ... fixed-purpose bitmap: 0000000000000000 worker2 # [ 0.269386] landlock: Up and running. lb # [ 0.263984] LSM support for eBPF active worker1 # [ 0.270550] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.270293] Yama: becoming mindful. client # [ 0.279543] ... value mask: 0000ffffffffffff worker2 # [ 0.271072] LSM support for eBPF active lb # [ 0.264787] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.280415] ... max period: 00007fffffffffff worker1 # [ 0.271533] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.281171] ... global_ctrl mask: 000000000000003f worker2 # [ 0.271847] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.265810] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.281654] signal: max sigframe size: 3376 worker2 # [ 0.273179] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.282442] rcu: Hierarchical SRCU implementation. worker1 # [ 0.273985] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.283143] rcu: Max phase no-delay instances is 400. lb # [ 0.268353] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) worker1 # [ 0.275046] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker2 # [ 0.275432] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) worker1 # [ 0.275502] ... version: 2 lb # [ 0.269296] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker1 # [ 0.276198] ... bit width: 48 lb # [ 0.269775] ... version: 2 worker1 # [ 0.276496] ... generic counters: 6 worker2 # [ 0.276877] Performance Events: Fam17h+ core perfctr, AMD PMU driver. lb # [ 0.270493] ... bit width: 48 worker2 # [ 0.277301] ... version: 2 worker1 # [ 0.277172] ... generic bitmap: 000000000000003f lb # [ 0.270774] ... generic counters: 6 worker2 # [ 0.277965] ... bit width: 48 worker1 # [ 0.277496] ... fixed-purpose counters: 0 lb # [ 0.271467] ... generic bitmap: 000000000000003f worker2 # [ 0.278295] ... generic counters: 6 client # [ 0.288101] smp: Bringing up secondary CPUs ... worker1 # [ 0.278187] ... fixed-purpose bitmap: 0000000000000000 lb # [ 0.271820] ... fixed-purpose counters: 0 client # [ 0.288557] smp: Brought up 1 node, 1 CPU worker2 # [ 0.279004] ... generic bitmap: 000000000000003f worker1 # [ 0.278496] ... value mask: 0000ffffffffffff lb # [ 0.272555] ... fixed-purpose bitmap: 0000000000000000 worker2 # [ 0.279295] ... fixed-purpose counters: 0 client # [ 0.289219] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker1 # [ 0.279335] ... max period: 00007fffffffffff worker2 # [ 0.279954] ... fixed-purpose bitmap: 0000000000000000 lb # [ 0.272774] ... value mask: 0000ffffffffffff worker1 # [ 0.280115] ... global_ctrl mask: 000000000000003f worker2 # [ 0.280295] ... value mask: 0000ffffffffffff lb # [ 0.273693] ... max period: 00007fffffffffff worker1 # [ 0.280592] signal: max sigframe size: 3376 client # [ 0.289726] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109304K reserved, 0K cma-reserved) worker2 # [ 0.281159] ... max period: 00007fffffffffff lb # [ 0.274477] ... global_ctrl mask: 000000000000003f worker1 # [ 0.281369] rcu: Hierarchical SRCU implementation. client # [ 0.290849] devtmpfs: initialized worker2 # [ 0.281957] ... global_ctrl mask: 000000000000003f lb # [ 0.274885] signal: max sigframe size: 3376 worker1 # [ 0.282092] rcu: Max phase no-delay instances is 400. worker2 # [ 0.282403] signal: max sigframe size: 3376 client # [ 0.291629] x86/mm: Memory block size: 128MB lb # [ 0.275638] rcu: Hierarchical SRCU implementation. worker2 # [ 0.283229] rcu: Hierarchical SRCU implementation. lb # [ 0.276341] rcu: Max phase no-delay instances is 400. worker2 # [ 0.283845] rcu: Max phase no-delay instances is 400. client # [ 0.293386] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.294463] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.295615] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.287002] smp: Bringing up secondary CPUs ... worker1 # [ 0.287509] smp: Brought up 1 node, 1 CPU client # [ 0.296710] PM: RTC time: 06:58:58, date: 2026-09-23 lb # [ 0.281288] smp: Bringing up secondary CPUs ... worker1 # [ 0.288211] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) lb # [ 0.281787] smp: Brought up 1 node, 1 CPU worker2 # [ 0.288843] smp: Bringing up secondary CPUs ... worker2 # [ 0.289309] smp: Brought up 1 node, 1 CPU lb # [ 0.282453] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.289944] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker1 # [ 0.288678] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker1 # [ 0.289761] devtmpfs: initialized client # [ 0.300453] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.282933] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved) worker1 # [ 0.290558] x86/mm: Memory block size: 128MB worker2 # [ 0.290502] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved) lb # [ 0.284009] devtmpfs: initialized client # [ 0.301723] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.291528] devtmpfs: initialized lb # [ 0.284800] x86/mm: Memory block size: 128MB client # [ 0.302567] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.292306] x86/mm: Memory block size: 128MB worker1 # [ 0.292331] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.303816] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.286610] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.293449] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.304553] audit: initializing netlink subsys (disabled) worker2 # [ 0.294057] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.294586] pinctrl core: initialized pinctrl subsystem lb # [ 0.287655] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.305717] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.295659] PM: RTC time: 06:58:58, date: 2026-09-23 worker2 # [ 0.295167] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.288857] pinctrl core: initialized pinctrl subsystem client # [ 0.305720] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.296380] pinctrl core: initialized pinctrl subsystem lb # [ 0.289987] PM: RTC time: 06:58:58, date: 2026-09-23 client # [ 0.306544] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.297515] PM: RTC time: 06:58:58, date: 2026-09-23 client # [ 0.307509] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.299306] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.308290] audit: type=2000 audit(1790146739.086:1): state=initialized audit_enabled=0 res=1 lb # [ 0.293531] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.309546] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.300752] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.309559] cpuidle: using governor menu worker2 # [ 0.301385] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.301520] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.294266] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.311773] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker2 # [ 0.302807] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.294799] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.302766] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.303317] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.312774] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.303507] audit: initializing netlink subsys (disabled) lb # [ 0.296045] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.313546] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker1 # [ 0.304606] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.304573] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.296791] audit: initializing netlink subsys (disabled) client # [ 0.314556] PCI: Using configuration type 1 for base access worker1 # [ 0.304609] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.305304] audit: initializing netlink subsys (disabled) lb # [ 0.297775] audit: type=2000 audit(1790146739.098:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.305497] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.306350] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.315670] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. lb # [ 0.298887] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.306434] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.306352] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.298890] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.307263] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.307218] audit: type=2000 audit(1790146739.098:1): state=initialized audit_enabled=0 res=1 lb # [ 0.299774] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.307971] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.308499] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.300693] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.308513] cpuidle: using governor menu worker2 # [ 0.308298] audit: type=2000 audit(1790146739.101:1): state=initialized audit_enabled=0 res=1 lb # [ 0.301487] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.301785] cpuidle: using governor menu worker2 # [ 0.310297] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.310732] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 client # [ 0.320813] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.310311] cpuidle: using governor menu client # [ 0.321544] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.303960] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.311755] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker2 # [ 0.312336] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.312499] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry lb # [ 0.305056] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker2 # [ 0.313519] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.313508] PCI: Using configuration type 1 for base access lb # [ 0.305777] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.314298] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry lb # [ 0.306785] PCI: Using configuration type 1 for base access worker1 # [ 0.314623] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.315306] PCI: Using configuration type 1 for base access client # [ 0.326544] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.307852] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.316388] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. client # [ 0.327543] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.319617] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.320497] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.321388] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.322296] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.314787] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.315773] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.325497] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.337992] ACPI: Added _OSI(Module Device) worker1 # [ 0.326495] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.338544] ACPI: Added _OSI(Processor Device) worker2 # [ 0.327296] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.339278] ACPI: Added _OSI(Processor Aggregator Device) worker2 # [ 0.328295] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.320775] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.321774] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.346920] ACPI: 1 ACPI AML tables successfully acquired and loaded worker1 # [ 0.336839] ACPI: Added _OSI(Module Device) worker1 # [ 0.337497] ACPI: Added _OSI(Processor Device) worker2 # [ 0.338679] ACPI: Added _OSI(Module Device) worker1 # [ 0.338257] ACPI: Added _OSI(Processor Aggregator Device) worker2 # [ 0.339296] ACPI: Added _OSI(Processor Device) worker2 # [ 0.340039] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.350950] ACPI: Interpreter enabled lb # [ 0.333064] ACPI: Added _OSI(Module Device) client # [ 0.351558] ACPI: PM: (supports S0 S3 S4 S5) lb # [ 0.333775] ACPI: Added _OSI(Processor Device) client # [ 0.352255] ACPI: Using IOAPIC for interrupt routing lb # [ 0.334580] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.354614] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.355543] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.346580] ACPI: 1 ACPI AML tables successfully acquired and loaded worker2 # [ 0.348359] ACPI: 1 ACPI AML tables successfully acquired and loaded lb # [ 0.341340] ACPI: 1 ACPI AML tables successfully acquired and loaded client # [ 0.358643] ACPI: Enabled 2 GPEs in block 00 to 3F worker1 # [ 0.350899] ACPI: Interpreter enabled worker1 # [ 0.351511] ACPI: PM: (supports S0 S3 S4 S5) worker2 # [ 0.352720] ACPI: Interpreter enabled lb # [ 0.345204] ACPI: Interpreter enabled worker1 # [ 0.352263] ACPI: Using IOAPIC for interrupt routing worker2 # [ 0.353310] ACPI: PM: (supports S0 S3 S4 S5) lb # [ 0.345791] ACPI: PM: (supports S0 S3 S4 S5) worker2 # [ 0.354026] ACPI: Using IOAPIC for interrupt routing lb # [ 0.346514] ACPI: Using IOAPIC for interrupt routing worker1 # [ 0.354579] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.366445] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker1 # [ 0.355497] PCI: Using E820 reservations for host bridge windows worker2 # [ 0.356360] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug lb # [ 0.348817] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.367289] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] worker2 # [ 0.357296] PCI: Using E820 reservations for host bridge windows lb # [ 0.349773] PCI: Using E820 reservations for host bridge windows client # [ 0.368622] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.358538] ACPI: Enabled 2 GPEs in block 00 to 3F client # [ 0.369668] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker2 # [ 0.360380] ACPI: Enabled 2 GPEs in block 00 to 3F lb # [ 0.352878] ACPI: Enabled 2 GPEs in block 00 to 3F client # [ 0.370932] PCI host bridge to bus 0000:00 client # [ 0.371548] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] client # [ 0.372543] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.373543] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.374543] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] client # [ 0.375543] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.365923] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.376544] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] lb # [ 0.359893] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker2 # [ 0.367930] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker1 # [ 0.366503] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.377554] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.360779] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] worker2 # [ 0.368302] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] worker1 # [ 0.367575] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] client # [ 0.378477] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.361849] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.368618] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker2 # [ 0.369375] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] client # [ 0.379962] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint lb # [ 0.362896] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker1 # [ 0.369862] PCI host bridge to bus 0000:00 worker2 # [ 0.370417] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] lb # [ 0.364140] PCI host bridge to bus 0000:00 worker1 # [ 0.370500] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] worker2 # [ 0.371646] PCI host bridge to bus 0000:00 lb # [ 0.364778] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] worker1 # [ 0.371496] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker2 # [ 0.372300] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.365773] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker2 # [ 0.373295] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker1 # [ 0.372496] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] lb # [ 0.366773] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker2 # [ 0.374295] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker1 # [ 0.373496] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] lb # [ 0.367773] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker2 # [ 0.375295] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.374496] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] lb # [ 0.368773] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.376295] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.375496] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] client # [ 0.382636] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.369773] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.376502] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.377299] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] client # [ 0.383555] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.370774] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.378295] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.377450] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint client # [ 0.384504] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.371765] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.379238] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.378989] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint client # [ 0.385512] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] lb # [ 0.373217] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.380782] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint client # [ 0.387283] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.390586] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.391482] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] client # [ 0.392310] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.381584] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] client # [ 0.392578] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] lb # [ 0.375839] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker1 # [ 0.382508] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] worker2 # [ 0.383434] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.376786] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] worker1 # [ 0.383449] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker2 # [ 0.384308] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.377740] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] client # [ 0.394230] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.385256] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker1 # [ 0.384477] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] lb # [ 0.378857] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker2 # [ 0.386337] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] client # [ 0.396583] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker1 # [ 0.386234] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.380662] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.397444] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.388016] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.398293] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] worker1 # [ 0.389532] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.383790] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.399125] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.390387] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker2 # [ 0.391336] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.384641] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker1 # [ 0.391201] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker2 # [ 0.392182] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.385507] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.391502] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker2 # [ 0.393050] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] lb # [ 0.385779] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] worker2 # [ 0.393302] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] client # [ 0.401852] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.393023] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.387316] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.403176] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker2 # [ 0.394855] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.403564] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker1 # [ 0.394532] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] lb # [ 0.388791] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker1 # [ 0.395403] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.397334] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] client # [ 0.405122] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.389635] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.398174] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker1 # [ 0.396202] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.390493] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] worker2 # [ 0.399028] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.406580] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.397072] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.391319] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.407440] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker2 # [ 0.399864] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.408269] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] client # [ 0.408549] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] worker1 # [ 0.399798] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] lb # [ 0.394047] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] client # [ 0.410082] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.402303] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.401174] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] lb # [ 0.395401] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker2 # [ 0.403930] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker1 # [ 0.401518] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] lb # [ 0.395795] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] client # [ 0.411587] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker2 # [ 0.404317] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] client # [ 0.412543] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker1 # [ 0.403070] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.397331] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.405876] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.414103] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.404530] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] lb # [ 0.398794] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker2 # [ 0.407334] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.405408] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] lb # [ 0.399654] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.415586] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.408181] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker1 # [ 0.406255] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] lb # [ 0.400497] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker2 # [ 0.409022] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] client # [ 0.416548] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] lb # [ 0.400780] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] worker1 # [ 0.406502] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] worker2 # [ 0.409303] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] client # [ 0.418148] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.402681] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.408043] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.410872] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.419586] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] lb # [ 0.404795] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker1 # [ 0.409538] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker2 # [ 0.412344] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] client # [ 0.420565] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] lb # [ 0.405745] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker1 # [ 0.410496] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker2 # [ 0.413285] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] client # [ 0.422076] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.407141] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.412049] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.414675] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.423586] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.408795] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker1 # [ 0.413538] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.416341] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] lb # [ 0.409747] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] client # [ 0.424565] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker1 # [ 0.414517] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker2 # [ 0.417300] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] client # [ 0.426073] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.411868] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.416133] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.418935] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.427561] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] lb # [ 0.413785] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker1 # [ 0.417538] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.420338] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] client # [ 0.428464] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] lb # [ 0.414742] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker1 # [ 0.418518] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker2 # [ 0.421251] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] client # [ 0.429300] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] lb # [ 0.416123] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.420043] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.422638] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.430110] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.417787] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker1 # [ 0.421538] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker2 # [ 0.424342] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.418773] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] client # [ 0.432580] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.422518] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.425294] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] client # [ 0.433411] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.420325] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.424039] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.434296] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker2 # [ 0.426874] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.421792] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] client # [ 0.435141] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.425513] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.428869] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] lb # [ 0.422647] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker1 # [ 0.426378] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] client # [ 0.436229] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.429302] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] lb # [ 0.423513] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.427242] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker2 # [ 0.430254] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] client # [ 0.436752] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.424321] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.428046] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.431654] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.438261] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] client # [ 0.438781] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.426151] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.429505] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker2 # [ 0.433303] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] client # [ 0.440232] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] lb # [ 0.427416] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker1 # [ 0.431164] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker2 # [ 0.434898] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.427795] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker1 # [ 0.431517] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker2 # [ 0.435317] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] lb # [ 0.429342] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.433065] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.436893] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.430529] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker1 # [ 0.434221] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.438030] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] lb # [ 0.430976] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.434691] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.438491] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.432415] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker1 # [ 0.436148] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker2 # [ 0.439973] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] lb # [ 0.432968] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.436687] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.440491] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.434489] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker2 # [ 0.441971] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker1 # [ 0.438199] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] client # [ 0.440757] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.443180] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.443949] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.444829] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.445789] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint client # [ 0.447602] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] client # [ 0.448513] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.449677] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.442549] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.434991] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.438717] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.451210] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.444945] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker1 # [ 0.441120] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] lb # [ 0.437397] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker2 # [ 0.445586] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker1 # [ 0.441801] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint lb # [ 0.438066] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker2 # [ 0.446625] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO lb # [ 0.439066] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO worker1 # [ 0.442794] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.455155] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 lb # [ 0.440029] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker1 # [ 0.443780] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.447554] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint client # [ 0.455685] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 client # [ 0.456643] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 lb # [ 0.441801] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker1 # [ 0.445524] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker2 # [ 0.449357] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] client # [ 0.457645] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 lb # [ 0.442642] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker1 # [ 0.446371] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker2 # [ 0.450199] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.458646] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 lb # [ 0.443794] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker1 # [ 0.447543] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint worker2 # [ 0.451334] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint client # [ 0.459643] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 lb # [ 0.445425] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.460645] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker1 # [ 0.449201] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.452919] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.461640] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 client # [ 0.462582] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 client # [ 0.463560] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 client # [ 0.464555] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 lb # [ 0.448798] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker1 # [ 0.452573] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker2 # [ 0.456733] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.465504] ACPI: PCI: Interrupt link GSID configured for IRQ 19 lb # [ 0.449882] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker2 # [ 0.457429] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker1 # [ 0.453603] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 client # [ 0.466273] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 lb # [ 0.450873] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker2 # [ 0.458427] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker1 # [ 0.454605] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 client # [ 0.466557] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.451874] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker2 # [ 0.459432] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker1 # [ 0.455604] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 client # [ 0.467514] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.452871] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker2 # [ 0.460398] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker1 # [ 0.456596] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 client # [ 0.468294] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 lb # [ 0.453870] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker2 # [ 0.461393] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.457593] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 lb # [ 0.454881] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker2 # [ 0.462397] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker1 # [ 0.458595] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 client # [ 0.469660] iommu: Default domain type: Translated lb # [ 0.455876] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker2 # [ 0.463395] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.459592] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 client # [ 0.470452] iommu: DMA domain TLB invalidation policy: lazy mode lb # [ 0.456817] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.464333] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker1 # [ 0.460531] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 client # [ 0.471543] ACPI: bus type USB registered lb # [ 0.457757] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 client # [ 0.472291] usbcore: registered new interface driver usbfs worker2 # [ 0.465272] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker1 # [ 0.461473] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 client # [ 0.472560] usbcore: registered new interface driver hub lb # [ 0.458497] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker1 # [ 0.462451] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker2 # [ 0.466260] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 client # [ 0.473423] usbcore: registered new device driver usb lb # [ 0.459565] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.463259] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker2 # [ 0.467066] ACPI: PCI: Interrupt link GSID configured for IRQ 19 lb # [ 0.460526] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker1 # [ 0.463514] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker2 # [ 0.468056] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 client # [ 0.475011] NetLabel: Initializing lb # [ 0.461504] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 client # [ 0.475545] NetLabel: domain hash size = 128 worker1 # [ 0.464481] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 worker2 # [ 0.468308] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 client # [ 0.476258] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.461786] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker1 # [ 0.465202] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker2 # [ 0.469270] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 client # [ 0.476579] NetLabel: unlabeled traffic allowed by default lb # [ 0.462741] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.470027] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker1 # [ 0.465508] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 client # [ 0.477460] PCI: Using ACPI for IRQ routing lb # [ 0.464539] iommu: Default domain type: Translated worker1 # [ 0.467524] iommu: Default domain type: Translated worker2 # [ 0.471429] iommu: Default domain type: Translated lb # [ 0.464781] iommu: DMA domain TLB invalidation policy: lazy mode worker1 # [ 0.468321] iommu: DMA domain TLB invalidation policy: lazy mode worker2 # [ 0.472225] iommu: DMA domain TLB invalidation policy: lazy mode lb # [ 0.466010] ACPI: bus type USB registered worker1 # [ 0.469511] ACPI: bus type USB registered worker2 # [ 0.473299] ACPI: bus type USB registered lb # [ 0.466754] usbcore: registered new interface driver usbfs worker2 # [ 0.474050] usbcore: registered new interface driver usbfs worker1 # [ 0.470257] usbcore: registered new interface driver usbfs lb # [ 0.467462] usbcore: registered new interface driver hub worker2 # [ 0.474309] usbcore: registered new interface driver hub worker1 # [ 0.470516] usbcore: registered new interface driver hub lb # [ 0.467782] usbcore: registered new device driver usb worker2 # [ 0.475129] usbcore: registered new device driver usb worker1 # [ 0.471382] usbcore: registered new device driver usb lb # [ 0.469491] NetLabel: Initializing worker1 # [ 0.473007] NetLabel: Initializing worker2 # [ 0.476831] NetLabel: Initializing lb # [ 0.469780] NetLabel: domain hash size = 128 worker2 # [ 0.477296] NetLabel: domain hash size = 128 worker1 # [ 0.473497] NetLabel: domain hash size = 128 lb # [ 0.470506] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker2 # [ 0.477978] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker1 # [ 0.474224] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.470831] NetLabel: unlabeled traffic allowed by default worker2 # [ 0.478376] NetLabel: unlabeled traffic allowed by default worker1 # [ 0.474551] NetLabel: unlabeled traffic allowed by default lb # [ 0.471726] PCI: Using ACPI for IRQ routing worker2 # [ 0.479258] PCI: Using ACPI for IRQ routing worker1 # [ 0.475450] PCI: Using ACPI for IRQ routing client # [ 0.521360] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.521539] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.521539] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.521548] vgaarb: loaded client # [ 0.522305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.522543] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.526631] clocksource: Switched to clocksource kvm-clock lb # [ 0.515584] pci 0000:00:01.0: vgaarb: setting as boot VGA device lb # [ 0.515769] pci 0000:00:01.0: vgaarb: bridge control possible worker1 # [ 0.520402] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker2 # [ 0.524110] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker1 # [ 0.520492] pci 0000:00:01.0: vgaarb: bridge control possible lb # [ 0.515769] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.530159] VFS: Disk quotas dquot_6.6.0 lb # [ 0.515778] vgaarb: loaded worker2 # [ 0.524291] pci 0000:00:01.0: vgaarb: bridge control possible worker1 # [ 0.520492] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.530830] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.516498] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker1 # [ 0.520501] vgaarb: loaded worker2 # [ 0.524291] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.532176] pnp: PnP ACPI init worker2 # [ 0.524300] vgaarb: loaded lb # [ 0.516773] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker1 # [ 0.521213] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.532934] ACPI: IRQ 4 override to edge(!), high(!) worker2 # [ 0.525001] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 worker1 # [ 0.521497] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker2 # [ 0.525296] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.533849] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved client # [ 0.535396] pnp: PnP ACPI: found 5 devices lb # [ 0.519863] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.524581] clocksource: Switched to clocksource kvm-clock lb # [ 0.521588] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.530385] clocksource: Switched to clocksource kvm-clock lb # [ 0.524120] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.528436] VFS: Disk quotas dquot_6.6.0 lb # [ 0.525476] pnp: PnP ACPI init worker1 # [ 0.529187] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.526305] ACPI: IRQ 4 override to edge(!), high(!) worker1 # [ 0.530640] pnp: PnP ACPI init worker2 # [ 0.533875] VFS: Disk quotas dquot_6.6.0 client # [ 0.543065] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.531496] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.527263] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker2 # [ 0.534655] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.544799] clocksource: Switched to clocksource acpi_pm worker2 # [ 0.536018] pnp: PnP ACPI init lb # [ 0.529420] pnp: PnP ACPI: found 5 devices worker1 # [ 0.532491] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved client # [ 0.545829] NET: Registered PF_INET protocol family worker2 # [ 0.536859] ACPI: IRQ 4 override to edge(!), high(!) worker1 # [ 0.534024] pnp: PnP ACPI: found 5 devices client # [ 0.547013] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.537875] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker2 # [ 0.539484] pnp: PnP ACPI: found 5 devices lb # [ 0.537328] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.541604] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns lb # [ 0.538791] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.543063] clocksource: Switched to clocksource acpi_pm lb # [ 0.539936] NET: Registered PF_INET protocol family worker1 # [ 0.544068] NET: Registered PF_INET protocol family worker2 # [ 0.547133] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns lb # [ 0.541143] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.548596] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.545344] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.549558] NET: Registered PF_INET protocol family worker2 # [ 0.550811] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.564388] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.566073] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.567497] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.568916] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.570267] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.571470] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.572781] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.573908] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.558134] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.561721] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.575168] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.576160] NET: Registered PF_XDP protocol family worker1 # [ 0.563311] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.559755] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.577032] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker2 # [ 0.567096] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.564584] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.561069] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.578012] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.568581] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.565853] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.562377] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.579030] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.569866] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.567052] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.563583] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.580090] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker2 # [ 0.571164] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.568143] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.581159] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] lb # [ 0.564688] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.572358] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.569384] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.565935] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.582224] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.570433] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.573508] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.566996] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.584015] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.571627] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.568178] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.574821] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.572573] NET: Registered PF_XDP protocol family lb # [ 0.569126] NET: Registered PF_XDP protocol family worker2 # [ 0.575948] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.569952] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker1 # [ 0.573415] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] client # [ 0.586861] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.577132] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.570946] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker1 # [ 0.574444] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.578056] NET: Registered PF_XDP protocol family worker1 # [ 0.575455] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.571951] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.578910] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] client # [ 0.588938] ACPI: \_SB_.GSIC: Enabled at IRQ 18 lb # [ 0.573024] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker1 # [ 0.576537] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker2 # [ 0.579881] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] lb # [ 0.574063] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.577595] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] client # [ 0.590816] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.580866] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker1 # [ 0.578638] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] lb # [ 0.575111] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker2 # [ 0.581927] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] client # [ 0.592499] PCI: CLS 0 bytes, default 64 worker2 # [ 0.583001] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.580304] ACPI: \_SB_.GSIA: Enabled at IRQ 16 lb # [ 0.576910] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker2 # [ 0.584131] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] client # [ 0.593438] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns client # [ 0.595225] Trying to unpack rootfs image as initramfs... worker1 # [ 0.582556] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.585910] ACPI: \_SB_.GSIA: Enabled at IRQ 16 lb # [ 0.579855] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.584891] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.588524] ACPI: \_SB_.GSIB: Enabled at IRQ 17 lb # [ 0.581960] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.586998] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.590451] ACPI: \_SB_.GSIC: Enabled at IRQ 18 lb # [ 0.583916] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.588740] PCI: CLS 0 bytes, default 64 lb # [ 0.585599] PCI: CLS 0 bytes, default 64 worker2 # [ 0.592355] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.589710] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.586523] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.594086] PCI: CLS 0 bytes, default 64 worker1 # [ 0.591443] Trying to unpack rootfs image as initramfs... lb # [ 0.588251] Trying to unpack rootfs image as initramfs... worker2 # [ 0.594981] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.596804] Trying to unpack rootfs image as initramfs... client # [ 0.640690] Initialise system trusted keyrings client # [ 0.644277] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker1 # [ 0.635344] Initialise system trusted keyrings lb # [ 0.633117] Initialise system trusted keyrings worker1 # [ 0.638361] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker2 # [ 0.641832] Initialise system trusted keyrings lb # [ 0.635913] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker2 # [ 0.644800] workingset: timestamp_bits=40 max_order=19 bucket_order=0 client # [ 0.665530] Key type asymmetric registered client # [ 0.668235] Asymmetric key parser 'x509' registered client # [ 0.669145] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) client # [ 0.672346] io scheduler mq-deadline registered client # [ 0.673147] io scheduler kyber registered worker1 # [ 0.661253] Key type asymmetric registered worker1 # [ 0.662005] Asymmetric key parser 'x509' registered worker2 # [ 0.666085] Key type asymmetric registered client # [ 0.675859] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.659857] Key type asymmetric registered worker1 # [ 0.662902] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) lb # [ 0.660587] Asymmetric key parser 'x509' registered client # [ 0.677116] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A lb # [ 0.661474] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker2 # [ 0.668755] Asymmetric key parser 'x509' registered worker1 # [ 0.666310] io scheduler mq-deadline registered worker2 # [ 0.669630] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker1 # [ 0.667113] io scheduler kyber registered lb # [ 0.664861] io scheduler mq-deadline registered lb # [ 0.665647] io scheduler kyber registered worker2 # [ 0.672847] io scheduler mq-deadline registered client # [ 0.683087] Linux agpgart interface v0.103 worker2 # [ 0.673666] io scheduler kyber registered worker1 # [ 0.670822] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.683930] ACPI: bus type drm_connector registered worker1 # [ 0.672095] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A lb # [ 0.669919] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.677326] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.671282] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A client # [ 0.688376] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.676010] Linux agpgart interface v0.103 worker2 # [ 0.678692] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.676754] ACPI: bus type drm_connector registered client # [ 0.689465] usbserial: USB Serial support registered for generic worker2 # [ 0.682580] Linux agpgart interface v0.103 lb # [ 0.676603] Linux agpgart interface v0.103 client # [ 0.692233] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.683370] ACPI: bus type drm_connector registered client # [ 0.692233] Please enable it if your BIOS has the CPPC option. lb # [ 0.677408] ACPI: bus type drm_connector registered worker1 # [ 0.681728] usbcore: registered new interface driver usbserial_generic client # [ 0.694459] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.682844] usbserial: USB Serial support registered for generic client # [ 0.697384] drop_monitor: Initializing network drop monitor service worker2 # [ 0.688159] usbcore: registered new interface driver usbserial_generic lb # [ 0.681892] usbcore: registered new interface driver usbserial_generic client # [ 0.698601] NET: Registered PF_INET6 protocol family worker2 # [ 0.689256] usbserial: USB Serial support registered for generic lb # [ 0.683007] usbserial: USB Serial support registered for generic worker1 # [ 0.686212] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker1 # [ 0.686212] Please enable it if your BIOS has the CPPC option. worker2 # [ 0.690218] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.690218] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.688458] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker2 # [ 0.692475] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled lb # [ 0.685731] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. lb # [ 0.685731] Please enable it if your BIOS has the CPPC option. client # [ 0.703782] Segment Routing with IPv6 lb # [ 0.687977] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.692360] drop_monitor: Initializing network drop monitor service client # [ 0.706292] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.693575] NET: Registered PF_INET6 protocol family client # [ 0.707443] IPI shorthand broadcast: enabled lb # [ 0.690891] drop_monitor: Initializing network drop monitor service worker2 # [ 0.697881] drop_monitor: Initializing network drop monitor service lb # [ 0.692092] NET: Registered PF_INET6 protocol family worker2 # [ 0.699037] NET: Registered PF_INET6 protocol family worker1 # [ 0.696665] Segment Routing with IPv6 worker1 # [ 0.697367] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.702221] Segment Routing with IPv6 worker1 # [ 0.699493] IPI shorthand broadcast: enabled lb # [ 0.697230] Segment Routing with IPv6 lb # [ 0.697926] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.705778] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.706907] IPI shorthand broadcast: enabled lb # [ 0.701039] IPI shorthand broadcast: enabled client # [ 0.716768] sched_clock: Marking stable (582014550, 134199039)->(791757681, -75544092) worker1 # [ 0.707869] sched_clock: Marking stable (574014539, 133172685)->(782748726, -75561502) client # [ 0.721384] registered taskstats version 1 client # [ 0.722402] Loading compiled-in X.509 certificates worker1 # [ 0.711391] registered taskstats version 1 worker1 # [ 0.712415] Loading compiled-in X.509 certificates worker2 # [ 0.715217] sched_clock: Marking stable (582014645, 132720716)->(786523767, -71788406) lb # [ 0.709445] sched_clock: Marking stable (575014719, 133698351)->(781070355, -72357285) lb # [ 0.712944] registered taskstats version 1 worker2 # [ 0.719853] registered taskstats version 1 lb # [ 0.713965] Loading compiled-in X.509 certificates worker2 # [ 0.720848] Loading compiled-in X.509 certificates client # [ 0.741236] Demotion targets for Node 0: null client # [ 0.742180] Key type .fscrypt registered worker1 # [ 0.730207] Demotion targets for Node 0: null client # [ 0.745220] Key type fscrypt-provisioning registered worker1 # [ 0.733233] Key type .fscrypt registered client # [ 0.746239] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.733969] Key type fscrypt-provisioning registered client # [ 0.747142] ima: Allocated hash algorithm: sha1 worker1 # [ 0.734940] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.732726] Demotion targets for Node 0: null client # [ 0.749256] ima: No architecture policies found worker2 # [ 0.739751] Demotion targets for Node 0: null lb # [ 0.733648] Key type .fscrypt registered worker2 # [ 0.740660] Key type .fscrypt registered worker1 # [ 0.739198] ima: Allocated hash algorithm: sha1 client # [ 0.752228] PM: Magic number: 10:823:974 worker1 # [ 0.740021] ima: No architecture policies found lb # [ 0.736720] Key type fscrypt-provisioning registered worker2 # [ 0.743740] Key type fscrypt-provisioning registered worker1 # [ 0.741008] PM: Magic number: 10:823:974 client # [ 0.753976] RAS: Correctable Errors collector initialized. lb # [ 0.737687] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.744744] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.738609] ima: Allocated hash algorithm: sha1 worker2 # [ 0.745650] ima: Allocated hash algorithm: sha1 worker2 # [ 0.746494] ima: No architecture policies found lb # [ 0.740753] ima: No architecture policies found worker1 # [ 0.744627] RAS: Correctable Errors collector initialized. worker2 # [ 0.748748] PM: Magic number: 10:823:974 lb # [ 0.743725] PM: Magic number: 10:823:974 worker2 # [ 0.750410] RAS: Correctable Errors collector initialized. lb # [ 0.745616] RAS: Correctable Errors collector initialized. client # [ 0.762928] clk: Disabling unused clocks client # [ 0.765241] PM: genpd: Disabling unused power domains worker1 # [ 0.753269] clk: Disabling unused clocks worker1 # [ 0.754001] PM: genpd: Disabling unused power domains worker2 # [ 0.760447] clk: Disabling unused clocks lb # [ 0.754593] clk: Disabling unused clocks worker2 # [ 0.762752] PM: genpd: Disabling unused power domains lb # [ 0.758743] PM: genpd: Disabling unused power domains worker1 # [ 0.918494] Freeing initrd memory: 29056K client # [ 0.933535] Freeing initrd memory: 29056K worker2 # [ 0.924203] Freeing initrd memory: 29060K worker1 # [ 0.921686] Freeing unused decrypted memory: 2028K client # [ 0.936928] Freeing unused decrypted memory: 2028K worker2 # [ 0.927390] Freeing unused decrypted memory: 2028K worker1 # [ 0.924275] Freeing unused kernel image (initmem) memory: 3652K worker1 # [ 0.925361] Write protecting the kernel read-only data: 32768k client # [ 0.939742] Freeing unused kernel image (initmem) memory: 3652K worker2 # [ 0.929989] Freeing unused kernel image (initmem) memory: 3652K worker1 # [ 0.927252] Freeing unused kernel image (text/rodata gap) memory: 1184K client # [ 0.940801] Write protecting the kernel read-only data: 32768k worker2 # [ 0.931139] Write protecting the kernel read-only data: 32768k worker1 # [ 0.928741] Freeing unused kernel image (rodata/data gap) memory: 720K client # [ 0.942730] Freeing unused kernel image (text/rodata gap) memory: 1184K worker2 # [ 0.933017] Freeing unused kernel image (text/rodata gap) memory: 1184K worker2 # [ 0.934542] Freeing unused kernel image (rodata/data gap) memory: 720K client # [ 0.944272] Freeing unused kernel image (rodata/data gap) memory: 720K lb # [ 0.929247] Freeing initrd memory: 29064K lb # [ 0.932545] Freeing unused decrypted memory: 2028K lb # [ 0.935447] Freeing unused kernel image (initmem) memory: 3652K lb # [ 0.936586] Write protecting the kernel read-only data: 32768k lb # [ 0.938533] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.940103] Freeing unused kernel image (rodata/data gap) memory: 720K worker1 # [ 0.979768] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker1 # [ 0.980911] Run /init as init process client # [ 0.994970] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker2 # [ 0.985577] x86/mm: Checked W+X mappings: passed, no W+X pages found. client # [ 0.996078] Run /init as init process worker2 # [ 0.986689] Run /init as init process worker1 # [ 0.991857] systemd[1]: Inserted module 'autofs4' worker2 # [ 0.997552] systemd[1]: Inserted module 'autofs4' lb # [ 0.991014] x86/mm: Checked W+X mappings: passed, no W+X pages found. client # [ 1.007824] systemd[1]: Inserted module 'autofs4' lb # [ 0.992077] Run /init as init process lb # [ 1.004008] systemd[1]: Inserted module 'autofs4' worker1 # [ 1.012046] fuse: init (API version 7.45) worker2 # [ 1.017813] fuse: init (API version 7.45) client # [ 1.028373] fuse: init (API version 7.45) worker1 # [ 1.019548] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.021940] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker2 # [ 1.025246] ACPI: \_SB_.GSIG: Enabled at IRQ 22 client # [ 1.036025] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker2 # [ 1.027959] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker1 # [ 1.024957] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.038807] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker1 # [ 1.026888] ACPI: \_SB_.GSIF: Enabled at IRQ 21 lb # [ 1.024443] fuse: init (API version 7.45) worker2 # [ 1.031339] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.042274] ACPI: \_SB_.GSIE: Enabled at IRQ 20 worker2 # [ 1.033594] ACPI: \_SB_.GSIF: Enabled at IRQ 21 client # [ 1.044562] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.032066] virtiofs virtio5: discovered new tag: nix-store worker1 # [ 1.033649] virtiofs virtio5: virtio_fs_setup_dax: No cache capability lb # [ 1.032053] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker2 # [ 1.039220] virtiofs virtio5: discovered new tag: nix-store client # [ 1.050294] virtiofs virtio5: discovered new tag: nix-store worker2 # [ 1.040758] virtiofs virtio5: virtio_fs_setup_dax: No cache capability lb # [ 1.034896] ACPI: \_SB_.GSIH: Enabled at IRQ 23 client # [ 1.051916] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker1 # [ 1.040573] virtiofs virtio6: discovered new tag: shared lb # [ 1.038163] ACPI: \_SB_.GSIE: Enabled at IRQ 20 worker1 # [ 1.042278] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 1.040526] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.045528] virtiofs virtio7: discovered new tag: xchg client # [ 1.058977] virtiofs virtio6: discovered new tag: shared worker2 # [ 1.050226] virtiofs virtio6: discovered new tag: shared worker1 # [ 1.046988] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 1.060561] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.051841] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 1.046243] virtiofs virtio5: discovered new tag: nix-store client # [ 1.064170] virtiofs virtio7: discovered new tag: xchg lb # [ 1.047900] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker2 # [ 1.055138] virtiofs virtio7: discovered new tag: xchg client # [ 1.065671] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 1.056662] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.055936] virtiofs virtio6: discovered new tag: shared lb # [ 1.057554] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 1.061306] virtiofs virtio7: discovered new tag: xchg lb # [ 1.062853] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 1.067989] systemd[1]: Successfully made /usr/ read-only. client # [ 1.088276] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.078850] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.084604] systemd[1]: Successfully made /usr/ read-only. client # [ 1.423876] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.404618] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.429067] systemd[1]: Detected virtualization kvm. client # [ 1.429942] systemd[1]: Detected architecture x86-64. worker1 # [ 1.416669] systemd[1]: Detected virtualization kvm. client # [ 1.430795] systemd[1]: Running in initrd. client # [ 1.431934] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.418907] systemd[1]: Detected architecture x86-64. client # [ 1.433175] systemd[1]: Hostname set to . worker1 # [ 1.421118] systemd[1]: Running in initrd. worker2 # [ 1.415628] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.423660] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.427671] systemd[1]: Detected virtualization kvm. worker1 # [ 1.426859] systemd[1]: Hostname set to . worker2 # [ 1.429868] systemd[1]: Detected architecture x86-64. worker2 # [ 1.432227] systemd[1]: Running in initrd. worker2 # [ 1.434911] systemd[1]: Initializing machine ID from random generator. lb # [ 1.421227] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.437881] systemd[1]: Hostname set to . lb # [ 1.433238] systemd[1]: Detected virtualization kvm. lb # [ 1.435414] systemd[1]: Detected architecture x86-64. lb # [ 1.437573] systemd[1]: Running in initrd. lb # [ 1.440086] systemd[1]: Initializing machine ID from random generator. lb # [ 1.442947] systemd[1]: Hostname set to . lb # [ 1.546644] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.584847] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.588030] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.589206] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.590663] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.591823] systemd[1]: Reached target Path Units. lb # [ 1.592738] systemd[1]: Reached target Slice Units. lb # [ 1.593630] systemd[1]: Reached target Swaps. lb # [ 1.594508] systemd[1]: Reached target Timer Units. lb # [ 1.595523] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.596799] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.597993] systemd[1]: Listening on Journal Sockets. lb # [ 1.599033] systemd[1]: Listening on udev Control Socket. lb # [ 1.600091] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.601094] systemd[1]: Reached target Socket Units. lb # [ 1.602877] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.606785] systemd[1]: Starting Load Kernel Module configfs... lb # [ 1.615834] systemd[1]: Starting Journal Service... client # [ 1.642451] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.641309] systemd[1]: Starting Load Kernel Modules... lb # [ 1.646288] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.649687] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.660811] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.666406] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.669490] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.676673] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.694945] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.680359] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 1.699518] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.700796] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.702287] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.686491] systemd[1]: Finished Load Kernel Module configfs. client # [ 1.703430] systemd[1]: Reached target Path Units. client # [ 1.704346] systemd[1]: Reached target Slice Units. client # [ 1.705266] systemd[1]: Reached target Swaps. client # [ 1.706085] systemd[1]: Reached target Timer Units. client # [ 1.707097] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.708367] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.709593] systemd[1]: Listening on Journal Sockets. lb # [ 1.693145] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config client # [ 1.710627] systemd[1]: Listening on udev Control Socket. client # [ 1.711671] systemd[1]: Listening on udev Kernel Socket. client # [ 1.712642] systemd[1]: Reached target Socket Units. client # [ 1.714484] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.704163] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.718288] systemd[1]: Starting Load Kernel Module configfs... lb # [ 1.704804] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.708639] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.709805] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.711260] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.712468] systemd[1]: Reached target Path Units. lb # [ 1.708307] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.713464] systemd[1]: Reached target Slice Units. worker1 # [ 1.714385] systemd[1]: Reached target Swaps. worker1 # [ 1.715273] systemd[1]: Reached target Timer Units. worker1 # [ 1.716293] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.717586] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.718755] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.721545] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.719752] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.720814] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.721780] systemd[1]: Reached target Socket Units. worker2 # [ 1.726139] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.723714] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.727365] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.728857] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.727282] systemd[1]: Starting Journal Service... lb # [ 1.721918] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.729948] systemd[1]: Reached target Path Units. worker2 # [ 1.730895] systemd[1]: Reached target Slice Units. worker1 # [ 1.728324] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 1.731803] systemd[1]: Reached target Swaps. worker2 # [ 1.732616] systemd[1]: Reached target Timer Units. worker2 # [ 1.733633] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.734902] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.736086] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.737122] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.738230] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.739196] systemd[1]: Reached target Socket Units. worker2 # [ 1.741048] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.753432] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.744808] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.757327] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.744259] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.737883] systemd[1]: Starting Journal Service... client # [ 1.766460] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.752921] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.755440] systemd[1]: Starting Journal Service... worker1 # [ 1.763825] systemd[1]: Starting Load Kernel Modules... client # [ 1.778550] systemd-journald[66]: Collecting audit messages is disabled. client # [ 1.779885] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.764250] systemd[1]: Finished Load Kernel Modules. worker1 # [ 1.768334] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.785831] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 1.772933] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.780347] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.778429] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.794594] systemd[1]: Finished Load Kernel Module configfs. worker2 # [ 1.784871] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.800645] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.789611] systemd-journald[66]: Collecting audit messages is disabled. worker2 # [ 1.792874] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.792275] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.791022] systemd[1]: Started Journal Service. worker1 # [ 1.797980] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 1.656851] systemd-modules-load[67]: Inserted module 'dm_mod' client # [ 1.811306] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.662583] systemd-modules-load[67]: Inserted module 'virtio_balloon' lb # [ 1.663606] systemd-modules-load[67]: Inserted module 'virtio_gpu' worker2 # [ 1.805314] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.666267] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.807102] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.804617] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.668441] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.670362] systemd[1]: Reached target Local File Systems. worker2 # [ 1.810702] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 1.822598] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.810615] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config lb # [ 1.675070] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.819139] systemd[1]: Finished Load Kernel Module configfs. client # [ 1.832274] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.682938] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.822416] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.825231] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config lb # [ 1.690032] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.833502] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.838815] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.851853] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.841445] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.849881] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.711028] systemd[1]: Finished Create System Files and Directories. client # [ 1.861414] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.858776] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.873289] systemd[1]: Finished Load Kernel Modules. client # [ 1.881429] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.871811] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.738612] systemd-udevd[76]: Using default interface naming scheme 'v261'. worker2 # [ 1.880337] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.880422] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.888960] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.768227] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.903645] systemd[1]: Started Journal Service. worker1 # [ 1.891746] systemd[1]: Finished Load Kernel Modules. client # client # [ 1.772321] systemd-modules-load[67]: Inserted module 'virtio_balloon' client # [ 1.773383] systemd-modules-load[67]: Inserted module 'virtio_gpu' worker2 # [ 1.901833] systemd[1]: Finished Load Kernel Modules. lb # [ 1.762056] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.899894] systemd[1]: Starting Apply Kernel Variables... client # [ 1.779069] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.785450] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.788079] systemd[1]: Reached target Local File Systems. worker2 # [ 1.912816] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 1.775262] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.910508] systemd[1]: Started Journal Service. worker1 # client # [ 1.789262] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.778279] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.781340] systemd-modules-load[67]: Inserted module 'virtio_gpu' client # [ 1.798071] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.799125] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.791211] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.930997] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.797838] systemd-modules-load[67]: Inserted module 'dm_mod' worker2 # [ 1.801970] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.799066] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.803291] systemd-modules-load[67]: Inserted module 'virtio_gpu' worker2 # [ 1.936866] systemd[1]: Started Journal Service. worker1 # [ 1.801078] systemd[1]: Reached target Local File Systems. worker1 # [ 1.806091] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.812650] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.816565] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.826186] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.818445] systemd[1]: Reached target Local File Systems. worker2 # [ 1.821078] systemd[1]: Starting Create System Files and Directories... lb # [ 1.814565] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 1.819719] systemd[1]: Finished Apply Kernel Variables. lb # [ 1.816166] systemd[1]: Reached target System Initialization. lb # [ 1.816954] systemd[1]: Reached target Basic System. worker2 # [ 1.827638] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.832545] systemd[1]: Finished Apply Kernel Variables. client # [ 1.848255] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 1.837131] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.857694] systemd[1]: Finished Create System Files and Directories. client # [ 1.870275] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.862704] systemd-udevd[80]: Using default interface naming scheme 'v261'. worker2 # [ 1.878721] systemd-udevd[80]: Using default interface naming scheme 'v261'. worker1 # [ 1.887119] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.901853] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 1.925057] systemd[1]: Finished Coldplug All udev Devices. client # [ 1.925939] systemd[1]: Reached target System Initialization. client # [ 1.926796] systemd[1]: Reached target Basic System. worker1 # [ 1.943464] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 1.945712] systemd[1]: Reached target System Initialization. worker1 # [ 1.946550] systemd[1]: Reached target Basic System. worker2 # [ 1.959079] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 1.960135] systemd[1]: Reached target System Initialization. worker2 # [ 1.960990] systemd[1]: Reached target Basic System. lb # [ 2.139414] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.152382] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 2.157926] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 lb # [ 2.164146] ehci-pci 0000:00:1d.7: EHCI Host Controller lb # [ 2.165092] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 lb # [ 2.166965] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.171474] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.176059] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 lb # [ 2.177023] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 lb # [ 2.180735] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.182232] usb usb1: Product: EHCI Host Controller lb # [ 2.183722] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd lb # [ 2.184532] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.187449] hub 1-0:1.0: USB hub found lb # [ 2.188509] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.189412] hub 1-0:1.0: 6 ports detected lb # [ 2.195610] uhci_hcd 0000:00:1d.0: UHCI Host Controller lb # [ 2.196423] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 lb # [ 2.213784] uhci_hcd 0000:00:1d.0: detected 2 ports lb # [ 2.214634] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.225386] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.226625] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.248413] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.249695] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.254877] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.259912] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.260668] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 lb # [ 2.245839] usb usb2: Product: UHCI Host Controller client # [ 2.262378] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.246590] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.266145] serio: i8042 KBD port at 0x60,0x64 irq 1 client # [ 2.270234] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 lb # [ 2.121202] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.271575] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.272684] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.273943] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.258835] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.275171] usb usb1: Product: EHCI Host Controller client # [ 2.276850] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd lb # [ 2.128054] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.269637] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.279788] usb usb1: SerialNumber: 0000:00:1d.7 client # [ 2.280815] hub 1-0:1.0: USB hub found client # [ 2.282008] hub 1-0:1.0: 6 ports detected worker1 # [ 2.270840] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.289926] uhci_hcd 0000:00:1d.0: UHCI Host Controller lb # [ 2.138797] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.290700] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker2 # [ 2.283289] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 2.281267] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 worker1 # [ 2.283074] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.280043] SCSI subsystem initialized worker1 # [ 2.284516] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 2.281900] hub 2-0:1.0: USB hub found worker1 # [ 2.288987] ehci-pci 0000:00:1d.7: EHCI Host Controller lb # [ 2.285746] hub 2-0:1.0: 2 ports detected worker1 # [ 2.289867] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker1 # [ 2.291647] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.306257] uhci_hcd 0000:00:1d.0: detected 2 ports client # [ 2.307054] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.157208] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.301343] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 worker1 # [ 2.300216] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker1 # [ 2.301528] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.314760] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.302659] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.315894] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.303636] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.165769] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.308791] serio: i8042 KBD port at 0x60,0x64 irq 1 worker1 # [ 2.306146] usb usb1: Product: EHCI Host Controller lb # [ 2.169105] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.309497] serio: i8042 AUX port at 0x60,0x64 irq 12 worker1 # [ 2.307293] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.309016] usb usb1: SerialNumber: 0000:00:1d.7 worker1 # [ 2.310356] hub 1-0:1.0: USB hub found worker2 # [ 2.314234] ehci-pci 0000:00:1d.7: EHCI Host Controller worker2 # [ 2.314963] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker1 # [ 2.313192] hub 1-0:1.0: 6 ports detected lb # [ 2.310063] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker2 # [ 2.316705] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 worker1 # [ 2.316736] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker1 # [ 2.317499] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.333709] usb usb2: Product: UHCI Host Controller worker2 # [ 2.324202] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 client # [ 2.336135] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.328208] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 worker2 # [ 2.329349] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.323318] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.340391] usb usb2: SerialNumber: 0000:00:1d.0 worker2 # [ 2.330944] usb usb1: Product: EHCI Host Controller worker2 # [ 2.331647] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.330063] uhci_hcd 0000:00:1d.0: detected 2 ports worker2 # [ 2.333546] usb usb1: SerialNumber: 0000:00:1d.7 worker1 # [ 2.331236] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 worker2 # [ 2.334832] hub 1-0:1.0: USB hub found lb # [ 2.329139] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker2 # [ 2.336344] hub 1-0:1.0: 6 ports detected client # [ 2.347401] hub 2-0:1.0: USB hub found lb # [ 2.199780] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 2.200872] systemd[1]: Reached target Initrd Root Device. client # [ 2.218769] systemd[1]: Starting Virtual Console Setup... lb # [ 2.203733] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.341385] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.338826] uhci_hcd 0000:00:1d.1: detected 2 ports worker1 # [ 2.342534] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.339639] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker2 # [ 2.346766] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker2 # [ 2.347510] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.357942] hub 2-0:1.0: 2 ports detected client # [ 2.358771] SCSI subsystem initialized lb # [ 2.346933] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.356628] uhci_hcd 0000:00:1d.0: detected 2 ports worker2 # [ 2.357856] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.351734] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.359215] usb usb2: Product: UHCI Host Controller worker1 # [ 2.359940] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd lb # [ 2.356735] usb usb3: Product: UHCI Host Controller lb # [ 2.357446] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.367915] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.369093] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.362910] usb usb3: SerialNumber: 0000:00:1d.1 worker1 # [ 2.366971] SCSI subsystem initialized lb # [ 2.364277] hub 3-0:1.0: USB hub found lb # [ 2.365768] hub 3-0:1.0: 2 ports detected worker1 # [ 2.370661] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.250227] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.371899] uhci_hcd 0000:00:1d.2: UHCI Host Controller client # [ 2.254501] systemd[1]: Finished Virtual Console Setup. lb # [ 2.373854] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker1 # [ 2.382819] hub 2-0:1.0: USB hub found lb # [ 2.379477] uhci_hcd 0000:00:1d.2: detected 2 ports client # [ 2.396025] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.396763] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.252101] systemd[1]: Starting Virtual Console Setup... lb # [ 2.381926] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 client # [ 2.264739] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.390171] usb usb2: Product: UHCI Host Controller lb # [ 2.249707] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 2.384754] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.259708] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.390410] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.268580] (udev-worker)[84]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.270545] (udev-worker)[84]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.390680] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.265944] systemd[1]: Starting Virtual Console Setup... lb # [ 2.259370] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 2.394164] usb usb4: Product: UHCI Host Controller worker1 # [ 2.399344] hub 2-0:1.0: 2 ports detected worker2 # [ 2.402785] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.403591] usb usb2: SerialNumber: 0000:00:1d.0 lb # [ 2.398760] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.415092] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 lb # [ 2.399593] usb usb4: SerialNumber: 0000:00:1d.2 client # [ 2.416890] uhci_hcd 0000:00:1d.1: detected 2 ports lb # [ 2.402078] hub 4-0:1.0: USB hub found client # [ 2.419039] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 lb # [ 2.403751] hub 4-0:1.0: 2 ports detected worker1 # [ 2.274770] (udev-worker)[87]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.423142] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.284313] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.286789] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.434257] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.290642] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.428236] hub 2-0:1.0: USB hub found worker1 # [ 2.293206] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.430684] SCSI subsystem initialized client # [ 2.440719] usb usb3: Product: UHCI Host Controller worker1 # [ 2.294930] (udev-worker)[85]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.432753] hub 2-0:1.0: 2 ports detected lb # [ 2.425648] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker1 # [ 2.296933] (udev-worker)[85]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.426842] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 2.310241] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 2.313069] systemd[1]: Reached target Initrd Root Device. client # [ 2.313873] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.305279] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.449047] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.449939] usb usb3: SerialNumber: 0000:00:1d.1 worker1 # [ 2.437204] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker1 # [ 2.437973] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker2 # [ 2.309428] systemd[1]: Finished Virtual Console Setup. client # [ 2.452526] hub 3-0:1.0: USB hub found client # [ 2.455253] hub 3-0:1.0: 2 ports detected lb # [ 2.440409] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) client # [ 2.457839] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.443721] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.451331] uhci_hcd 0000:00:1d.1: detected 2 ports lb # [ 2.448301] scsi host0: ahci worker1 # [ 2.452140] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 client # [ 2.465238] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.450050] scsi host1: ahci lb # [ 2.451651] scsi host2: ahci worker2 # [ 2.458299] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker2 # [ 2.459054] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 lb # [ 2.453145] scsi host3: ahci client # [ 2.470350] uhci_hcd 0000:00:1d.2: detected 2 ports lb # [ 2.455048] scsi host4: ahci lb # [ 2.456671] scsi host5: ahci client # [ 2.473454] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.460355] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.457679] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.461526] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.465678] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 lb # [ 2.459375] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 client # [ 2.475682] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.460559] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.331490] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 2.461747] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 client # [ 2.478240] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.333061] systemd[1]: Reached target Initrd Root Device. worker2 # [ 2.336889] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 2.462919] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 client # [ 2.480244] usb usb4: Product: UHCI Host Controller lb # [ 2.464049] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.480992] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.336275] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.340619] systemd[1]: Reached target Initrd Root Device. client # [ 2.483276] usb usb4: SerialNumber: 0000:00:1d.2 worker1 # [ 2.472203] usb usb3: Product: UHCI Host Controller worker2 # [ 2.343064] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.472894] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.477766] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.478566] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker1 # [ 2.476322] usb usb3: SerialNumber: 0000:00:1d.1 client # [ 2.489810] hub 4-0:1.0: USB hub found client # [ 2.356735] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.482284] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.493125] hub 4-0:1.0: 2 ports detected worker1 # [ 2.483358] hub 3-0:1.0: USB hub found worker1 # [ 2.485227] hub 3-0:1.0: 2 ports detected client # [ 2.364772] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 2.490893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.493422] usb usb3: Product: UHCI Host Controller worker1 # [ 2.491879] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker2 # [ 2.497267] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.498420] usb usb3: SerialNumber: 0000:00:1d.1 worker1 # [ 2.495238] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker2 # [ 2.500273] hub 3-0:1.0: USB hub found worker1 # [ 2.498281] uhci_hcd 0000:00:1d.2: detected 2 ports worker2 # [ 2.501786] hub 3-0:1.0: 2 ports detected worker1 # [ 2.499102] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.502754] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.508224] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker1 # [ 2.504932] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.506305] usb usb4: Product: UHCI Host Controller worker1 # [ 2.507134] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.509888] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 client # [ 2.524256] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.514825] uhci_hcd 0000:00:1d.2: detected 2 ports worker2 # [ 2.515664] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.379357] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.385314] systemd-fsck[106]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 2.528072] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.520874] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.531044] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker1 # [ 2.385495] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 2.532682] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.520627] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.393412] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.523642] hub 4-0:1.0: USB hub found worker2 # [ 2.527279] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.524743] hub 4-0:1.0: 2 ports detected worker2 # [ 2.529839] usb usb4: Product: UHCI Host Controller client # [ 2.540084] scsi host0: ahci worker2 # [ 2.530566] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.542091] scsi host1: ahci worker2 # [ 2.534804] usb usb4: SerialNumber: 0000:00:1d.2 client # [ 2.544820] scsi host2: ahci client # [ 2.546676] scsi host3: ahci worker2 # [ 2.537825] hub 4-0:1.0: USB hub found worker2 # [ 2.538621] hub 4-0:1.0: 2 ports detected client # [ 2.549749] scsi host4: ahci client # [ 2.552268] scsi host5: ahci client # [ 2.552828] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 client # [ 2.554966] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 client # [ 2.556175] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 client # [ 2.557352] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 client # [ 2.558504] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.545900] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode client # [ 2.559658] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker1 # [ 2.547055] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker1 # [ 2.553214] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 2.556856] ahci 0000:00:1f.2: flags: 64bit ncq only lb # [ 2.556976] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.561423] scsi host0: ahci lb # [ 2.558115] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 lb # [ 2.559468] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.563126] scsi host1: ahci lb # [ 2.560164] usb 1-1: Manufacturer: QEMU worker2 # [ 2.566520] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode lb # [ 2.560857] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker1 # [ 2.564776] scsi host2: ahci worker2 # [ 2.567659] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker1 # [ 2.566133] scsi host3: ahci worker1 # [ 2.567993] scsi host4: ahci worker1 # [ 2.569553] scsi host5: ahci worker1 # [ 2.570143] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker2 # [ 2.574484] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.572306] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker1 # [ 2.573489] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.574607] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker2 # [ 2.578754] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 2.575774] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker1 # [ 2.576932] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker2 # [ 2.581302] scsi host0: ahci lb # [ 2.575374] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.583145] scsi host1: ahci worker2 # [ 2.584775] scsi host2: ahci worker2 # [ 2.586372] scsi host3: ahci worker2 # [ 2.588137] scsi host4: ahci worker2 # [ 2.589694] scsi host5: ahci worker2 # [ 2.591141] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker2 # [ 2.592449] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker2 # [ 2.593795] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker2 # [ 2.594984] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker2 # [ 2.596129] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker2 # [ 2.597286] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.657811] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 client # [ 2.660807] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 client # [ 2.664022] usb 1-1: Product: QEMU USB Tablet client # [ 2.666054] usb 1-1: Manufacturer: QEMU client # [ 2.667690] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 lb # [ 2.536363] systemd[1]: Mounting /sysroot... client # [ 2.697072] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 2.682578] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.685432] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker1 # [ 2.689009] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.690859] usb 1-1: Manufacturer: QEMU worker1 # [ 2.692577] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker2 # [ 2.710222] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker2 # [ 2.713154] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker2 # [ 2.716388] usb 1-1: Product: QEMU USB Tablet worker2 # [ 2.718221] usb 1-1: Manufacturer: QEMU worker2 # [ 2.720140] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker1 # [ 2.724410] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.752329] hid: raw HID events driver (C) Jiri Kosina client # [ 2.644259] systemd[1]: Mounting /sysroot... lb # [ 2.777511] ata1: SATA link down (SStatus 0 SControl 300) lb # [ 2.778503] ata4: SATA link down (SStatus 0 SControl 300) lb # [ 2.779639] ata2: SATA link down (SStatus 0 SControl 300) lb # [ 2.780596] ata6: SATA link down (SStatus 0 SControl 300) lb # [ 2.781563] ata5: SATA link down (SStatus 0 SControl 300) lb # [ 2.782491] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) lb # [ 2.783519] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 lb # [ 2.784366] ata3.00: applying bridge limits lb # [ 2.785079] ata3.00: configured for UDMA/100 worker1 # [ 2.654723] systemd[1]: Mounting /sysroot... lb # [ 2.786079] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker2 # [ 2.670930] systemd[1]: Mounting /sysroot... lb # [ 2.839154] usbcore: registered new interface driver usbhid lb # [ 2.846144] usbhid: USB HID core driver client # [ 2.874856] ata6: SATA link down (SStatus 0 SControl 300) client # [ 2.877157] ata2: SATA link down (SStatus 0 SControl 300) client # [ 2.879779] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) client # [ 2.882420] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 client # [ 2.884340] ata3.00: applying bridge limits client # [ 2.886337] ata4: SATA link down (SStatus 0 SControl 300) client # [ 2.888605] ata1: SATA link down (SStatus 0 SControl 300) lb # [ 2.873950] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 client # [ 2.890670] ata5: SATA link down (SStatus 0 SControl 300) client # [ 2.893015] ata3.00: configured for UDMA/100 client # [ 2.895494] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.881853] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 lb # [ 2.885969] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker1 # [ 2.889731] ata5: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.892010] ata6: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.894644] ata4: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.896920] ata2: SATA link down (SStatus 0 SControl 300) lb # [ 2.895939] cdrom: Uniform CD-ROM driver Revision: 3.20 worker1 # [ 2.899305] ata1: SATA link down (SStatus 0 SControl 300) lb # [ 2.897416] EXT4-fs (vda): mounted filesystem f865ffbd-45df-49ba-a98a-fa2b73b6401c r/w with ordered data mode. Quota mode: none. worker1 # [ 2.901812] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker2 # [ 2.906758] ata1: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.904498] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker1 # [ 2.906358] ata3.00: applying bridge limits worker2 # [ 2.909258] ata4: SATA link down (SStatus 0 SControl 300) lb # [ 2.771275] systemd[1]: Mounted /sysroot. worker1 # [ 2.908337] ata3.00: configured for UDMA/100 lb # [ 2.772028] systemd[1]: Reached target Initrd Root File System. worker2 # [ 2.911520] ata2: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.913963] ata5: SATA link down (SStatus 0 SControl 300) lb # [ 2.775353] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 2.910757] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker2 # [ 2.916083] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker2 # [ 2.918580] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker2 # [ 2.920771] ata3.00: applying bridge limits worker2 # [ 2.922651] ata6: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.924945] ata3.00: configured for UDMA/100 worker2 # [ 2.927312] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.790631] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 2.793530] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 2.795169] systemd[1]: Reloading... client # [ 2.961295] usbcore: registered new interface driver usbhid client # [ 2.962092] usbhid: USB HID core driver client # [ 2.979280] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 client # [ 2.983688] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker1 # [ 2.985417] usbcore: registered new interface driver usbhid worker1 # [ 2.988908] usbhid: USB HID core driver worker2 # [ 3.003963] usbcore: registered new interface driver usbhid worker2 # [ 3.006733] usbhid: USB HID core driver client # [ 3.021454] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker1 # [ 3.011734] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 3.013627] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 client # [ 3.028010] EXT4-fs (vda): mounted filesystem b0304305-2254-42bb-91b1-71de4a72a3ed r/w with ordered data mode. Quota mode: none. client # [ 2.898239] systemd[1]: Mounted /sysroot. lb # [ 2.882399] systemd[1]: Reloading finished in 85 ms. worker1 # [ 3.019572] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray client # [ 2.898920] systemd[1]: Reached target Initrd Root File System. client # [ 3.034024] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 2.901471] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 2.891252] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.031208] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 3.030335] cdrom: Uniform CD-ROM driver Revision: 3.20 worker2 # [ 3.032985] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 lb # [ 2.895236] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... worker2 # [ 3.037028] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker1 # [ 3.034123] EXT4-fs (vda): mounted filesystem b6ead55f-799f-489d-8de3-4e78c873c878 r/w with ordered data mode. Quota mode: none. lb # [ 2.898548] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 2.899907] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 2.916125] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 2.901000] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 2.907262] systemd[1]: Mounted /sysroot. client # [ 2.918860] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 2.921583] systemd[1]: Reloading... worker2 # [ 3.045879] cdrom: Uniform CD-ROM driver Revision: 3.20 worker1 # [ 2.910109] systemd[1]: Reached target Initrd Root File System. worker1 # [ 2.914237] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.051991] EXT4-fs (vda): mounted filesystem 0df8a1e4-2b77-4449-8a89-10da3c9f98ba r/w with ordered data mode. Quota mode: none. worker1 # [ 2.920811] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 2.925805] systemd[1]: Mounted /sysroot. worker1 # [ 2.924140] systemd[1]: Mounting /sysroot/run... worker2 # [ 2.927722] systemd[1]: Reached target Initrd Root File System. worker2 # [ 2.931064] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 2.931436] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 2.936146] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 2.944099] systemd-sysroot-fstab-check[139]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 2.949890] systemd[1]: Reload requested from client PID 139 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 2.951409] systemd[1]: Reloading... worker1 # [ 2.949168] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 2.976926] systemd-sysroot-fstab-check[146]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.004258] systemd[1]: Reloading finished in 83 ms. worker1 # [ 2.994752] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 2.997670] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 2.999140] systemd[1]: Mounted /sysroot/run. worker1 # [ 2.999827] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 3.011823] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.000527] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.014128] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... client # [ 3.020608] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.009167] systemd[1]: Reload requested from client PID 146 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.021950] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.010632] systemd[1]: Reloading... client # [ 3.022961] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.032068] systemd[1]: Reloading finished in 83 ms. worker2 # [ 3.040082] systemd-sysroot-fstab-check[139]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.043145] systemd-sysroot-fstab-check[139]: Requesting swap.target/start/replace... worker2 # [ 3.047699] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.049121] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.051191] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.074730] systemd[1]: Reloading finished in 66 ms. worker1 # [ 3.084475] systemd-sysroot-fstab-check[146]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.087123] systemd-sysroot-fstab-check[146]: Requesting swap.target/start/replace... worker1 # [ 3.089340] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.091669] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.094179] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.096140] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.106953] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.108541] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 3.538093] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.541065] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.545148] systemd[1]: Mounting /sysroot/run... lb # [ 3.549148] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.553903] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.587238] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.588301] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.590060] systemd[1]: Mounted /sysroot/run. lb # [ 3.590728] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.591479] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.595152] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.605272] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 3.606732] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.645184] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.651166] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.660175] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.655347] systemd[1]: Mounting /sysroot/nix/store... client # [ 3.670172] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.677397] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.672278] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.679206] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.687310] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.695303] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.704896] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.717384] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.706535] systemd[1]: Reached target Initrd File Systems. worker2 # [ 3.710304] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.719948] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.708336] systemd[1]: Starting Find NixOS closure... worker1 # [ 3.711141] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 3.723779] systemd[1]: Mounted /sysroot/run. client # [ 3.724629] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 3.728620] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.731419] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.741298] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.742752] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.731034] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 3.732733] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 3.740833] systemd[1]: Finished Find NixOS closure. worker2 # [ 3.745637] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.742287] systemd[1]: Reached target Initrd Default Target. worker1 # [ 3.744064] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 3.753502] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.755115] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.755770] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.757771] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.757306] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 3.760864] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.758371] systemd[1]: Stopped target Basic System. worker1 # [ 3.759321] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 3.760363] systemd[1]: Stopped target Path Units. worker1 # [ 3.762107] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 3.763182] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 3.764366] systemd[1]: Stopped target Slice Units. worker1 # [ 3.765125] systemd[1]: Stopped target Socket Units. worker1 # [ 3.765835] systemd[1]: Stopped target System Initialization. worker1 # [ 3.766591] systemd[1]: Stopped target Swaps. worker1 # [ 3.767202] systemd[1]: Stopped target Timer Units. worker2 # [ 3.770976] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.767883] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 3.772460] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.769979] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 3.770918] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 3.772042] systemd[1]: Stopped Find NixOS closure. worker1 # [ 3.773337] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.774488] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 3.776303] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 3.778107] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 3.779086] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 3.779978] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 3.781733] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 3.783260] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 3.784299] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 3.786119] systemd[1]: Stopped target Local File Systems. worker1 # [ 3.786883] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 3.787834] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 3.789489] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 3.791064] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 3.792175] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 3.794411] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 3.804437] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 3.808144] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 3.809247] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 3.811109] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 3.815259] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 3.816462] systemd[1]: Closed udev Control Socket. worker1 # [ 3.818116] systemd[1]: Starting Cleanup udev Database... worker1 # [ 3.818948] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 3.821129] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 3.822193] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 3.824130] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 3.825080] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 3.826039] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 3.826917] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.827947] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 3.844438] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 3.846110] systemd[1]: Finished Cleanup udev Database. worker1 # [ 3.847298] systemd[1]: Reached target Switch Root. worker1 # [ 3.848916] systemd[1]: Starting NixOS Activation... worker1 # [ 3.910399] initrd-nixos-activation-start[192]: booting system configuration /nix/store/7ivsh1xpnbmbn73qmia5fjss23hffswz-nixos-system-worker1-test worker1 # [ 3.934917] initrd-nixos-activation-start[192]: running activation script... worker1 # [ 4.108844] initrd-nixos-activation-start[215]: setting up /etc... worker1 # [ 4.205729] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 4.207190] systemd[1]: Finished NixOS Activation. worker1 # [ 4.209262] systemd[1]: Starting Switch Root... worker1 # [ 4.222581] systemd[1]: Switching root. worker1 # [ 4.488688] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker1 # [ 4.588814] NET: Registered PF_VSOCK protocol family lb # [ 4.538833] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.588607] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.591106] systemd[1]: Reached target Initrd File Systems. lb # [ 4.592148] systemd[1]: Starting Find NixOS closure... lb # [ 4.594686] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.615074] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.617346] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.624733] systemd[1]: Finished Find NixOS closure. lb # [ 4.626325] systemd[1]: Reached target Initrd Default Target. lb # [ 4.628294] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.644873] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.641683] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.642913] systemd[1]: Stopped target Basic System. lb # [ 4.643858] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.646130] systemd[1]: Stopped target Path Units. lb # [ 4.646880] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.647923] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.649180] systemd[1]: Stopped target Slice Units. lb # [ 4.650120] systemd[1]: Stopped target Socket Units. lb # [ 4.650864] systemd[1]: Stopped target System Initialization. lb # [ 4.651689] systemd[1]: Stopped target Swaps. lb # [ 4.652311] systemd[1]: Stopped target Timer Units. lb # [ 4.652993] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.654133] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.655116] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.656519] systemd[1]: Stopped Find NixOS closure. lb # [ 4.657746] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.659273] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.660395] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.661455] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.663134] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.664053] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.671866] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.665290] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.666593] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.668126] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.669260] systemd[1]: Stopped target Local File Systems. lb # [ 4.670261] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.672189] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.673424] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.676622] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.677687] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.693444] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.678647] systemd[1]: Stopped Virtual Console Setup. client # [ 4.695286] systemd[1]: Reached target Initrd File Systems. client # [ 4.697065] systemd[1]: Starting Find NixOS closure... client # [ 4.699493] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.687190] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 4.689893] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 4.696589] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.698094] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.700229] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.701507] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.702688] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 4.718672] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.703810] systemd[1]: Closed udev Control Socket. lb # [ 4.705427] systemd[1]: Starting Cleanup udev Database... client # [ 4.720640] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.706255] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.707732] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.708643] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.710159] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.711141] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.712121] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 4.728668] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.720599] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.730172] systemd[1]: Reached target Initrd Default Target. client # [ 4.731423] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.723077] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.726736] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.732059] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.727975] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 4.744401] systemd[1]: Stopped target Initrd Default Target. client # [ 4.745526] systemd[1]: Stopped target Basic System. lb # [ 4.730173] systemd[1]: Finished Cleanup udev Database. client # [ 4.746487] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.730922] systemd[1]: Reached target Switch Root. client # [ 4.748109] systemd[1]: Stopped target Path Units. lb # [ 4.732427] systemd[1]: Starting NixOS Activation... client # [ 4.748802] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.749918] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.750989] systemd[1]: Stopped target Slice Units. client # [ 4.751662] systemd[1]: Stopped target Socket Units. client # [ 4.752627] systemd[1]: Stopped target System Initialization. client # [ 4.753603] systemd[1]: Stopped target Swaps. client # [ 4.754526] systemd[1]: Stopped target Timer Units. client # [ 4.756138] systemd[1]: dbus.socket: Deactivated successfully. client # [ 4.756948] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 4.757799] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 4.758798] systemd[1]: Stopped Find NixOS closure. client # [ 4.760051] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.761146] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 4.762289] systemd[1]: Stopped Apply Kernel Variables. client # [ 4.764191] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 4.765190] systemd[1]: Stopped Load Kernel Modules. client # [ 4.765897] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.758319] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.767104] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 4.768111] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 4.769094] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 4.761398] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.771109] systemd[1]: Stopped target Local File Systems. client # [ 4.771923] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 4.772844] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 4.773794] systemd[1]: Stopped Coldplug All udev Devices. client # [ 4.775446] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 4.769118] systemd[1]: Finished Find NixOS closure. client # [ 4.777464] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.770267] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.771950] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.780552] systemd[1]: Stopped Virtual Console Setup. client # [ 4.788311] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 4.790189] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 4.785597] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 4.786749] systemd[1]: Stopped target Basic System. client # [ 4.794898] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 4.788108] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.788903] systemd[1]: Stopped target Path Units. worker2 # [ 4.789684] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.798089] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 4.790679] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.799143] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 4.791888] systemd[1]: Stopped target Slice Units. client # [ 4.800195] systemd[1]: Closed udev Control Socket. worker2 # [ 4.792851] systemd[1]: Stopped target Socket Units. worker2 # [ 4.793888] systemd[1]: Stopped target System Initialization. client # [ 4.802951] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.794958] systemd[1]: Stopped target Swaps. client # [ 4.803797] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.796309] systemd[1]: Stopped target Timer Units. client # [ 4.805184] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 4.797059] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 4.797856] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.798648] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 4.806902] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.800070] systemd[1]: Stopped Find NixOS closure. client # [ 4.807991] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 4.809020] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.801135] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.809928] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.802126] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 4.810775] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.804216] systemd[1]: Stopped Apply Kernel Variables. client # [ 4.813114] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.805220] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 4.806390] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.807498] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.809118] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 4.810165] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 4.811258] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 4.812134] systemd[1]: Stopped target Local File Systems. worker2 # [ 4.814116] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.806552] initrd-nixos-activation-start[190]: booting system configuration /nix/store/5f2pyghh7bdk9s7schxljxgh1845dmij-nixos-system-lb-test worker2 # [ 4.815043] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 4.816133] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 4.817544] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 4.827907] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 4.820528] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.821532] systemd[1]: Stopped Virtual Console Setup. client # [ 4.830153] systemd[1]: Finished Cleanup udev Database. client # [ 4.830886] systemd[1]: Reached target Switch Root. worker1 # [ 4.944781] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 4.832624] systemd[1]: Starting NixOS Activation... worker1 # [ 4.954590] systemd[1]: Detected virtualization kvm. worker2 # [ 4.825942] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 4.956377] systemd[1]: Detected architecture x86-64. worker1 # [ 4.958338] systemd[1]: Detected first boot. worker2 # [ 4.830491] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 4.962427] systemd[1]: Initializing machine ID from random generator. worker2 # [ 4.837766] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 4.838755] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 4.841361] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 4.834525] initrd-nixos-activation-start[190]: running activation script... worker2 # [ 4.842449] systemd[1]: Closed udev Control Socket. worker2 # [ 4.843611] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.844455] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.846134] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 4.847055] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.848175] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.849140] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.851113] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.852070] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.853436] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.868898] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 4.871083] systemd[1]: Finished Cleanup udev Database. worker2 # [ 4.871943] systemd[1]: Reached target Switch Root. worker2 # [ 4.873608] systemd[1]: Starting NixOS Activation... client # [ 4.905168] initrd-nixos-activation-start[190]: booting system configuration /nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test client # [ 4.932846] initrd-nixos-activation-start[190]: running activation script... worker2 # [ 4.945433] initrd-nixos-activation-start[191]: booting system configuration /nix/store/xis419qg6ssac1mxl23xpjki6jlxkhwd-nixos-system-worker2-test worker1 # [ 5.083443] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 4.973076] initrd-nixos-activation-start[191]: running activation script... worker1 # [ 5.168264] systemd[1]: Applying preset policy. lb # [ 5.033864] initrd-nixos-activation-start[213]: setting up /etc... client # [ 5.137404] initrd-nixos-activation-start[213]: setting up /etc... lb # [ 5.140473] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.143070] systemd[1]: Finished NixOS Activation. lb # [ 5.145154] systemd[1]: Starting Switch Root... lb # [ 5.158254] systemd[1]: Switching root. worker2 # [ 5.166930] initrd-nixos-activation-start[214]: setting up /etc... worker1 # [ 5.355243] systemd[1]: Populated /etc with preset unit settings. client # [ 5.237701] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.240161] systemd[1]: Finished NixOS Activation. client # [ 5.241150] systemd[1]: Starting Switch Root... client # [ 5.255301] systemd[1]: Switching root. worker2 # [ 5.266791] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.269074] systemd[1]: Finished NixOS Activation. worker2 # [ 5.270248] systemd[1]: Starting Switch Root... worker2 # [ 5.282079] systemd[1]: Switching root. lb # [ 5.417043] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 5.520604] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). lb # [ 5.515314] NET: Registered PF_VSOCK protocol family worker1 # [ 5.538588] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 5.539931] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 5.543842] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.542498] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 5.544509] systemd[1]: Created slice Slice /system/getty. worker1 # [ 5.545857] systemd[1]: Created slice User and Session Slice. worker1 # [ 5.546748] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.547961] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 5.549060] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 5.549817] systemd[1]: Expecting device /dev/ttyS0... worker1 # [ 5.550566] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 5.551390] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 5.552187] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 5.552977] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 5.553837] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 5.554720] systemd[1]: Reached target Path Units. worker1 # [ 5.555448] systemd[1]: Reached target Remote File Systems. worker1 # [ 5.556241] systemd[1]: Reached target Slice Units. worker1 # [ 5.556956] systemd[1]: Reached target Swaps. worker1 # [ 5.558697] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 5.561055] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 5.562891] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 5.564713] systemd[1]: Listening on Factory Reset Management. worker1 # [ 5.577262] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 5.580003] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 5.581385] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 5.584086] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 5.585243] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 5.586373] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.587646] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 5.591523] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 5.592553] systemd[1]: Listening on udev Control Socket. worker1 # [ 5.593456] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 5.595612] systemd[1]: Mounting Huge Pages File System... worker1 # [ 5.599420] systemd[1]: Mounting POSIX Message Queue File System... client # [ 5.616896] NET: Registered PF_VSOCK protocol family worker1 # [ 5.605253] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 5.612361] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 5.620583] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 5.629694] NET: Registered PF_VSOCK protocol family worker1 # [ 5.628510] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 5.631089] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 5.635299] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 5.640720] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 5.665255] systemd[1]: Mounting FUSE Control File System... worker1 # [ 5.670205] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 5.685351] systemd[1]: Starting Journal Service... worker1 # [ 5.692254] systemd[1]: Starting Load Kernel Modules... worker1 # [ 5.698127] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 5.707410] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 5.709257] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.716232] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 5.720737] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 5.723366] systemd[1]: Mounted Huge Pages File System. worker1 # [ 5.725263] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 5.727474] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 5.729540] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 5.733262] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 5.735404] systemd[1]: Mounted FUSE Control File System. worker1 # [ 5.741862] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 5.757837] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 5.760218] systemd[1]: Finished Load Kernel Module configfs. worker1 # [ 5.766220] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 5.792385] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 5.801357] EXT4-fs (vda): re-mounted b6ead55f-799f-489d-8de3-4e78c873c878. worker1 # [ 5.812130] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 5.813468] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 5.816694] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 5.818461] systemd-journald[285]: Collecting audit messages is enabled. worker1 # [ 5.824639] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 5.826315] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 5.838909] loop: module loaded worker1 # [ 5.845818] systemd[1]: Finished Load Kernel Modules. worker1 # [ 5.850794] systemd[1]: Starting Firewall... worker1 # [ 5.856522] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 5.869978] systemd[1]: Started Journal Service. worker1 # [ 5.745159] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 5.746320] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 5.754405] systemd-modules-load[286]: Inserted module 'loop' lb # [ 5.875452] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 5.755804] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 5.885196] systemd[1]: Detected virtualization kvm. lb # [ 5.887031] systemd[1]: Detected architecture x86-64. lb # [ 5.888930] systemd[1]: Detected first boot. lb # [ 5.892802] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.781282] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 5.787172] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 5.805139] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 5.806247] systemd[1]: Reached target First Boot Complete. worker1 # [ 5.832681] systemd-oomd[287]: No swap; memory pressure usage will be degraded worker1 # [ 5.969708] systemd-journald[285]: Received client request to flush runtime journal. client # [ 5.977173] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 5.986961] systemd[1]: Detected virtualization kvm. client # [ 5.988747] systemd[1]: Detected architecture x86-64. client # [ 5.990710] systemd[1]: Detected first boot. client # [ 5.995172] systemd[1]: Initializing machine ID from random generator. worker2 # [ 5.987086] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 5.996790] systemd[1]: Detected virtualization kvm. worker2 # [ 5.998614] systemd[1]: Detected architecture x86-64. worker2 # [ 6.000556] systemd[1]: Detected first boot. worker2 # [ 6.004192] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.950106] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 5.952174] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 5.958774] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 5.960123] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 5.961323] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 5.964924] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 6.117444] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.131871] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.045269] systemd-udevd[320]: Using default interface naming scheme 'v261'. client # [ 6.200996] systemd[1]: Applying preset policy. worker2 # [ 6.242341] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.238392] systemd[1]: Applying preset policy. worker1 # [ 6.119361] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 6.179421] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 6.353380] systemd[1]: Applying preset policy. client # [ 6.389976] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.267761] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.426029] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.361711] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 6.519248] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 6.386287] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker1 # [ 6.409133] systemd[1]: Mounting /run/wrappers... worker1 # [ 6.410960] (udev-worker)[359]: Network interface NamePolicy= disabled on kernel command line. client # [ 6.571846] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 6.573992] systemd[1]: Stopped initrd-switch-root.service. client # [ 6.576967] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 6.579024] systemd[1]: Created slice Slice /system/getty. client # [ 6.580441] systemd[1]: Created slice User and Session Slice. client # [ 6.581403] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 6.434852] (udev-worker)[361]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 6.582620] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 6.583718] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 6.438116] (udev-worker)[361]: Network interface NamePolicy= disabled on kernel command line. client # [ 6.584435] systemd[1]: Expecting device /dev/ttyS0... client # [ 6.585222] systemd[1]: Reached target Local Encrypted Volumes. client # [ 6.586043] systemd[1]: Stopped target initrd-fs.target. client # [ 6.586889] systemd[1]: Stopped target initrd-root-fs.target. client # [ 6.587684] systemd[1]: Stopped target initrd-switch-root.target. client # [ 6.588530] systemd[1]: Reached target Virtual Machines and Containers. client # [ 6.589454] systemd[1]: Reached target Path Units. client # [ 6.590160] systemd[1]: Reached target Remote File Systems. client # [ 6.590957] systemd[1]: Reached target Slice Units. client # [ 6.591652] systemd[1]: Reached target Swaps. client # [ 6.593808] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 6.596385] systemd[1]: Listening on Process Core Dump Socket. client # [ 6.598221] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 6.454756] systemd[1]: Mounted /run/wrappers. worker1 # [ 6.457184] systemd[1]: Reached target Local File Systems. worker1 # [ 6.461097] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.600137] systemd[1]: Listening on Factory Reset Management. client # [ 6.612636] systemd[1]: Listening on Hostname Service Socket. client # [ 6.615625] systemd[1]: Starting Journal Log Access Socket... client # [ 6.617406] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 6.469281] systemd[1]: Starting register-nix-paths.service... client # [ 6.620158] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.621346] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.622461] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.623749] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 6.480138] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 6.481035] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 6.627959] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 6.629013] systemd[1]: Listening on udev Control Socket. client # [ 6.629865] systemd[1]: Listening on udev Varlink Socket. client # [ 6.632083] systemd[1]: Mounting Huge Pages File System... worker1 # [ 6.484939] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 6.635230] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 6.489766] systemd[1]: Starting Create System Files and Directories... client # [ 6.642364] systemd[1]: Mounting Kernel Debug File System... client # [ 6.649617] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.639400] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.641071] systemd[1]: Stopped initrd-switch-root.service. client # [ 6.658424] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.643240] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.645398] systemd[1]: Created slice Slice /system/getty. lb # [ 6.646740] systemd[1]: Created slice User and Session Slice. lb # [ 6.647643] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.648895] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.649972] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.650723] systemd[1]: Expecting device /dev/ttyS0... lb # [ 6.651475] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.652326] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.653100] systemd[1]: Stopped target initrd-root-fs.target. client # [ 6.669621] systemd[1]: Starting Load Kernel Module configfs... lb # [ 6.653890] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 6.654751] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.655700] systemd[1]: Reached target Path Units. lb # [ 6.656405] systemd[1]: Reached target Remote File Systems. lb # [ 6.657175] systemd[1]: Reached target Slice Units. lb # [ 6.657878] systemd[1]: Reached target Swaps. lb # [ 6.659853] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 6.676429] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 6.662381] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.664167] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 6.685390] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.666084] systemd[1]: Listening on Factory Reset Management. client # [ 6.693789] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.678617] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.681552] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.683149] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.686214] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.687303] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 6.688459] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.705288] systemd[1]: Mounting FUSE Control File System... lb # [ 6.689789] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 6.706160] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.694088] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.695114] systemd[1]: Listening on udev Control Socket. worker2 # [ 6.701896] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.695968] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 6.703260] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.698169] systemd[1]: Mounting Huge Pages File System... worker2 # [ 6.705232] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.700415] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 6.707379] systemd[1]: Created slice Slice /system/getty. worker2 # [ 6.708771] systemd[1]: Created slice User and Session Slice. worker2 # [ 6.709669] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 6.720784] systemd[1]: Starting Journal Service... worker2 # [ 6.710907] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 6.712024] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 6.576205] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 6.712778] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 6.713523] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 6.714363] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.708131] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 6.715101] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 6.715934] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 6.579987] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 6.716801] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 6.717725] systemd[1]: Reached target Path Units. worker2 # [ 6.718433] systemd[1]: Reached target Remote File Systems. worker2 # [ 6.719196] systemd[1]: Reached target Slice Units. worker2 # [ 6.719897] systemd[1]: Reached target Swaps. worker2 # [ 6.721548] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.715318] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 6.723825] systemd[1]: Listening on Process Core Dump Socket. client # [ 6.733583] systemd[1]: Starting Load Kernel Modules... lb # [ 6.724068] systemd[1]: Starting Create List of Static Device Nodes... client # [ 6.743743] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 6.725439] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 6.738956] systemd[1]: Listening on Factory Reset Management. worker2 # [ 6.739942] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.735625] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 6.742525] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 6.743890] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 6.746261] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.756326] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 6.747683] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 6.748828] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.742304] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 6.750129] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 6.763282] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.754074] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 6.755117] systemd[1]: Listening on udev Control Socket. worker2 # [ 6.756021] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 6.758162] systemd[1]: Mounting Huge Pages File System... lb # [ 6.753087] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 6.760536] systemd[1]: Mounting POSIX Message Queue File System... client # [ 6.771053] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 6.629207] systemd[1]: Finished Create System Files and Directories. worker2 # [ 6.766105] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.760959] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 6.633571] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 6.636896] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 6.773934] systemd[1]: Mounting Kernel Trace File System... client # [ 6.784411] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.787490] systemd[1]: Mounted Huge Pages File System. worker2 # [ 6.782395] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.775805] systemd[1]: Mounting FUSE Control File System... client # [ 6.793935] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 6.778814] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.799417] systemd-journald[283]: Collecting audit messages is enabled. client # [ 6.801433] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 6.792308] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 6.796845] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 6.807371] systemd[1]: Mounted Kernel Trace File System. lb # [ 6.791135] systemd[1]: Starting Journal Service... client # [ 6.811420] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 6.805752] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.817115] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 6.801588] systemd[1]: Starting Load Kernel Modules... worker2 # [ 6.815329] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.825982] systemd[1]: Finished Load Kernel Module configfs. lb # [ 6.809850] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 6.827320] EXT4-fs (vda): re-mounted b0304305-2254-42bb-91b1-71de4a72a3ed. client # [ 6.831586] systemd[1]: Mounted FUSE Control File System. client # [ 6.699566] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.824571] systemd[1]: Mounting FUSE Control File System... client # [ 6.837116] systemd[1]: Started Journal Service. lb # [ 6.822901] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 6.705376] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 6.828889] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.842621] loop: module loaded lb # [ 6.827829] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.840686] systemd[1]: Starting Journal Service... client # [ 6.712994] systemd-modules-load[284]: Inserted module 'loop' lb # [ 6.835653] systemd[1]: Starting Coldplug All udev Devices... client # [ 6.718152] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 6.721817] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.846983] systemd[1]: Starting Load Kernel Modules... worker1 # [ 6.715541] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 6.845840] systemd-journald[283]: Collecting audit messages is enabled. lb # [ 6.847665] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.730791] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 6.856791] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.853371] systemd[1]: Mounted Huge Pages File System. worker2 # [ 6.860796] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 6.737131] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 6.863392] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.741081] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 6.742071] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 6.861940] systemd[1]: Mounted POSIX Message Queue File System. client # [ 6.746092] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.870562] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 6.875508] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.752636] systemd-oomd[285]: No swap; memory pressure usage will be degraded worker2 # [ 6.877631] systemd[1]: Mounted Huge Pages File System. lb # [ 6.737703] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.879501] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 6.739913] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 6.881545] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.876837] systemd[1]: Started Journal Service. worker2 # [ 6.883924] systemd[1]: Mounted Kernel Trace File System. client # [ 6.761930] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 6.886228] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 6.888509] systemd[1]: Mounted FUSE Control File System. lb # [ 6.749396] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.750623] systemd[1]: Mounted Kernel Trace File System. lb # [ 6.753056] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.753928] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 6.894325] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 6.771080] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 6.772137] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.773317] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 6.760369] systemd[1]: Finished Load Kernel Module configfs. worker2 # [ 6.905252] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 6.769058] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 6.906585] systemd[1]: Finished Load Kernel Module configfs. worker1 # [ 6.771182] systemd[1]: Starting Update is Completed... worker2 # [ 6.911276] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.906357] loop: module loaded lb # [ 6.777670] systemd-modules-load[284]: Inserted module 'loop' lb # [ 6.781691] systemd[1]: Mounting Kernel Configuration File System... lb # [ 6.786156] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.796130] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.938927] systemd[1]: Mounted Kernel Configuration File System. lb # [ 6.808149] systemd[1]: Starting Firewall... lb # [ 6.942717] EXT4-fs (vda): re-mounted f865ffbd-45df-49ba-a98a-fa2b73b6401c. worker1 # [ 6.819326] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 6.967051] systemd-journald[283]: Received client request to flush runtime journal. lb # [ 6.818107] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 6.822175] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 6.824378] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 6.821232] systemd-oomd[286]: No swap; memory pressure usage will be degraded lb # [ 6.822538] systemd[1]: Mounted FUSE Control File System. worker1 # [ 6.827567] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.826264] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.966771] EXT4-fs (vda): re-mounted 0df8a1e4-2b77-4449-8a89-10da3c9f98ba. worker1 # [ 6.830130] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 6.833118] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.835124] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.972358] systemd-journald[284]: Collecting audit messages is enabled. lb # [ 6.834175] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 6.975174] loop: module loaded lb # [ 6.835145] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 6.976982] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.979081] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 6.845241] systemd[1]: Finished Update is Completed. worker2 # [ 6.980816] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.988110] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 6.850070] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 6.990764] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.851051] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.993812] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.999225] systemd[1]: Starting Firewall... lb # [ 6.860948] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 6.862182] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.004166] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.023596] systemd[1]: Started Journal Service. worker2 # [ 6.897078] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.898263] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 6.904818] systemd-modules-load[285]: Inserted module 'loop' client # [ 6.915446] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.908620] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 6.917343] systemd[1]: Finished Load/Save OS Random Seed. client # [ 6.919780] systemd[1]: Reached target First Boot Complete. client # [ 6.923429] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 6.928392] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.054761] systemd-journald[283]: Received client request to flush runtime journal. worker2 # [ 6.936449] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 6.940268] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 6.965087] systemd[1]: Finished Load/Save OS Random Seed. client # [ 6.973759] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 6.965974] systemd[1]: Reached target First Boot Complete. client # [ 6.978736] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.117048] systemd-journald[284]: Received client request to flush runtime journal. client # [ 7.004616] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 7.005195] systemd[1]: Finished Firewall. client # [ 7.022337] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 7.024230] systemd[1]: Reached target Preparation for Local File Systems. client # [ 7.026067] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.025646] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.027990] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.032114] systemd[1]: Reached target First Boot Complete. lb # [ 7.032890] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.034140] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 7.036197] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.038255] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.058147] systemd-udevd[315]: Using default interface naming scheme 'v261'. worker1 # [ 7.184251] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 client # [ 7.090110] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 7.215586] ACPI: button: Power Button [PWRF] worker2 # [ 7.093391] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.099373] systemd-oomd[286]: No swap; memory pressure usage will be degraded worker2 # [ 7.100756] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.101624] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.098944] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 7.103684] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 7.100371] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 7.107139] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 7.109616] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 7.268427] mousedev: PS/2 mouse device common for all mice lb # [ 7.131330] systemd[1]: Finished Coldplug All udev Devices. lb # [ 7.139873] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.142172] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.145286] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 7.173706] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.305820] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 7.188065] systemd-udevd[319]: Using default interface naming scheme 'v261'. worker1 # [ 7.321851] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 7.192806] systemd-udevd[320]: Using default interface naming scheme 'v261'. worker1 # [ 7.332787] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 7.337833] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 7.227513] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 7.238201] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. client # [ 7.259412] (udev-worker)[322]: Network interface NamePolicy= disabled on kernel command line. lb # [ 7.245274] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 7.264563] (udev-worker)[317]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 7.266885] (udev-worker)[317]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 7.266173] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 7.321381] systemd[1]: Finished Coldplug All udev Devices. client # [ 7.336634] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 7.338322] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.339939] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.342403] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.344133] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.345728] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.363855] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.353437] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T06:59:05 UTC (1790146745) client # [ 7.537144] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker1 # [ 7.353527] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker1 # [ 7.354045] parport_pc 00:02: reported by Plug and Play ACPI worker1 # [ 7.354125] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] client # [ 7.548640] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker2 # [ 7.407534] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.419737] systemd[1]: Finished register-nix-paths.service. worker1 # [ 7.423731] systemd[1]: Reached target System Initialization. worker1 # [ 7.424665] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 7.427361] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 7.430128] systemd[1]: Reached target Timer Units. client # [ 7.442565] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.430887] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 7.433209] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 7.436240] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 7.437114] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 7.439323] systemd[1]: Reached target Socket Units. worker1 # [ 7.441907] systemd[1]: Reached target Basic System. worker1 # [ 7.444644] systemd[1]: Started backdoor.service. worker1 # [ 7.446116] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 7.448609] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 7.446721] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 7.451170] systemd[1]: Starting Post-Boot Actions... worker1 # [ 7.458166] systemd[1]: Started Reset console on configuration changes. worker1 # [ 7.460142] systemd[1]: Starting resolvconf update... worker1 # [ 7.464813] systemd[1]: Starting D-Bus System Message Bus... lb # [ 7.470077] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. client # [ 7.486393] systemd[1]: Mounted /run/wrappers. client # [ 7.487151] systemd[1]: Reached target Local File Systems. client # [ 7.491905] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 7.355403] systemd-journald[285]: Time jumped backwards, rotating. worker1 # [ 7.432852] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 7.494088] systemd[1]: Starting register-nix-paths.service... client # [ 7.500856] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 7.501755] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 7.494759] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 7.507065] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 7.518963] systemd[1]: Starting Create System Files and Directories... lb # [ 7.510540] systemd[1]: Mounting /run/wrappers... worker2 # [ 7.525231] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. lb # [ 7.521616] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. lb # [ 7.524280] (udev-worker)[348]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 7.526896] (udev-worker)[348]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 7.545466] (udev-worker)[352]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 7.548385] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 7.548050] systemd[1]: Finished Post-Boot Actions. worker2 # [ 7.556518] (udev-worker)[355]: Network interface NamePolicy= disabled on kernel command line. lb # [ 7.557077] systemd[1]: Mounted /run/wrappers. lb # [ 7.559462] systemd[1]: Reached target Local File Systems. lb # [ 7.563497] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 7.573141] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.570476] nsncd[515]: Sep 23 06:59:05.849 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 7.567613] systemd[1]: Starting register-nix-paths.service... worker1 # [ 7.521950] Console: switching to colour dummy device 80x25 worker1 # [ 7.540549] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker1 # [ 7.540660] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker1 # [ 7.576274] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 7.572727] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.573568] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.578176] systemd[1]: Reached target Host and Network Name Lookups. worker1 # connecting to host... worker1 # [ 7.584270] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 7.585362] systemd[1]: Starting User Login Management... lb # [ 7.585965] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 7.606616] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 7.592148] systemd[1]: Starting Create System Files and Directories... client # [ 7.614095] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 7.620668] systemd[1]: Mounted /run/wrappers. worker1 # [ 7.617439] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 7.622626] systemd[1]: Reached target Local File Systems. worker2 # [ 7.625132] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 7.628140] systemd[1]: Starting register-nix-paths.service... client # [ 7.640200] systemd[1]: Finished Create System Files and Directories. worker2 # [ 7.635134] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 7.635958] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 7.644582] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 7.640085] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 7.636950] dbus-broker-launch[519]: Looking up NSS user entry for 'systemd-timesync'... client # [ 7.650809] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 7.737719] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker2 # [ 7.643461] systemd[1]: Starting Create System Files and Directories... worker1 # [ 7.737968] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker1 # [ 7.774139] [drm] Found bochs VGA, ID 0xb0c5. worker1 # [ 7.774141] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 7.651988] dbus-broker-launch[519]: NSS returned no entry for 'systemd-timesync' worker1 # [ 7.653226] dbus-broker-launch[519]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 7.667138] systemd[1]: Started D-Bus System Message Bus. client # [ 7.575130] ACPI: button: Power Button [PWRF] client # [ 7.588682] mousedev: PS/2 mouse device common for all mice worker1 # [ 7.675560] systemd[1]: Starting Virtual Console Setup... lb # [ 7.688872] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 7.696460] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 7.707236] dbus-broker-launch[519]: Ready client # [ 7.723839] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 7.718492] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 7.721378] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 7.619131] rtc_cmos PNP0B00:00: RTC can wake from S4 client # [ 7.656413] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 7.749402] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 7.873636] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client # [ 7.754349] systemd[1]: Starting Update is Completed... client # [ 7.656478] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T06:59:06 UTC (1790146746) client # [ 7.656558] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 7.751845] systemd[1]: Finished Create System Files and Directories. worker1 # [ 7.758590] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 7.759693] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 7.762945] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 7.760534] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 7.767753] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 7.771497] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 7.767771] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 7.773108] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 7.776614] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 7.779331] systemd-logind[539]: New seat seat0. worker2 # [ 7.784272] systemd[1]: Finished Create System Files and Directories. worker2 # [ 7.787699] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 7.784106] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 7.784959] systemd[1]: Started User Login Management. worker2 # [ 7.789944] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 7.787676] systemd[1]: Starting linger-users.service... client # [ 7.708075] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 client # [ 7.758412] parport_pc 00:02: reported by Plug and Play ACPI client # [ 7.805090] systemd[1]: Finished Update is Completed. worker1 # [ 7.930716] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 client # [ 7.758521] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] client # [ 7.766783] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 7.814323] Console: switching to colour dummy device 80x25 client # [ 7.960166] [drm] Found bochs VGA, ID 0xb0c5. client # [ 7.960169] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 7.845285] systemd-logind[539]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker1 # [ 7.854959] systemd[1]: linger-users.service: Deactivated successfully. client # [ 8.001727] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt client # [ 8.002475] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker1 # [ 7.857837] systemd[1]: Finished linger-users.service. worker1 # [ 7.858554] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 8.005386] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client # [ 8.006072] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 7.860388] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 7.863322] nsncd[589]: Sep 23 06:59:06.140 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 7.865860] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 7.870315] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.014335] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 client # [ 8.015977] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 7.876129] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 7.879427] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 7.888796] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 7.883315] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.891681] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.884789] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.890287] systemd[1]: Finished resolvconf update. worker2 # [ 7.893889] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.891787] systemd[1]: Reached target Preparation for Network. lb # [ 7.887705] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.896903] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.890328] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.892256] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.900303] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.897457] systemd[1]: Starting DHCP Client... lb # [ 7.894380] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.902435] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.900143] systemd[1]: Starting Address configuration of eth1... worker2 # [ 7.904783] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.902509] systemd[1]: Starting Extra networking commands.... worker1 # [ 7.915967] systemd-logind[539]: Watching system buttons on /dev/input/event2 (Power Button) lb # [ 7.916631] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 7.924709] systemd[1]: Starting Update is Completed... worker2 # [ 7.935800] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 7.937503] systemd[1]: Starting Update is Completed... client # [ 8.007234] systemd[1]: Starting Virtual Console Setup... worker2 # [ 8.022053] systemd[1]: Finished Update is Completed. worker1 # [ 8.023478] network-addresses-eth1-start[619]: adding address 192.168.1.3/24... done lb # [ 8.020049] systemd[1]: Finished Update is Completed. lb # [ 8.161251] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 lb # [ 8.170072] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console client # [ 8.198320] Console: switching to colour frame buffer device 160x50 client # [ 8.066514] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 8.069213] systemd[1]: Stopped Virtual Console Setup. client # [ 8.072697] systemd[1]: Starting Virtual Console Setup... lb # [ 8.200111] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 8.213445] mousedev: PS/2 mouse device common for all mice client # [ 8.253271] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device client # [ 8.150112] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 8.152811] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 8.278302] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker2 # [ 8.286323] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console client # [ 8.306645] ppdev: user-space parallel port driver worker2 # [ 8.165867] systemd[1]: Finished Firewall. worker1 # [ 8.226499] ppdev: user-space parallel port driver worker1 # [ 8.261735] Console: switching to colour frame buffer device 160x50 worker1 # [ 8.299843] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker1 # [ 8.026379] systemd-vconsole-setup[562]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.173751] systemd[1]: Finished Address configuration of eth1. worker1 # [ 8.175284] dhcpcd[628]: dhcpcd-10.3.2 starting worker1 # [ 8.177514] network-addresses-eth1-start[619]: adding address 2001:db8:1::3/64... done worker1 # [ 8.178936] systemd[1]: Finished Extra networking commands.. worker1 # [ 8.183358] dhcpcd[685]: dev: loaded udev worker1 # [ 8.318369] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 8.318814] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 8.189587] systemd[1]: Reached target Network. worker1 # [ 8.193140] systemd[1]: Starting Permit User Sessions... worker1 # [ 8.193959] systemd-logind[539]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 8.196368] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.197490] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.200766] systemd[1]: Starting Virtual Console Setup... client # [ 8.347297] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 8.353266] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) client # [ 8.222808] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 8.225675] systemd[1]: Stopped Virtual Console Setup. client # [ 8.235099] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.227168] systemd[1]: Finished Permit User Sessions. worker1 # [ 8.235530] systemd[1]: Started Getty on tty1. worker1 # [ 8.236270] systemd[1]: Reached target Login Prompts. worker2 # [ 8.256065] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 8.258451] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 8.405788] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker1 # [ 8.442266] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) client # [ 8.456402] kvm_amd: TSC scaling supported client # [ 8.456874] kvm_amd: Nested Virtualization enabled client # [ 8.457627] kvm_amd: Nested Paging enabled client # [ 8.458053] kvm_amd: LBR virtualization supported client # [ 8.458753] kvm_amd: Virtual VMLOAD VMSAVE supported client # [ 8.459459] kvm_amd: Virtual GIF supported client # [ 8.459878] kvm_amd: Virtual NMI enabled lb # [ 8.237906] ACPI: button: Power Button [PWRF] lb # [ 8.325066] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 8.420827] rtc_cmos PNP0B00:00: RTC can wake from S4 lb # [ 8.425980] rtc_cmos PNP0B00:00: registered as rtc0 lb # [ 8.426049] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T06:59:07 UTC (1790146747) worker1 # [ 8.466646] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.426132] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 8.330980] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 8.330372] ACPI: button: Power Button [PWRF] worker2 # [ 8.379148] mousedev: PS/2 mouse device common for all mice worker1 # [ 8.344061] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 8.416198] rtc_cmos PNP0B00:00: RTC can wake from S4 worker2 # [ 8.446824] rtc_cmos PNP0B00:00: registered as rtc0 worker2 # [ 8.446892] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T06:59:07 UTC (1790146747) client # [ 8.498090] EDAC MC: Ver: 3.0.0 worker2 # [ 8.446962] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker1 # [ 8.358357] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.360736] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.369808] systemd[1]: Starting Virtual Console Setup... lb # [ 8.445205] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized lb # [ 8.450455] parport_pc 00:02: reported by Plug and Play ACPI worker1 # [ 8.515451] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 8.516136] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 8.517683] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 8.518545] cfg80211: failed to load regulatory.db lb # [ 8.450559] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] lb # [ 8.457323] Console: switching to colour dummy device 80x25 worker2 # [ 8.457235] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 worker2 # [ 8.461658] parport_pc 00:02: reported by Plug and Play ACPI lb # [ 8.539411] [drm] Found bochs VGA, ID 0xb0c5. lb # [ 8.539414] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. lb # [ 8.408211] systemd[1]: Finished register-nix-paths.service. lb # [ 8.409171] systemd[1]: Reached target System Initialization. lb # [ 8.543800] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic lb # [ 8.543810] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 lb # [ 8.415137] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 8.416143] systemd[1]: Started niks3 garbage collection timer. lb # [ 8.416947] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 8.417863] systemd[1]: Reached target Timer Units. lb # [ 8.418625] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 8.423719] systemd[1]: Listening on niks3 server socket. lb # [ 8.424752] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 8.464740] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker2 # [ 8.473908] Console: switching to colour dummy device 80x25 lb # [ 8.426073] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 8.427894] systemd[1]: Reached target Socket Units. lb # [ 8.429206] systemd[1]: Reached target Basic System. lb # [ 8.433568] systemd[1]: Started backdoor.service. lb # [ 8.439391] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 8.445381] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 8.503475] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized worker2 # [ 8.601091] [drm] Found bochs VGA, ID 0xb0c5. worker2 # [ 8.601587] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker1 # [ 8.600091] kvm_amd: TSC scaling supported lb # [ 8.463194] systemd[1]: Starting Post-Boot Actions... worker1 # [ 8.600645] kvm_amd: Nested Virtualization enabled worker1 # [ 8.601120] kvm_amd: Nested Paging enabled worker1 # [ 8.601748] kvm_amd: LBR virtualization supported worker1 # [ 8.602812] kvm_amd: Virtual VMLOAD VMSAVE supported worker1 # [ 8.603513] kvm_amd: Virtual GIF supported worker1 # [ 8.603928] kvm_amd: Virtual NMI enabled worker2 # [ 8.617194] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker2 # [ 8.617977] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD lb # [ 8.478294] systemd[1]: Started Reset console on configuration changes. worker2 # [ 8.620639] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker1 # [ 8.624849] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 8.630075] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 8.494487] dhcpcd[685]: eth0: waiting for carrier worker1 # [ 8.495799] dhcpcd[685]: libudev: received NULL device worker1 # [ 8.497026] dhcpcd[685]: libudev: received NULL device lb # [ 8.494175] systemd[1]: Starting resolvconf update... client # [ 8.510845] systemd[1]: Finished register-nix-paths.service. worker1 # [ 8.499326] dhcpcd[685]: eth0: carrier acquired worker1 # [ 8.507407] dhcpcd[685]: DUID 00:01:00:01:32:46:35:3a:52:54:00:12:34:56 lb # [ 8.504397] systemd[1]: Started rustfs.service. worker1 # [ 8.508495] dhcpcd[685]: eth0: IAID 00:12:34:56 worker1 # [ 8.509196] dhcpcd[685]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 8.526848] systemd-vconsole-setup[419]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 8.530076] systemd[1]: Finished Virtual Console Setup. client # [ 8.531876] systemd[1]: Reached target System Initialization. client # [ 8.533396] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 8.534628] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 8.536148] systemd[1]: Reached target Timer Units. client # [ 8.536859] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 8.537819] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 8.659859] EDAC MC: Ver: 3.0.0 client # [ 8.539239] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 8.540400] systemd[1]: Reached target Socket Units. client # [ 8.541198] systemd[1]: Reached target Basic System. client # [ 8.543971] systemd[1]: Started backdoor.service. client # [ 8.546066] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 8.550723] systemd[1]: Starting nftables firewall... lb # [ 8.535080] systemd[1]: Starting rustfs-bucket.service... worker2 # [ 8.682849] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 client # [ 8.559156] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 8.683933] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 8.546740] systemd[1]: Finished Firewall. client # [ 8.567073] systemd[1]: Starting Post-Boot Actions... lb # connecting to host... client # [ 8.580175] systemd[1]: Started Reset console on configuration changes. lb # [ 8.574927] systemd[1]: Finished Post-Boot Actions. worker1 # [ 8.579638] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.583137] systemd[1]: Finished Virtual Console Setup. client # [ 8.595630] systemd[1]: Starting resolvconf update... worker2 # [ 8.592821] systemd[1]: Finished register-nix-paths.service. worker2 # [ 8.593820] systemd[1]: Reached target System Initialization. worker2 # [ 8.596321] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 8.597465] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 8.599244] systemd[1]: Reached target Timer Units. worker2 # [ 8.600579] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 8.593388] nsncd[515]: Sep 23 06:59:07.799 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 8.602285] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 8.603258] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 8.605069] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 8.598291] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 8.606220] systemd[1]: Reached target Socket Units. worker2 # [ 8.608637] systemd[1]: Reached target Basic System. worker2 # [ 8.610391] systemd[1]: Started backdoor.service. client # connecting to host... worker2 # [ 8.614158] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 8.615735] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 8.621181] systemd[1]: Starting Post-Boot Actions... worker2 # [ 8.627383] systemd[1]: Started Reset console on configuration changes. worker2 # [ 8.636921] systemd[1]: Starting resolvconf update... client: Guest shell says: b'Spawning backdoor root shell...\n' lb # [ 8.631232] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 8.632136] systemd[1]: Reached target User and Group Name Lookups. client # [ 8.647104] nsncd[427]: Sep 23 06:59:07.624 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 8.641225] systemd[1]: Starting D-Bus System Message Bus... client # [ 8.652142] systemd[1]: Starting D-Bus System Message Bus... lb # [ 8.655143] systemd[1]: Starting User Login Management... client # [ 8.657058] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 8.641332] systemd[1]: Starting D-Bus System Message Bus... client: connected to guest root shell client # [ 8.662970] systemd[1]: Finished Post-Boot Actions. lb # [ 8.683475] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 8.669928] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 8.670685] systemd[1]: Starting Virtual Console Setup... client # [ 8.670915] systemd[1]: Reached target User and Group Name Lookups. client: (connecting took 9.40 seconds) client # [ 8.684094] systemd[1]: Starting User Login Management... client: (finished: waiting for the VM to finish booting, in 9.40 seconds) client # [ 8.685902] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 8.724737] systemd[1]: Finished Post-Boot Actions. worker2 # connecting to host... client # [ 8.775397] dbus-broker-launch[444]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 8.904075] Console: switching to colour frame buffer device 160x50 client # [ 8.785968] dbus-broker-launch[444]: NSS returned no entry for 'systemd-timesync' lb # [ 8.649998] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt client # [ 8.787045] dbus-broker-launch[444]: Invalid user-name in /nix/store/c4mprcfibsx637q0hk2g869f3wvd2k8k-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 8.810872] systemd-logind[456]: New seat seat0. client # [ 8.813945] systemd-logind[456]: Watching system buttons on /dev/input/event2 (Power Button) client # [ 8.818877] systemd-logind[456]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 8.820302] systemd-logind[456]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 8.977874] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker2 # [ 8.765456] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 8.848748] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 8.850328] dbus-broker-launch[519]: Looking up NSS user entry for 'systemd-timesync'... client # [ 8.860832] systemd[1]: Started User Login Management. client # [ 8.861599] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 8.854669] dbus-broker-launch[519]: NSS returned no entry for 'systemd-timesync' lb # [ 8.650133] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD lb # [ 8.692200] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker2 # [ 8.861245] dbus-broker-launch[519]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 8.863661] nsncd[515]: Sep 23 06:59:07.947 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 8.873294] dbus-broker-launch[444]: Ready client # [ 8.876538] systemd[1]: Starting linger-users.service... worker2 # [ 8.868424] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 8.877196] systemd[1]: Starting User Login Management... worker2 # [ 8.877958] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 8.878923] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 8.879691] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 8.881527] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 8.883115] systemd[1]: Starting Virtual Console Setup... worker2 # [ 8.896468] dbus-broker-launch[519]: Ready client # [ 8.906993] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 8.909429] systemd[1]: Stopping Host and Network Name Lookups... client # [ 8.910965] systemd[1]: Stopped target User and Group Name Lookups. client # [ 8.913277] systemd[1]: Stopping User and Group Name Lookups... client # [ 8.914820] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 8.917925] systemd[1]: nscd.service: Deactivated successfully. client # [ 8.921247] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 8.692453] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 lb # [ 8.903973] Console: switching to colour frame buffer device 160x50 client # [ 8.936522] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 8.940282] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 8.941711] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 8.944587] systemd[1]: Stopped target User and Group Name Lookups. client # [ 8.954640] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 8.946842] systemd[1]: Stopping User and Group Name Lookups... client # [ 8.958677] systemd[1]: Finished linger-users.service. worker2 # [ 8.951456] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 8.955470] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 8.958946] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 8.960696] systemd-logind[540]: Watching system buttons on /dev/input/event2 (Power Button) client # [ 8.976325] systemd[1]: Finished resolvconf update. worker2 # [ 8.964206] systemd-logind[540]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 8.969558] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 8.971741] systemd-logind[540]: New seat seat0. client # [ 8.981388] systemd[1]: Starting DHCP Client... worker2 # [ 8.974124] systemd[1]: Started User Login Management. worker2 # [ 8.974867] systemd[1]: Starting linger-users.service... client # [ 8.987892] nsncd[526]: Sep 23 06:59:07.965 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 8.990802] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 8.994574] systemd[1]: Reached target Host and Network Name Lookups. client # [ 8.995580] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.119920] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 8.886716] dbus-broker-launch[547]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 8.992846] dbus-broker-launch[547]: NSS returned no entry for 'systemd-timesync' lb # [ 8.993826] dbus-broker-launch[547]: Invalid user-name in /nix/store/xhp17752g31xx87ahxdgrix2v9kism59-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 9.142525] ppdev: user-space parallel port driver lb # [ 8.998475] systemd[1]: Starting Virtual Console Setup... lb # [ 9.004349] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.005643] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.007951] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 9.010965] systemd[1]: Stopping User and Group Name Lookups... lb # [ 9.013564] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 9.015209] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.016089] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 9.016916] systemd[1]: Started D-Bus System Message Bus. lb # [ 9.021171] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.022334] systemd-logind[550]: New seat seat0. worker2 # [ 9.030119] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 9.023669] dbus-broker-launch[547]: Ready worker2 # [ 9.033100] systemd[1]: Finished linger-users.service. lb # [ 9.025692] systemd-logind[550]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) lb # [ 9.026992] systemd-logind[550]: Watching system buttons on /dev/input/event2 (Power Button) lb # [ 9.029124] systemd[1]: Started User Login Management. lb # [ 9.029847] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.033316] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 9.041804] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 9.043636] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 9.044495] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 9.046347] nsncd[589]: Sep 23 06:59:08.231 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 9.064400] systemd[1]: Finished resolvconf update. lb # [ 9.057623] systemd[1]: Starting linger-users.service... worker2 # [ 9.065538] systemd[1]: Reached target Preparation for Network. lb # [ 9.063313] systemd[1]: Starting Virtual Console Setup... worker2 # [ 9.071160] systemd[1]: Starting DHCP Client... client # [ 9.081284] dhcpcd[554]: dhcpcd-10.3.2 starting worker2 # [ 9.073601] systemd[1]: Starting Address configuration of eth1... worker2 # [ 9.075900] systemd[1]: Starting Extra networking commands.... client # [ 9.086112] dhcpcd[562]: dev: loaded udev client # [ 9.228834] 8021q: 802.1Q VLAN Support v1.8 lb # [ 9.079819] nsncd[613]: Sep 23 06:59:08.286 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.083063] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.083952] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.087480] systemd[1]: Reached target User and Group Name Lookups. lb # [ 9.109632] systemd[1]: Finished resolvconf update. lb # [ 9.117461] systemd[1]: Reached target Preparation for Network. client # [ 9.136240] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.121866] systemd-logind[550]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 9.131159] systemd[1]: Starting DHCP Client... lb # [ 9.140470] systemd[1]: Starting Address configuration of eth1... client # [ 9.305844] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 9.156246] systemd[1]: Starting Extra networking commands.... worker2 # [ 9.180182] network-addresses-eth1-start[619]: adding address 192.168.1.4/24... done client # [ 9.324194] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 9.324841] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 9.326863] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 9.327690] cfg80211: failed to load regulatory.db lb # [ 9.184647] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 9.195561] network-addresses-eth1-start[619]: adding address 2001:db8:1::4/64... done worker2 # [ 9.199128] systemd-vconsole-setup[570]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 9.194804] systemd[1]: Finished linger-users.service. worker2 # [ 9.203314] systemd[1]: Finished Virtual Console Setup. client # [ 9.213894] systemd[1]: Finished nftables firewall. client # [ 9.216436] systemd[1]: Reached target Preparation for Network. lb # [ 9.336572] ppdev: user-space parallel port driver client # [ 9.221919] systemd[1]: Starting Address configuration of eth1... client # [ 9.225632] systemd[1]: Starting Extra networking commands.... worker2 # [ 9.221485] systemd[1]: Finished Address configuration of eth1. worker2 # [ 9.231443] systemd-logind[540]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 9.376835] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 9.245680] dhcpcd[562]: eth0: waiting for carrier client # [ 9.246461] dhcpcd[562]: eth0: carrier acquired client # [ 9.255559] dhcpcd[562]: DUID 00:01:00:01:32:46:35:3c:52:54:00:12:34:56 client # [ 9.257070] dhcpcd[562]: eth0: IAID 00:12:34:56 client # [ 9.257793] dhcpcd[562]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 9.413733] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 9.274162] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 9.275182] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 9.275932] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 9.278783] systemd[1]: Starting Virtual Console Setup... client # [ 9.291331] network-addresses-eth1-start[580]: adding address 192.168.1.1/24... done worker2 # [ 9.420242] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 9.300427] network-addresses-eth1-start[580]: adding address 2001:db8:1::1/64... done client # [ 9.313114] systemd[1]: Finished Address configuration of eth1. worker2 # [ 9.318815] systemd[1]: Finished Extra networking commands.. worker2 # [ 9.319688] systemd[1]: Reached target Network. worker2 # [ 9.323303] systemd[1]: Starting Permit User Sessions... client # [ 9.333241] systemd[1]: Finished Extra networking commands.. client # [ 9.334810] systemd[1]: Reached target Network. client # [ 9.337792] systemd[1]: Starting Permit User Sessions... lb # [ 9.455608] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) client # [ 9.354434] systemd[1]: Finished Permit User Sessions. client # [ 9.357494] systemd[1]: Started Getty on tty1. client # [ 9.359119] systemd[1]: Reached target Login Prompts. worker2 # [ 9.486635] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) lb # [ 9.351632] network-addresses-eth1-start[641]: adding address 192.168.1.2/24... done worker2 # [ 9.364991] systemd[1]: Finished Permit User Sessions. worker2 # [ 9.367757] systemd[1]: Started Getty on tty1. worker2 # [ 9.369134] systemd[1]: Reached target Login Prompts. lb # [ 9.375384] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.384262] systemd[1]: Stopped Virtual Console Setup. lb # [ 9.392164] network-addresses-eth1-start[641]: adding address 2001:db8:1::2/64... done worker2 # [ 9.401069] dhcpcd[647]: dhcpcd-10.3.2 starting lb # [ 9.402884] systemd[1]: Starting Virtual Console Setup... worker2 # [ 9.413894] dhcpcd[692]: dev: loaded udev lb # [ 9.431231] systemd[1]: Finished Address configuration of eth1. lb # [ 9.571387] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker2 # [ 9.601892] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 9.602333] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 9.604438] kvm_amd: TSC scaling supported worker2 # [ 9.604859] kvm_amd: Nested Virtualization enabled worker2 # [ 9.605297] kvm_amd: Nested Paging enabled worker2 # [ 9.607570] kvm_amd: LBR virtualization supported worker2 # [ 9.608202] kvm_amd: Virtual VMLOAD VMSAVE supported worker2 # [ 9.609224] kvm_amd: Virtual GIF supported worker2 # [ 9.609660] kvm_amd: Virtual NMI enabled worker2 # [ 9.500682] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.513873] dhcpcd[657]: dhcpcd-10.3.2 starting lb # [ 9.530270] dhcpcd[713]: dev: loaded udev worker2 # [ 9.693897] EDAC MC: Ver: 3.0.0 worker2 # [ 9.698440] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 9.710281] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 9.711007] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 9.712983] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 9.713845] cfg80211: failed to load regulatory.db lb # [ 9.713861] 8021q: 802.1Q VLAN Support v1.8 lb # [ 9.714353] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 9.590215] systemd[1]: Finished Extra networking commands.. lb # [ 9.591188] systemd[1]: Reached target Network. lb # [ 9.598570] systemd[1]: Starting PostgreSQL Server... worker2 # [ 9.742588] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 9.611133] dhcpcd[692]: eth0: waiting for carrier lb # [ 9.604918] systemd[1]: Starting Permit User Sessions... worker2 # [ 9.613295] dhcpcd[692]: eth0: carrier acquired worker2 # [ 9.618350] dhcpcd[692]: DUID 00:01:00:01:32:46:35:3c:52:54:00:12:34:56 worker2 # [ 9.619500] dhcpcd[692]: eth0: IAID 00:12:34:56 worker2 # [ 9.620378] dhcpcd[692]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 9.634567] systemd-vconsole-setup[679]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 9.637332] systemd[1]: Finished Virtual Console Setup. lb # [ 9.671719] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.806870] kvm_amd: TSC scaling supported lb # [ 9.807344] kvm_amd: Nested Virtualization enabled lb # [ 9.810039] kvm_amd: Nested Paging enabled lb # [ 9.810465] kvm_amd: LBR virtualization supported lb # [ 9.815252] kvm_amd: Virtual VMLOAD VMSAVE supported lb # [ 9.815745] kvm_amd: Virtual GIF supported lb # [ 9.816126] kvm_amd: Virtual NMI enabled lb # [ 9.719205] systemd[1]: Finished Permit User Sessions. lb # [ 9.731934] systemd[1]: Started Getty on tty1. lb # [ 9.732654] systemd[1]: Reached target Login Prompts. lb # [ 9.943918] EDAC MC: Ver: 3.0.0 lb # [ 9.949644] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 9.987868] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 9.988591] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 9.992660] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 9.993597] cfg80211: failed to load regulatory.db client # [ 9.923979] dhcpcd[562]: eth0: soliciting a DHCP lease client # [ 10.076346] NET: Registered PF_PACKET protocol family client # [ 9.948761] dhcpcd[562]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 9.951353] dhcpcd[562]: eth0: probing address 10.0.2.15/24 lb # [ 10.072376] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 9.940826] dhcpcd[713]: eth0: waiting for carrier lb # [ 9.942129] dhcpcd[713]: eth0: carrier acquired lb # [ 9.952440] postgresql-pre-start[733]: The files belonging to this database system will be owned by user "postgres". lb # [ 9.954541] postgresql-pre-start[733]: This user must also own the server process. lb # [ 9.955875] dhcpcd[713]: DUID 00:01:00:01:32:46:35:3d:52:54:00:12:34:56 lb # [ 9.957118] dhcpcd[713]: eth0: IAID 00:12:34:56 lb # [ 9.957854] dhcpcd[713]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 9.962831] postgresql-pre-start[733]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 9.964064] postgresql-pre-start[733]: The default database encoding has accordingly been set to "UTF8". lb # [ 9.965193] postgresql-pre-start[733]: The default text search configuration will be set to "english". lb # [ 9.966358] postgresql-pre-start[733]: Data page checksums are enabled. lb # [ 9.967274] postgresql-pre-start[733]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 9.968433] postgresql-pre-start[733]: creating subdirectories ... ok lb # [ 9.969313] postgresql-pre-start[733]: selecting dynamic shared memory implementation ... posix client: (finished: waiting for unit network-addresses-eth1.service, in 10.66 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 10.061474] dhcpcd[685]: eth0: soliciting a DHCP lease lb # [ 10.059203] postgresql-pre-start[733]: selecting default "max_connections" ... 100 lb: (finished: waiting for unit network-addresses-eth1.service, in 0.07 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker1 # [ 10.212440] NET: Registered PF_PACKET protocol family worker1 # [ 10.086223] dhcpcd[685]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 10.089443] dhcpcd[685]: eth0: probing address 10.0.2.15/24 lb # [ 10.109833] systemd-vconsole-setup[680]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 10.115676] systemd[1]: Finished Virtual Console Setup. lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 10.130314] postgresql-pre-start[733]: selecting default "shared_buffers" ... 128MB worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.03 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.02 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2 # [ 10.237553] dhcpcd[692]: eth0: soliciting a DHCP lease worker2 # [ 10.382251] NET: Registered PF_PACKET protocol family worker2 # [ 10.256549] dhcpcd[692]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 10.257565] dhcpcd[692]: eth0: probing address 10.0.2.15/24 worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.04 seconds) lb: waiting for unit niks3.service lb # [ 10.771611] postgresql-pre-start[733]: selecting default time zone ... UTC lb # [ 10.774298] postgresql-pre-start[733]: creating configuration files ... ok lb # [ 10.938905] postgresql-pre-start[733]: running bootstrap script ... ok worker2 # [ 11.243950] dhcpcd[692]: eth0: soliciting an IPv6 router worker2 # [ 11.246947] dhcpcd[692]: eth0: Router Advertisement from fe80::2 worker2 # [ 11.249137] dhcpcd[692]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 11.251355] dhcpcd[692]: eth0: adding route to fec0::/64 worker2 # [ 11.253289] dhcpcd[692]: eth0: adding default route via fe80::2 worker1 # [ 11.267084] dhcpcd[685]: eth0: soliciting an IPv6 router worker1 # [ 11.270396] dhcpcd[685]: eth0: Router Advertisement from fe80::2 worker1 # [ 11.272756] dhcpcd[685]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 11.275209] dhcpcd[685]: eth0: adding route to fec0::/64 worker1 # [ 11.277239] dhcpcd[685]: eth0: adding default route via fe80::2 lb # [ 11.301623] postgresql-pre-start[733]: performing post-bootstrap initialization ... ok lb # [ 11.400454] dhcpcd[713]: eth0: soliciting a DHCP lease lb # [ 11.545087] NET: Registered PF_PACKET protocol family lb # [ 11.416503] dhcpcd[713]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.418156] dhcpcd[713]: eth0: probing address 10.0.2.15/24 lb # [ 11.492903] postgresql-pre-start[733]: syncing data to disk ... ok lb # [ 11.494938] postgresql-pre-start[733]: initdb: warning: enabling "trust" authentication for local connections lb # [ 11.496158] postgresql-pre-start[733]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 11.498027] postgresql-pre-start[733]: Success. You can now start the database server using: lb # [ 11.499083] postgresql-pre-start[733]: pg_ctl -D /var/lib/postgresql/18 -l logfile start client # [ 11.566510] dhcpcd[562]: eth0: soliciting an IPv6 router client # [ 11.569512] dhcpcd[562]: eth0: Router Advertisement from fe80::2 client # [ 11.571670] dhcpcd[562]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 11.573803] dhcpcd[562]: eth0: adding route to fec0::/64 client # [ 11.575641] dhcpcd[562]: eth0: adding default route via fe80::2 lb # [ 11.572790] postgres[786]: [786] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 11.575609] postgres[786]: [786] LOG: listening on IPv6 address "::1", port 5432 lb # [ 11.576696] postgres[786]: [786] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 11.579786] postgres[786]: [786] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 11.593262] postgres[795]: [795] LOG: database system was shut down at 2026-09-23 06:59:10 GMT lb # [ 11.599469] postgres[786]: [786] LOG: database system is ready to accept connections lb # [ 11.605120] systemd[1]: Started PostgreSQL Server. lb # [ 11.610216] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 11.732387] postgresql-setup-start[806]: CREATE DATABASE lb # [ 11.760952] postgresql-setup-start[811]: CREATE ROLE lb # [ 11.773180] postgresql-setup-start[813]: ALTER DATABASE lb # [ 11.777475] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 11.779384] systemd[1]: Reached target PostgreSQL. lb # [ 12.453911] dhcpcd[713]: eth0: soliciting an IPv6 router lb # [ 12.456554] dhcpcd[713]: eth0: Router Advertisement from fe80::2 lb # [ 12.458568] dhcpcd[713]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 12.461243] dhcpcd[713]: eth0: adding route to fec0::/64 lb # [ 12.462761] dhcpcd[713]: eth0: adding default route via fe80::2 client # [ 14.200237] dhcpcd[562]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 14.201801] dhcpcd[562]: eth0: adding route to 10.0.2.0/24 client # [ 14.202816] dhcpcd[562]: eth0: adding default route via 10.0.2.2 client # [ 14.261656] systemd[1]: Started DHCP Client. client # [ 14.264130] systemd[1]: Reached target Multi-User System. client # [ 14.265800] systemd[1]: Startup finished in 865ms (kernel) + 4.592s (initrd) + 8.807s (userspace) = 14.265s. worker1 # [ 15.100341] dhcpcd[685]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 15.103536] dhcpcd[685]: eth0: adding route to 10.0.2.0/24 worker1 # [ 15.105421] dhcpcd[685]: eth0: adding default route via 10.0.2.2 worker1 # [ 15.188734] systemd[1]: Started DHCP Client. worker1 # [ 15.190864] systemd[1]: Reached target Multi-User System. worker1 # [ 15.191750] systemd[1]: Startup finished in 851ms (kernel) + 3.585s (initrd) + 10.753s (userspace) = 15.190s. worker2 # [ 15.428223] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 15.431513] dhcpcd[692]: eth0: adding route to 10.0.2.0/24 worker2 # [ 15.433856] dhcpcd[692]: eth0: adding default route via 10.0.2.2 worker2 # [ 15.515235] systemd[1]: Started DHCP Client. worker2 # [ 15.517311] systemd[1]: Reached target Multi-User System. worker2 # [ 15.518193] systemd[1]: Startup finished in 857ms (kernel) + 4.618s (initrd) + 10.040s (userspace) = 15.517s. lb # [ 15.847601] dhcpcd[713]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 15.850450] dhcpcd[713]: eth0: adding route to 10.0.2.0/24 lb # [ 15.852988] dhcpcd[713]: eth0: adding default route via 10.0.2.2 lb # [ 15.933501] systemd[1]: Started DHCP Client. lb # [ 15.935148] systemd[1]: Reached target Network is Online. lb # [ 15.938719] systemd[1]: Started Envoy reverse proxy. lb # [ 15.942261] systemd[1]: Started mock-oidc.service. lb # [ 16.054968] mock-oidc-server[913]: Mock OIDC Server running lb # [ 16.055925] mock-oidc-server[913]: OIDC Address: 192.168.1.2:8080 lb # [ 16.056778] mock-oidc-server[913]: Issue Address: 0.0.0.0:8081 lb # [ 16.057745] mock-oidc-server[913]: Issuer: http://192.168.1.2:8080/oidc lb # [ 16.058638] mock-oidc-server[913]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 16.059641] mock-oidc-server[913]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 16.060871] mock-oidc-server[913]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 16.204282] envoy[912]: [2026-09-23 06:59:15.411][912][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104) lb # [ 16.207069] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 16.208541] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 16.210472] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 16.212412] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 16.224271] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 16.226226] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 16.228598] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 16.232445] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 16.236611] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 16.238288] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 16.240994] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 16.243035] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 16.244635] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 16.246370] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 16.248978] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 16.251220] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 16.254260] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 16.256286] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 16.258382] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 16.260111] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 16.261715] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 16.265772] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 16.269456] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 16.271920] envoy[912]: [2026-09-23 06:59:15.413][912][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 16.276037] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 16.278239] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 16.281527] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 16.283854] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 16.286074] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 16.287603] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 16.290249] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 16.295938] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 16.297978] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 16.303149] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 16.304884] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 16.306665] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 16.309315] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 16.315857] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 16.317414] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 16.320150] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 16.322682] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 16.325467] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 16.352357] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 16.354036] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 16.355958] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 16.358362] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 16.369926] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 16.371850] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 16.374727] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 16.378619] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 16.380218] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 16.382316] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 16.384173] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 16.386932] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 16.391800] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 16.393907] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 16.397508] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 16.399030] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 16.402029] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 16.403864] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 16.405429] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 16.407059] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.409051] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 16.411355] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 16.412922] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 16.416232] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 16.418077] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 16.421268] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.425116] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 16.431174] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 16.432908] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 16.434294] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 16.436782] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 16.442887] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 16.444926] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 16.446616] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 16.448797] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 16.450410] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 16.455865] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 16.462314] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 16.464057] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 16.465916] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 16.468055] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.469922] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 16.474133] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 16.483456] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 16.486422] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 16.488841] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 16.491153] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 16.494443] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 16.496287] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 16.498704] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 16.503155] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 16.504716] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 16.506390] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 16.509133] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 16.517043] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 16.519876] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 16.521565] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 16.523872] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 16.526838] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 16.528618] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 16.530954] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 16.534110] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 16.535909] envoy[912]: [2026-09-23 06:59:15.414][912][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 16.537720] envoy[912]: [2026-09-23 06:59:15.448][912][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 16.539038] envoy[912]: [2026-09-23 06:59:15.479][912][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 16.552036] envoy[912]: [2026-09-23 06:59:15.479][912][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 16.553379] envoy[912]: [2026-09-23 06:59:15.479][912][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 16.562540] envoy[912]: [2026-09-23 06:59:15.479][912][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 16.572854] envoy[912]: [2026-09-23 06:59:15.780][912][info][main] [source/server/server.cc:968] runtime: {} lb # [ 16.575786] envoy[912]: [2026-09-23 06:59:15.783][912][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 16.577398] envoy[912]: [2026-09-23 06:59:15.783][912][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 16.579266] envoy[912]: [2026-09-23 06:59:15.786][912][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 16.581170] envoy[912]: [2026-09-23 06:59:15.788][912][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 16.582902] envoy[912]: [2026-09-23 06:59:15.790][912][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 16.584326] envoy[912]: [2026-09-23 06:59:15.790][912][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 16.603892] envoy[912]: [2026-09-23 06:59:15.811][912][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 16.609472] envoy[912]: [2026-09-23 06:59:15.817][912][info][config] [source/server/configuration_impl.cc:164] loading stats configuration worker2 # [ 16.615780] systemd[1]: Reached target Network is Online. lb # [ 16.611544] envoy[912]: [2026-09-23 06:59:15.819][912][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 16.614335] envoy[912]: [2026-09-23 06:59:15.819][912][info][main] [source/server/server.cc:1077] starting main dispatch loop worker2 # [ 16.622842] systemd[1]: Starting scheduler-token.service... worker1 # [ 16.617475] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 16.686780] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 16.690182] systemd[1]: Finished scheduler-token.service. worker2 # [ 16.692237] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 66ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 16.698936] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 16.742842] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 16.753964] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 16.756498] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 16.767993] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 16.769589] nix-grpc-daemon[783]: E0000 00:00:1790146755.047288 783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 16.775781] nix-grpc-daemon[783]: E0000 00:00:1790146755.055266 783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 16.779651] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 16.782401] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 16.789680] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 16.799514] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 16.796081] nix-grpc-daemon[783]: E0000 00:00:1790146755.075218 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 16.802202] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 16.814038] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 16.805320] envoy[912]: [2026-09-23 06:59:16.012][912][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization worker2 # [ 16.815540] nix-grpc-daemon[783]: E0000 00:00:1790146755.999417 783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 16.813674] nix-grpc-daemon[783]: E0000 00:00:1790146755.093263 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 16.810388] envoy[912]: [2026-09-23 06:59:16.016][912][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker2 # [ 16.821154] nix-grpc-daemon[783]: E0000 00:00:1790146756.006917 783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 16.818055] nix-grpc-daemon[783]: E0000 00:00:1790146755.095541 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 16.824511] nix-grpc-daemon[783]: ts=2026-09-23T06:59:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 16.826141] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 16.829128] nix-grpc-daemon[783]: E0000 00:00:1790146755.108750 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 16.834544] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 16.839921] nix-grpc-daemon[783]: E0000 00:00:1790146756.024878 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 16.856393] nix-grpc-daemon[783]: E0000 00:00:1790146756.042087 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 16.861056] nix-grpc-daemon[783]: E0000 00:00:1790146756.044325 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 16.871713] nix-grpc-daemon[783]: E0000 00:00:1790146756.057423 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 16.877420] nix-grpc-daemon[783]: ts=2026-09-23T06:59:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 17.035491] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 17.078193] nix-grpc-daemon[783]: ts=2026-09-23T06:59:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 17.436249] nix-grpc-daemon[783]: ts=2026-09-23T06:59:15Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 17.478844] nix-grpc-daemon[783]: ts=2026-09-23T06:59:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 18.237717] nix-grpc-daemon[783]: ts=2026-09-23T06:59:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.280200] nix-grpc-daemon[783]: ts=2026-09-23T06:59:17Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 19.837870] nix-grpc-daemon[783]: ts=2026-09-23T06:59:18Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 19.880500] nix-grpc-daemon[783]: ts=2026-09-23T06:59:19Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 22.148706] rustfs-bucket-start[983]: mb s3://farm lb # [ 22.155272] systemd[1]: Finished rustfs-bucket.service. lb # [ 22.161161] systemd[1]: Starting niks3 server... lb # [ 22.272775] postgres[997]: [997] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 22.275153] postgres[997]: [997] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 22.298075] niks3-server[991]: 2026/09/23 06:59:21 OK 20241026095416_initial_model.sql (14.16ms) lb # [ 22.301759] niks3-server[991]: 2026/09/23 06:59:21 OK 20251210153512_drop_unused_gin_index.sql (3.9ms) lb # [ 22.307359] niks3-server[991]: 2026/09/23 06:59:21 OK 20251218171726_add_pins.sql (5.56ms) lb # [ 22.311821] niks3-server[991]: 2026/09/23 06:59:21 OK 20260628120000_add_object_size_and_stats.sql (4.52ms) lb # [ 22.316669] niks3-server[991]: 2026/09/23 06:59:21 OK 20260905000000_add_claims.sql (3.42ms) lb # [ 22.320885] niks3-server[991]: 2026/09/23 06:59:21 OK 20260920000000_drop_claims.sql (4.11ms) lb # [ 22.322151] niks3-server[991]: 2026/09/23 06:59:21 goose: successfully migrated database to version: 20260920000000 lb # [ 22.326314] niks3-server[991]: 2026/09/23 06:59:21 OK 1_commit_pending_closure.sql (5.43ms) lb # [ 22.329233] niks3-server[991]: 2026/09/23 06:59:21 OK 2_object_stats_trigger.sql (2.96ms) lb # [ 22.330505] niks3-server[991]: 2026/09/23 06:59:21 goose: up to current file version: 2 lb # [ 22.333662] niks3-server[991]: 2026/09/23 06:59:21 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 22.361281] niks3-server[991]: 2026/09/23 06:59:21 INFO Read proxy enabled — serving cache objects from S3 lb # [ 22.362752] niks3-server[991]: 2026/09/23 06:59:21 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 22.367750] systemd[1]: Started niks3 server. lb # [ 22.368482] niks3-server[991]: 2026/09/23 06:59:21 INFO systemd watchdog enabled interval=15s lb # [ 22.369590] systemd[1]: Reached target Multi-User System. lb # [ 22.371633] niks3-server[991]: 2026/09/23 06:59:21 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 22.374037] systemd[1]: Startup finished in 862ms (kernel) + 4.495s (initrd) + 17.015s (userspace) = 22.373s. worker1 # [ 23.038568] nix-grpc-daemon[783]: ts=2026-09-23T06:59:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 23.081238] nix-grpc-daemon[783]: ts=2026-09-23T06:59:22Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb: (finished: waiting for unit niks3.service, in 12.88 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.02 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 28.039281] nix-grpc-daemon[783]: ts=2026-09-23T06:59:26Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker2 # [ 28.081871] nix-grpc-daemon[783]: ts=2026-09-23T06:59:27Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 31.806501] envoy[912]: [2026-09-23 06:59:31.013][912][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 31.813605] envoy[912]: [2026-09-23 06:59:31.020][912][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 31.817645] envoy[912]: [2026-09-23 06:59:31.020][912][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 31.823979] envoy[912]: [2026-09-23 06:59:31.031][912][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # [ 32.381706] niks3-server[991]: 2026/09/23 06:59:31 INFO lead: acquired remote=192.168.1.3:56008 worker1 # [ 32.386345] nix-grpc-daemon[783]: ts=2026-09-23T06:59:30Z level=info event=scheduler_take_over lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 32.549889] envoy[912]: conn peer=[::1]:35412 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=7 lb: (finished: waiting for TCP port 50051 on localhost, in 9.34 seconds) worker2 # [ 32.676366] systemd[1]: Starting scheduler-token.service... worker2 # [ 32.713294] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 32.714816] systemd[1]: Finished scheduler-token.service. worker2 # [ 32.717198] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 38ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 33.040134] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.053057] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53052 worker1 # [ 33.056700] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 33.083845] nix-grpc-daemon[783]: ts=2026-09-23T06:59:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.089159] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 33.092861] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53052 worker1 # [ 33.096717] nix-grpc-daemon[783]: ts=2026-09-23T06:59:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 42.144514] nix-grpc-daemon[783]: ts=2026-09-23T06:59:40Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:53058 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 9.48 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.39 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 42.784968] nix-grpc-daemon[783]: ts=2026-09-23T06:59:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 42.795767] systemd[1]: Started Nix Daemon. worker2 # [ 42.905582] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 42.918354] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5) worker2 # [ 42.935452] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 2/5) worker2 # [ 43.104189] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 3/5) worker2 # [ 43.232924] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 585 ms (attempt 4/5) worker2 # [ 43.821513] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 43.826145] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 43.853606] nix-grpc-daemon[783]: ts=2026-09-23T06:59:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 43.861540] systemd[1]: Started Nix Daemon. worker1 # [ 43.973792] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 44.000166] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5) worker1 # [ 44.027632] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5) worker1 # [ 44.185083] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 3/5) worker1 # [ 44.341167] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 4/5) worker1 # [ 44.497665] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 44.501556] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 44.513144] nix-grpc-daemon[783]: ts=2026-09-23T06:59:42Z level=debug event=path_info_miss path=8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test error="error: path '/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 44.523644] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 44.534758] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=path_info_miss path=8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test error="error: path '/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds) client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds) worker1 # [ 44.759645] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:43700 worker1 # [ 44.773463] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:43700 worker1 # [ 44.782464] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=path_info_miss path=8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test error="error: path '/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 44.800117] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 44.803734] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40882 worker2 # [ 44.812868] nix-grpc-daemon[783]: ts=2026-09-23T06:59:43Z level=debug event=path_info_miss path=8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test error="error: path '/nix/store/8icb5zsbqv3dxvnzhf8ddq1cyhg5f86j-nixos-system-client-test' is required, but there is no substituter that can build it" lb # [ 46.607741] envoy[912]: conn peer=[2001:db8:1::1]:35768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=1780 lb # [ 46.611759] envoy[912]: conn peer=[2001:db8:1::1]:49332 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=59 worker1 # [ 46.799881] nix-grpc-daemon[783]: E0000 00:00:1790146785.078175 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client: must succeed: rm -r /var/lib/nix-grpc-store worker1 # [ 46.834169] nix-grpc-daemon[783]: E0000 00:00:1790146785.113278 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 46.842114] nix-grpc-daemon[783]: E0000 00:00:1790146785.118619 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 46.842760] nix-grpc-daemon[783]: E0000 00:00:1790146786.027558 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 46.860808] nix-grpc-daemon[783]: E0000 00:00:1790146785.140403 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 46.873136] nix-grpc-daemon[783]: E0000 00:00:1790146786.058457 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 46.883128] nix-grpc-daemon[783]: E0000 00:00:1790146786.065015 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 46.905278] nix-grpc-daemon[783]: E0000 00:00:1790146786.090958 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 46.993989] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 47.137138] nix-grpc-daemon[783]: ts=2026-09-23T06:59:46Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.57 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker2 # [ 47.277870] nix-grpc-daemon[783]: ts=2026-09-23T06:59:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker1 # [ 47.283647] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 47.312165] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=0 worker1 # [ 47.317390] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 47.362184] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 47.375591] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying 1 paths to 'lb:50051'... worker1 # [ 47.390534] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'... worker1 # [ 47.413337] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 47.438668] nix-grpc-daemon[783]: ts=2026-09-23T06:59:45Z level=info event=niks3_push_started pid=883 worker1 # [ 47.581272] nix-grpc-daemon[883]: time=2026-09-23T06:59:45.859Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 47.668887] nix-daemon[869]: accepted connection from pid 896, user nix-grpc-daemon (trusted) worker1 # [ 47.682335] nix-daemon[869]: reaped child process 899, status = succeeded lb # [ 47.687495] niks3-server[991]: 2026/09/23 06:59:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 47.707199] nix-grpc-daemon[883]: time=2026-09-23T06:59:45.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 47.708736] nix-grpc-daemon[883]: time=2026-09-23T06:59:45.988Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)" lb # [ 47.750341] niks3-server[991]: 2026/09/23 06:59:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 47.754341] niks3-server[991]: 2026/09/23 06:59:46 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls lb # [ 47.776194] niks3-server[991]: 2026/09/23 06:59:46 INFO Signed narinfos id=1 count=1 worker1 # [ 47.783195] nix-grpc-daemon[883]: time=2026-09-23T06:59:46.062Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.784574] niks3-server[991]: 2026/09/23 06:59:46 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst lb # [ 47.790463] niks3-server[991]: 2026/09/23 06:59:46 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 47.798849] niks3-server[991]: 2026/09/23 06:59:47 INFO Completed upload id=1 worker1 # [ 47.803428] nix-grpc-daemon[883]: time=2026-09-23T06:59:46.082Z level=INFO msg="Upload complete. (219ms)" lb # [ 47.803705] niks3-server[991]: 2026/09/23 06:59:47 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo worker1 # [ 47.808993] nix-grpc-daemon[783]: ts=2026-09-23T06:59:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 47.815729] nix-grpc-daemon[783]: ts=2026-09-23T06:59:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 47.827265] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 47.838341] nix-daemon[869]: reaped child process 881, status = succeeded worker1 # [ 47.842301] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051... client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 48.672630] nix-daemon[869]: reaped child process 903, status = succeeded worker1 # [ 48.760612] nix-daemon[869]: accepted connection from pid 919, user nix-grpc-daemon (trusted) worker1 # [ 48.773088] nix-daemon[869]: reaped child process 922, status = succeeded lb # [ 48.775893] niks3-server[991]: 2026/09/23 06:59:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 48.793051] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.072Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 48.796068] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.074Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)" lb # [ 48.816726] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 48.831345] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv lb # [ 48.837087] niks3-server[991]: 2026/09/23 06:59:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 48.843118] niks3-server[991]: 2026/09/23 06:59:48 INFO Signed narinfos id=2 count=1 worker1 # [ 48.847609] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.126Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.852855] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls lb # [ 48.855554] niks3-server[991]: 2026/09/23 06:59:48 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 48.860350] niks3-server[991]: 2026/09/23 06:59:48 INFO Completed upload id=2 worker1 # [ 48.864723] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.144Z level=INFO msg="Upload complete. (213ms)" lb # [ 48.864058] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo worker1 # [ 48.870666] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=1 worker1 # [ 48.885426] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=0 worker1 # [ 48.890830] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 48.899447] nix-daemon[869]: reaped child process 901, status = succeeded worker1 # [ 48.930230] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 48.943500] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 48.957059] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.236Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 48.965688] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'... worker1 # [ 48.983358] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 49.087818] nix-daemon[869]: accepted connection from pid 928, user nix-grpc-daemon (trusted) worker1 # [ 49.102047] nix-daemon[869]: reaped child process 931, status = succeeded lb # [ 49.102770] niks3-server[991]: 2026/09/23 06:59:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 49.115316] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.394Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 49.117380] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.396Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)" lb # [ 49.133190] niks3-server[991]: 2026/09/23 06:59:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign worker1 # [ 49.149477] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.428Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.145291] niks3-server[991]: 2026/09/23 06:59:48 INFO Signed narinfos id=3 count=1 lb # [ 49.150752] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls lb # [ 49.156429] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst lb # [ 49.170453] niks3-server[991]: 2026/09/23 06:59:48 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 49.178490] niks3-server[991]: 2026/09/23 06:59:48 INFO Completed upload id=3 worker1 # [ 49.184252] nix-grpc-daemon[883]: time=2026-09-23T06:59:47.463Z level=INFO msg="Upload complete. (186ms)" lb # [ 49.181145] niks3-server[991]: 2026/09/23 06:59:48 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo worker1 # [ 49.188503] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 49.193836] nix-grpc-daemon[783]: ts=2026-09-23T06:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 49.203545] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 49.212141] nix-daemon[869]: reaped child process 926, status = succeeded worker1 # [ 49.217204] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051... client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 50.654944] nix-daemon[869]: reaped child process 935, status = succeeded worker1 # [ 50.742360] nix-daemon[869]: accepted connection from pid 951, user nix-grpc-daemon (trusted) worker1 # [ 50.753813] nix-daemon[869]: reaped child process 954, status = succeeded lb # [ 50.756760] niks3-server[991]: 2026/09/23 06:59:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 50.769369] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.048Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 50.771275] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.050Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)" lb # [ 50.795448] niks3-server[991]: 2026/09/23 06:59:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 50.812515] niks3-server[991]: 2026/09/23 06:59:50 INFO Signed narinfos id=4 count=1 worker1 # [ 50.819888] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.098Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.818124] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst lb # [ 50.823613] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv lb # [ 50.828491] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls lb # [ 50.836847] niks3-server[991]: 2026/09/23 06:59:50 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 50.839577] niks3-server[991]: 2026/09/23 06:59:50 INFO Completed upload id=4 worker1 # [ 50.843778] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.123Z level=INFO msg="Upload complete. (219ms)" lb # [ 50.842288] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo worker1 # [ 50.851524] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=1 worker1 # [ 50.862436] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=0 worker1 # [ 50.866493] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 50.874960] nix-daemon[869]: reaped child process 933, status = succeeded worker1 # [ 50.906979] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 50.919203] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 50.932238] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.211Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 50.941386] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'... worker1 # [ 50.953569] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 51.051935] nix-daemon[869]: accepted connection from pid 960, user nix-grpc-daemon (trusted) worker1 # [ 51.064831] nix-daemon[869]: reaped child process 963, status = succeeded lb # [ 51.067148] niks3-server[991]: 2026/09/23 06:59:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 51.081726] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.361Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 51.083209] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.362Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)" lb # [ 51.103471] niks3-server[991]: 2026/09/23 06:59:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 51.107861] niks3-server[991]: 2026/09/23 06:59:50 INFO Signed narinfos id=5 count=1 worker1 # [ 51.115192] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.394Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.117659] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls lb # [ 51.124276] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst lb # [ 51.134644] niks3-server[991]: 2026/09/23 06:59:50 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 51.142199] niks3-server[991]: 2026/09/23 06:59:50 INFO Completed upload id=5 worker1 # [ 51.146081] nix-grpc-daemon[883]: time=2026-09-23T06:59:49.425Z level=INFO msg="Upload complete. (179ms)" lb # [ 51.145510] niks3-server[991]: 2026/09/23 06:59:50 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo worker1 # [ 51.150628] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 51.156589] nix-grpc-daemon[783]: ts=2026-09-23T06:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 51.166924] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 51.181514] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051... worker1 # [ 51.183594] nix-daemon[869]: reaped child process 958, status = succeeded client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 52.421690] nix-daemon[869]: reaped child process 967, status = succeeded worker1 # [ 52.501614] nix-daemon[869]: accepted connection from pid 983, user nix-grpc-daemon (trusted) worker1 # [ 52.516512] nix-daemon[869]: reaped child process 986, status = succeeded lb # [ 52.516644] niks3-server[991]: 2026/09/23 06:59:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 52.530224] nix-grpc-daemon[883]: time=2026-09-23T06:59:50.809Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 52.532050] nix-grpc-daemon[883]: time=2026-09-23T06:59:50.811Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)" lb # [ 52.558119] niks3-server[991]: 2026/09/23 06:59:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 52.565139] niks3-server[991]: 2026/09/23 06:59:51 INFO Signed narinfos id=6 count=1 worker1 # [ 52.572534] nix-grpc-daemon[883]: time=2026-09-23T06:59:50.851Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.583875] niks3-server[991]: 2026/09/23 06:59:51 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv lb # [ 52.589401] niks3-server[991]: 2026/09/23 06:59:51 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls lb # [ 52.592137] niks3-server[991]: 2026/09/23 06:59:51 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst lb # [ 52.594796] niks3-server[991]: 2026/09/23 06:59:51 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 52.601754] niks3-server[991]: 2026/09/23 06:59:51 INFO Completed upload id=6 worker1 # [ 52.606293] nix-grpc-daemon[883]: time=2026-09-23T06:59:50.885Z level=INFO msg="Upload complete. (216ms)" lb # [ 52.603779] niks3-server[991]: 2026/09/23 06:59:51 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo worker1 # [ 52.613281] nix-grpc-daemon[783]: ts=2026-09-23T06:59:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=1 worker1 # [ 52.628195] nix-daemon[869]: reaped child process 965, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.50 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds) client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client # copying 3 paths... client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.85 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds) worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker1 # [ 53.463628] nix-grpc-daemon[783]: ts=2026-09-23T06:59:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # [ 53.487875] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 53.509847] nix-grpc-daemon[783]: ts=2026-09-23T06:59:51Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.21 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 0.81 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker1 # [ 54.042966] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.049709] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.055819] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.333Z level=INFO msg="All 2 paths already cached" worker1 # [ 54.064823] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=0 worker1 # [ 54.071411] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 54.075873] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.086477] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.098501] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.378Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'... worker1 # [ 54.106305] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.117337] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 54.209801] nix-daemon[869]: accepted connection from pid 1015, user nix-grpc-daemon (trusted) worker1 # [ 54.222742] nix-daemon[869]: reaped child process 1018, status = succeeded lb # [ 54.226554] niks3-server[991]: 2026/09/23 06:59:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.238884] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 54.240423] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.520Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)" lb # [ 54.259753] niks3-server[991]: 2026/09/23 06:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker1 # [ 54.271880] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.551Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.268463] niks3-server[991]: 2026/09/23 06:59:53 INFO Signed narinfos id=7 count=1 lb # [ 54.275985] niks3-server[991]: 2026/09/23 06:59:53 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls lb # [ 54.281733] niks3-server[991]: 2026/09/23 06:59:53 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst lb # [ 54.294976] niks3-server[991]: 2026/09/23 06:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 54.300529] niks3-server[991]: 2026/09/23 06:59:53 INFO Completed upload id=7 worker1 # [ 54.307528] nix-grpc-daemon[883]: time=2026-09-23T06:59:52.587Z level=INFO msg="Upload complete. (180ms)" lb # [ 54.305471] niks3-server[991]: 2026/09/23 06:59:53 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo worker1 # [ 54.311730] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 54.318378] nix-grpc-daemon[783]: ts=2026-09-23T06:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 54.330112] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 54.337533] nix-daemon[869]: reaped child process 1013, status = succeeded worker1 # [ 54.344484] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051... client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 55.380941] nix-daemon[869]: reaped child process 1022, status = succeeded worker1 # [ 55.464095] nix-daemon[869]: accepted connection from pid 1038, user nix-grpc-daemon (trusted) worker1 # [ 55.475612] nix-daemon[869]: reaped child process 1041, status = succeeded lb # [ 55.479370] niks3-server[991]: 2026/09/23 06:59:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 55.492594] nix-grpc-daemon[883]: time=2026-09-23T06:59:53.772Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 55.495563] nix-grpc-daemon[883]: time=2026-09-23T06:59:53.773Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)" lb # [ 55.525393] niks3-server[991]: 2026/09/23 06:59:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker1 # [ 55.535960] nix-grpc-daemon[883]: time=2026-09-23T06:59:53.815Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.533960] niks3-server[991]: 2026/09/23 06:59:54 INFO Signed narinfos id=8 count=1 lb # [ 55.537980] niks3-server[991]: 2026/09/23 06:59:54 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv lb # [ 55.545237] niks3-server[991]: 2026/09/23 06:59:54 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls lb # [ 55.556273] niks3-server[991]: 2026/09/23 06:59:54 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst lb # [ 55.563988] niks3-server[991]: 2026/09/23 06:59:54 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 55.569151] niks3-server[991]: 2026/09/23 06:59:54 INFO Completed upload id=8 worker1 # [ 55.573740] nix-grpc-daemon[883]: time=2026-09-23T06:59:53.853Z level=INFO msg="Upload complete. (221ms)" lb # [ 55.576674] niks3-server[991]: 2026/09/23 06:59:54 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo worker1 # [ 55.581729] nix-grpc-daemon[783]: ts=2026-09-23T06:59:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=1 worker1 # [ 55.596448] nix-daemon[869]: reaped child process 1020, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.71 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.04 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2 worker1 # [ 55.991914] nix-grpc-daemon[783]: ts=2026-09-23T06:59:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 client # don't know how to build these paths: client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker2 # [ 56.006173] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 56.023501] nix-grpc-daemon[783]: ts=2026-09-23T06:59:54Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=0 worker2 # [ 56.029834] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 56.027938] nix-grpc-daemon[783]: ts=2026-09-23T06:59:54Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 56.041762] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'... worker2 # [ 56.053738] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 56.064946] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 56.083396] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=info event=niks3_push_started pid=880 worker2 # [ 56.204291] nix-grpc-daemon[880]: time=2026-09-23T06:59:55.389Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 56.295151] nix-daemon[855]: accepted connection from pid 893, user nix-grpc-daemon (trusted) worker2 # [ 56.308507] nix-daemon[855]: reaped child process 896, status = succeeded lb # [ 56.307062] niks3-server[991]: 2026/09/23 06:59:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 56.325788] nix-grpc-daemon[880]: time=2026-09-23T06:59:55.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 56.327277] nix-grpc-daemon[880]: time=2026-09-23T06:59:55.513Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)" lb # [ 56.345912] niks3-server[991]: 2026/09/23 06:59:55 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls lb # [ 56.353744] niks3-server[991]: 2026/09/23 06:59:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign worker2 # [ 56.370240] nix-grpc-daemon[880]: time=2026-09-23T06:59:55.555Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.362334] niks3-server[991]: 2026/09/23 06:59:55 INFO Signed narinfos id=9 count=1 lb # [ 56.376130] niks3-server[991]: 2026/09/23 06:59:55 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst lb # [ 56.382831] niks3-server[991]: 2026/09/23 06:59:55 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 56.392827] niks3-server[991]: 2026/09/23 06:59:55 INFO Completed upload id=9 worker2 # [ 56.400887] nix-grpc-daemon[880]: time=2026-09-23T06:59:55.586Z level=INFO msg="Upload complete. (192ms)" lb # [ 56.395598] niks3-server[991]: 2026/09/23 06:59:55 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo worker2 # [ 56.404994] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 56.411425] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 56.420499] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 56.428222] nix-daemon[855]: reaped child process 878, status = succeeded worker2 # [ 56.432488] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051... client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 56.608197] envoy[912]: conn peer=[2001:db8:1::1]:49348 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=8 lb # [ 56.611612] envoy[912]: conn peer=[2001:db8:1::1]:49354 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=8 lb # [ 56.615229] envoy[912]: conn peer=[2001:db8:1::1]:49356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9477 tx=4369 ms=5364 lb # [ 56.618713] envoy[912]: conn peer=[2001:db8:1::1]:51680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=60 lb # [ 56.623233] envoy[912]: conn peer=[2001:db8:1::1]:51690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1897 ms=1548 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 57.471772] nix-daemon[855]: reaped child process 900, status = succeeded worker2 # [ 57.550668] nix-daemon[855]: accepted connection from pid 916, user nix-grpc-daemon (trusted) worker2 # [ 57.561486] nix-daemon[855]: reaped child process 919, status = succeeded lb # [ 57.562346] niks3-server[991]: 2026/09/23 06:59:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 57.579248] nix-grpc-daemon[880]: time=2026-09-23T06:59:56.764Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 57.582942] nix-grpc-daemon[880]: time=2026-09-23T06:59:56.766Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)" lb # [ 57.603903] niks3-server[991]: 2026/09/23 06:59:56 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 57.608808] niks3-server[991]: 2026/09/23 06:59:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 57.613886] niks3-server[991]: 2026/09/23 06:59:56 INFO Signed narinfos id=10 count=1 worker2 # [ 57.626474] nix-grpc-daemon[880]: time=2026-09-23T06:59:56.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.623985] niks3-server[991]: 2026/09/23 06:59:56 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 57.633232] niks3-server[991]: 2026/09/23 06:59:56 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls lb # [ 57.638879] niks3-server[991]: 2026/09/23 06:59:56 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 57.643975] niks3-server[991]: 2026/09/23 06:59:56 INFO Completed upload id=10 worker2 # [ 57.653202] nix-grpc-daemon[880]: time=2026-09-23T06:59:56.838Z level=INFO msg="Upload complete. (213ms)" lb # [ 57.648441] niks3-server[991]: 2026/09/23 06:59:56 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo worker2 # [ 57.657996] nix-grpc-daemon[783]: ts=2026-09-23T06:59:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 worker1 # [ 57.662563] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=0 worker1 # [ 57.666645] nix-grpc-daemon[783]: ts=2026-09-23T06:59:55Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 57.673354] nix-daemon[855]: reaped child process 898, status = succeeded worker2 # [ 57.710277] nix-grpc-daemon[783]: ts=2026-09-23T06:59:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 57.722665] nix-grpc-daemon[783]: ts=2026-09-23T06:59:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 57.735664] nix-grpc-daemon[880]: time=2026-09-23T06:59:56.921Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'... worker2 # [ 57.744222] nix-grpc-daemon[783]: ts=2026-09-23T06:59:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 57.756606] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 57.856537] nix-daemon[855]: accepted connection from pid 923, user nix-grpc-daemon (trusted) worker2 # [ 57.871792] nix-daemon[855]: reaped child process 926, status = succeeded lb # [ 57.868838] niks3-server[991]: 2026/09/23 06:59:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 57.885582] nix-grpc-daemon[880]: time=2026-09-23T06:59:57.071Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 57.887036] nix-grpc-daemon[880]: time=2026-09-23T06:59:57.072Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)" lb # [ 57.903743] niks3-server[991]: 2026/09/23 06:59:57 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst lb # [ 57.911375] niks3-server[991]: 2026/09/23 06:59:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker2 # [ 57.927941] nix-grpc-daemon[880]: time=2026-09-23T06:59:57.113Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.920158] niks3-server[991]: 2026/09/23 06:59:57 INFO Signed narinfos id=11 count=1 lb # [ 57.928153] niks3-server[991]: 2026/09/23 06:59:57 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls lb # [ 57.942414] niks3-server[991]: 2026/09/23 06:59:57 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 57.945087] niks3-server[991]: 2026/09/23 06:59:57 INFO Completed upload id=11 lb # [ 57.947479] niks3-server[991]: 2026/09/23 06:59:57 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo worker2 # [ 57.954658] nix-grpc-daemon[880]: time=2026-09-23T06:59:57.140Z level=INFO msg="Upload complete. (184ms)" worker2 # [ 57.959127] nix-grpc-daemon[783]: ts=2026-09-23T06:59:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 57.965289] nix-grpc-daemon[783]: ts=2026-09-23T06:59:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 57.976175] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 57.985397] nix-daemon[855]: reaped child process 921, status = succeeded worker2 # [ 57.990410] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051... client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 59.004951] nix-daemon[855]: reaped child process 930, status = succeeded worker2 # [ 59.079066] nix-daemon[855]: accepted connection from pid 946, user nix-grpc-daemon (trusted) worker2 # [ 59.091810] nix-daemon[855]: reaped child process 949, status = succeeded lb # [ 59.092787] niks3-server[991]: 2026/09/23 06:59:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.108952] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.294Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 59.110429] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.296Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)" lb # [ 59.132172] niks3-server[991]: 2026/09/23 06:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 59.140593] niks3-server[991]: 2026/09/23 06:59:58 INFO Signed narinfos id=12 count=1 worker2 # [ 59.151514] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.337Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.147920] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst lb # [ 59.154499] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls lb # [ 59.160189] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv lb # [ 59.169493] niks3-server[991]: 2026/09/23 06:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 59.178106] niks3-server[991]: 2026/09/23 06:59:58 INFO Completed upload id=12 lb # [ 59.179789] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo worker2 # [ 59.186057] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.371Z level=INFO msg="Upload complete. (189ms)" worker2 # [ 59.192848] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 worker1 # [ 59.195825] nix-grpc-daemon[783]: ts=2026-09-23T06:59:57Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=0 worker1 # [ 59.200251] nix-grpc-daemon[783]: ts=2026-09-23T06:59:57Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.207681] nix-daemon[855]: reaped child process 928, status = succeeded worker2 # [ 59.244080] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 59.256679] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 59.269309] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.454Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 59.278459] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'... worker2 # [ 59.303175] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.403453] nix-daemon[855]: accepted connection from pid 953, user nix-grpc-daemon (trusted) worker2 # [ 59.415481] nix-daemon[855]: reaped child process 956, status = succeeded lb # [ 59.415854] niks3-server[991]: 2026/09/23 06:59:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.431470] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.617Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 59.433424] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.618Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)" lb # [ 59.448774] niks3-server[991]: 2026/09/23 06:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 59.454263] niks3-server[991]: 2026/09/23 06:59:58 INFO Signed narinfos id=13 count=1 worker2 # [ 59.465153] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.650Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.458078] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst lb # [ 59.472253] niks3-server[991]: 2026/09/23 06:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 59.478967] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls lb # [ 59.483538] niks3-server[991]: 2026/09/23 06:59:58 INFO Completed upload id=13 worker2 # [ 59.491218] nix-grpc-daemon[880]: time=2026-09-23T06:59:58.676Z level=INFO msg="Upload complete. (174ms)" lb # [ 59.486814] niks3-server[991]: 2026/09/23 06:59:58 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo worker2 # [ 59.495157] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 59.501381] nix-grpc-daemon[783]: ts=2026-09-23T06:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 59.511202] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.514739] nix-daemon[855]: reaped child process 951, status = succeeded worker2 # [ 59.519898] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051... client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 60.735708] nix-daemon[855]: reaped child process 960, status = succeeded worker2 # [ 60.811090] nix-daemon[855]: accepted connection from pid 976, user nix-grpc-daemon (trusted) worker2 # [ 60.823225] nix-daemon[855]: reaped child process 979, status = succeeded lb # [ 60.822296] niks3-server[991]: 2026/09/23 07:00:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 60.901114] nix-grpc-daemon[880]: time=2026-09-23T07:00:00.086Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 60.905936] nix-grpc-daemon[880]: time=2026-09-23T07:00:00.089Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)" lb # [ 60.928923] niks3-server[991]: 2026/09/23 07:00:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 60.937701] niks3-server[991]: 2026/09/23 07:00:00 INFO Signed narinfos id=14 count=1 worker2 # [ 60.949387] nix-grpc-daemon[880]: time=2026-09-23T07:00:00.134Z level=INFO msg="Uploading 1 narinfos" lb # [ 60.947512] niks3-server[991]: 2026/09/23 07:00:00 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst lb # [ 60.956424] niks3-server[991]: 2026/09/23 07:00:00 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv lb # [ 60.959509] niks3-server[991]: 2026/09/23 07:00:00 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 60.977469] niks3-server[991]: 2026/09/23 07:00:00 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls lb # [ 60.986833] niks3-server[991]: 2026/09/23 07:00:00 INFO Completed upload id=14 worker2 # [ 60.994609] nix-grpc-daemon[880]: time=2026-09-23T07:00:00.180Z level=INFO msg="Upload complete. (267ms)" lb # [ 60.995710] niks3-server[991]: 2026/09/23 07:00:00 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo worker2 # [ 61.000594] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 worker2 # [ 61.015775] nix-daemon[855]: reaped child process 958, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.18 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 61.257955] nix-grpc-daemon[783]: ts=2026-09-23T06:59:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:52422 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 61.302315] nix-grpc-daemon[783]: ts=2026-09-23T06:59:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:52422 worker1 # [ 61.308855] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'... worker1 # [ 61.322411] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'... worker1 # [ 61.333910] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 61.490921] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:46540 worker2 # [ 61.497431] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:46540 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.18 seconds) client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/887" client # removing stale temporary roots file "/nix/var/nix/temproots/858" client # removing stale temporary roots file "/nix/var/nix/temproots/879" client # removing stale temporary roots file "/nix/var/nix/temproots/866" client # removing stale temporary roots file "/nix/var/nix/temproots/846" client # removing stale temporary roots file "/nix/var/nix/temproots/825" client # removing stale temporary roots file "/nix/var/nix/temproots/810" client # removing stale temporary roots file "/nix/var/nix/temproots/902" client # deleting specified paths... client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 61.792938] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 61.800210] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 61.810309] nix-grpc-daemon[883]: time=2026-09-23T07:00:00.089Z level=INFO msg="All 1 paths already cached" worker1 # [ 61.816537] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 worker2 # [ 61.828800] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 61.832177] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying 3 paths... worker1 # [ 61.842339] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'... worker2 # [ 61.863391] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 61.868827] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 61.866626] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 61.872181] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 61.876915] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 61.881528] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.41 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 6.11 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 61.999667] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: fbf4cd748318489f9cf43ecc7004074d client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup worker2 # [ 62.121760] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 62.135704] nix[955]: don't know how to build these paths: client # [ 62.136617] nix[955]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker1 # [ 62.131283] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker1 # [ 62.135422] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 62.143349] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 62.155562] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 client # [ 62.177094] nix[955]: copying 1 paths to 'lb:50051'... client # [ 62.177949] nix[955]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'... worker1 # [ 62.166591] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 62.175735] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 62.276668] nix-daemon[869]: accepted connection from pid 1061, user nix-grpc-daemon (trusted) worker1 # [ 62.290161] nix-daemon[869]: reaped child process 1064, status = succeeded lb # [ 62.293677] niks3-server[991]: 2026/09/23 07:00:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 62.306208] nix-grpc-daemon[883]: time=2026-09-23T07:00:00.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 62.307780] nix-grpc-daemon[883]: time=2026-09-23T07:00:00.587Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)" lb # [ 62.326653] niks3-server[991]: 2026/09/23 07:00:01 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst lb # [ 62.332987] niks3-server[991]: 2026/09/23 07:00:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 62.340762] niks3-server[991]: 2026/09/23 07:00:01 INFO Signed narinfos id=15 count=1 worker1 # [ 62.348081] nix-grpc-daemon[883]: time=2026-09-23T07:00:00.627Z level=INFO msg="Uploading 1 narinfos" lb # [ 62.349437] niks3-server[991]: 2026/09/23 07:00:01 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls lb # [ 62.359566] niks3-server[991]: 2026/09/23 07:00:01 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 62.366669] niks3-server[991]: 2026/09/23 07:00:01 INFO Completed upload id=15 worker1 # [ 62.370619] nix-grpc-daemon[883]: time=2026-09-23T07:00:00.650Z level=INFO msg="Upload complete. (181ms)" lb # [ 62.369387] niks3-server[991]: 2026/09/23 07:00:01 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo worker1 # [ 62.374966] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 62.381761] nix-grpc-daemon[783]: ts=2026-09-23T07:00:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 62.392250] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 62.396690] nix-daemon[869]: reaped child process 1059, status = succeeded worker1 # [ 62.401660] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # [ 62.413208] nix[955]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client # [ 63.055568] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 4e3e341583f64383aabc52791df66e13 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds) worker1 # [ 63.207962] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 client # [ 63.228395] nix[971]: this derivation will be built: worker2 # [ 63.225667] nix-grpc-daemon[783]: ts=2026-09-23T07:00:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 63.236808] nix-grpc-daemon[783]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'... client # [ 63.261101] nix[971]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker1 # [ 63.254385] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker1 # [ 63.256691] nix-grpc-daemon[783]: ts=2026-09-23T07:00:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 client # [ 63.271096] nix[971]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.02 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 66.432635] nix-daemon[869]: reaped child process 1068, status = succeeded worker1 # [ 66.506213] nix-daemon[869]: accepted connection from pid 1100, user nix-grpc-daemon (trusted) worker1 # [ 66.518502] nix-daemon[869]: reaped child process 1103, status = succeeded lb # [ 66.522552] niks3-server[991]: 2026/09/23 07:00:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.536122] nix-grpc-daemon[883]: time=2026-09-23T07:00:04.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.540452] nix-grpc-daemon[883]: time=2026-09-23T07:00:04.817Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)" lb # [ 66.568692] niks3-server[991]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 66.585098] niks3-server[991]: 2026/09/23 07:00:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker1 # [ 66.593422] nix-grpc-daemon[883]: time=2026-09-23T07:00:04.872Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.589088] niks3-server[991]: 2026/09/23 07:00:05 INFO Signed narinfos id=16 count=1 lb # [ 66.598097] niks3-server[991]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv lb # [ 66.602086] niks3-server[991]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls lb # [ 66.607746] envoy[912]: conn peer=[2001:db8:1::1]:51694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=4354 ms=5018 lb # [ 66.609260] envoy[912]: conn peer=[2001:db8:1::1]:51700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=151 lb # [ 66.610765] envoy[912]: conn peer=[2001:db8:1::1]:51720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=70 lb # [ 66.612180] envoy[912]: conn peer=[2001:db8:1::1]:51708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=84 lb # [ 66.613653] envoy[912]: conn peer=[2001:db8:1::1]:51710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=78 lb # [ 66.616495] niks3-server[991]: 2026/09/23 07:00:05 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 66.624429] niks3-server[991]: 2026/09/23 07:00:05 INFO Completed upload id=16 worker1 # [ 66.628903] nix-grpc-daemon[883]: time=2026-09-23T07:00:04.908Z level=INFO msg="Upload complete. (217ms)" lb # [ 66.626724] niks3-server[991]: 2026/09/23 07:00:05 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo worker1 # [ 66.636793] nix-grpc-daemon[783]: ts=2026-09-23T07:00:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 66.644291] nix-grpc-daemon[783]: ts=2026-09-23T07:00:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 66.658380] nix-daemon[869]: reaped child process 1066, status = succeeded client # [ 66.698529] systemd[1]: dedup1.service: Deactivated successfully. client # [ 66.701828] systemd[1]: dedup1.service: Consumed 117ms CPU time over 4.701s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 66.707899] systemd[1]: dedup2.service: Deactivated successfully. client # [ 66.710606] systemd[1]: dedup2.service: Consumed 141ms CPU time over 3.654s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: two clients wanting the same drv build it once, in 5.72 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 67.831234] nix-grpc-daemon[783]: ts=2026-09-23T07:00:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 67.842163] nix-grpc-daemon[783]: ts=2026-09-23T07:00:06Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=0 worker1 # [ 67.846372] nix-grpc-daemon[783]: ts=2026-09-23T07:00:06Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 67.858597] nix-grpc-daemon[783]: ts=2026-09-23T07:00:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 67.870748] nix-grpc-daemon[783]: ts=2026-09-23T07:00:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 67.882479] nix-grpc-daemon[783]: ts=2026-09-23T07:00:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 67.893249] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 67.994539] nix-daemon[855]: accepted connection from pid 1005, user nix-grpc-daemon (trusted) worker2 # [ 68.007883] nix-daemon[855]: reaped child process 1008, status = succeeded lb # [ 68.007640] niks3-server[991]: 2026/09/23 07:00:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 68.024171] nix-grpc-daemon[880]: time=2026-09-23T07:00:07.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 68.026143] nix-grpc-daemon[880]: time=2026-09-23T07:00:07.211Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)" lb # [ 68.038832] niks3-server[991]: 2026/09/23 07:00:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 68.054293] nix-grpc-daemon[880]: time=2026-09-23T07:00:07.239Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.046923] niks3-server[991]: 2026/09/23 07:00:07 INFO Signed narinfos id=17 count=1 lb # [ 68.055603] niks3-server[991]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst lb # [ 68.068954] niks3-server[991]: 2026/09/23 07:00:07 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 68.074698] niks3-server[991]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls lb # [ 68.079610] niks3-server[991]: 2026/09/23 07:00:07 INFO Completed upload id=17 worker2 # [ 68.087500] nix-grpc-daemon[880]: time=2026-09-23T07:00:07.273Z level=INFO msg="Upload complete. (183ms)" lb # [ 68.082887] niks3-server[991]: 2026/09/23 07:00:07 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo worker2 # [ 68.092342] nix-grpc-daemon[783]: ts=2026-09-23T07:00:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 68.098292] nix-grpc-daemon[783]: ts=2026-09-23T07:00:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 68.108741] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 68.119095] nix-daemon[855]: reaped child process 1003, status = succeeded worker2 # [ 68.122206] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 74.134264] nix-grpc-daemon[783]: ts=2026-09-23T07:00:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 74.145559] nix-daemon[855]: reaped child process 1010, status = succeeded worker2 # [ 74.146614] nix-daemon[855]: reaped child process 1012, status = succeeded worker1 # [ 74.339607] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 74.350648] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=0 worker1 # [ 74.357209] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 74.362065] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 74.369792] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 74.380925] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 74.391545] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 74.487131] nix-daemon[869]: accepted connection from pid 1120, user nix-grpc-daemon (trusted) worker1 # [ 74.500430] nix-daemon[869]: reaped child process 1123, status = succeeded lb # [ 74.504408] niks3-server[991]: 2026/09/23 07:00:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 74.517140] nix-grpc-daemon[883]: time=2026-09-23T07:00:12.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 74.519063] nix-grpc-daemon[883]: time=2026-09-23T07:00:12.798Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)" lb # [ 74.539311] niks3-server[991]: 2026/09/23 07:00:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 74.549385] nix-grpc-daemon[883]: time=2026-09-23T07:00:12.828Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.545179] niks3-server[991]: 2026/09/23 07:00:13 INFO Signed narinfos id=18 count=1 lb # [ 74.558268] niks3-server[991]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls lb # [ 74.564415] niks3-server[991]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst lb # [ 74.572247] niks3-server[991]: 2026/09/23 07:00:13 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 74.579891] niks3-server[991]: 2026/09/23 07:00:13 INFO Completed upload id=18 worker1 # [ 74.586127] nix-grpc-daemon[883]: time=2026-09-23T07:00:12.865Z level=INFO msg="Upload complete. (183ms)" lb # [ 74.585282] niks3-server[991]: 2026/09/23 07:00:13 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo worker1 # [ 74.590195] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 74.595450] nix-grpc-daemon[783]: ts=2026-09-23T07:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 74.605130] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 74.617671] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 74.619822] nix-daemon[869]: reaped child process 1118, status = succeeded lb # [ 76.608793] envoy[912]: conn peer=[2001:db8:1::1]:51736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1543 ms=4535 lb # [ 76.612638] envoy[912]: conn peer=[2001:db8:1::1]:33796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1598 ms=3458 lb # [ 76.616217] envoy[912]: conn peer=[2001:db8:1::1]:33806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=935 ms=6357 worker1 # [ 80.630641] nix-grpc-daemon[783]: ts=2026-09-23T07:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 80.642433] nix-daemon[869]: reaped child process 1125, status = succeeded worker1 # [ 80.643617] nix-daemon[869]: reaped child process 1127, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.12 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share client # [ 80.852746] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 14aaa88bd1c84eeaaf2eeca1823c1a66 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share client # [ 80.999545] nix[1044]: don't know how to build these paths: client # [ 81.000516] nix[1044]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker1 # [ 80.986355] nix-grpc-daemon[783]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 80.997244] nix-grpc-daemon[783]: ts=2026-09-23T07:00:19Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker1 # [ 81.001721] nix-grpc-daemon[783]: ts=2026-09-23T07:00:19Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 81.003983] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 81.017925] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 81.035842] nix[1044]: copying 1 paths to 'lb:50051'... client # [ 81.036743] nix[1044]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'... worker2 # [ 81.028461] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 81.039370] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 81.137548] nix-daemon[855]: accepted connection from pid 1035, user nix-grpc-daemon (trusted) worker2 # [ 81.149550] nix-daemon[855]: reaped child process 1038, status = succeeded lb # [ 81.150337] niks3-server[991]: 2026/09/23 07:00:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 81.167047] nix-grpc-daemon[880]: time=2026-09-23T07:00:20.352Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 81.169023] nix-grpc-daemon[880]: time=2026-09-23T07:00:20.354Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)" lb # [ 81.189866] niks3-server[991]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst lb # [ 81.197317] niks3-server[991]: 2026/09/23 07:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 81.203683] niks3-server[991]: 2026/09/23 07:00:20 INFO Signed narinfos id=19 count=1 worker2 # [ 81.214997] nix-grpc-daemon[880]: time=2026-09-23T07:00:20.400Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.218249] niks3-server[991]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls lb # [ 81.226512] niks3-server[991]: 2026/09/23 07:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 81.232808] niks3-server[991]: 2026/09/23 07:00:20 INFO Completed upload id=19 lb # [ 81.234276] niks3-server[991]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo worker2 # [ 81.240651] nix-grpc-daemon[880]: time=2026-09-23T07:00:20.426Z level=INFO msg="Upload complete. (190ms)" worker2 # [ 81.245355] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 81.252317] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 81.262944] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 81.272440] nix-daemon[855]: reaped child process 1033, status = succeeded worker2 # [ 81.279709] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # [ 81.287764] nix[1044]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... client # [ 81.920962] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 6bbdea94b5e14775989fe63181888af4 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds) client: must succeed: sleep 2; systemctl kill -s INT share1 worker2 # [ 82.068668] nix-grpc-daemon[783]: ts=2026-09-23T07:00:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 82.084653] nix[1060]: this derivation will be built: worker1 # [ 82.075238] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 82.086363] nix-grpc-daemon[783]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'... client # [ 82.112770] nix[1060]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker1 # [ 82.107450] nix-grpc-daemon[783]: ts=2026-09-23T07:00:20Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker2 # [ 82.117848] nix-grpc-daemon[783]: ts=2026-09-23T07:00:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 82.128615] nix[1060]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... client # [ 84.082066] systemd[1]: share1.service: Sent signal SIGINT to main process 1044 (nix) on client request. client # [ 84.088845] nix[1044]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 84.095734] nix[1044]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 84.139935] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 84.141419] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 84.143365] systemd[1]: share1.service: Consumed 118ms CPU time over 3.290s wall clock time, 41.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 85.520875] nix-daemon[855]: reaped child process 1042, status = succeeded worker2 # [ 85.598246] nix-daemon[855]: accepted connection from pid 1072, user nix-grpc-daemon (trusted) worker2 # [ 85.610435] nix-daemon[855]: reaped child process 1075, status = succeeded lb # [ 85.610823] niks3-server[991]: 2026/09/23 07:00:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 85.629567] nix-grpc-daemon[880]: time=2026-09-23T07:00:24.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 85.631336] nix-grpc-daemon[880]: time=2026-09-23T07:00:24.816Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)" lb # [ 85.655407] niks3-server[991]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv lb # [ 85.664852] niks3-server[991]: 2026/09/23 07:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 85.674646] niks3-server[991]: 2026/09/23 07:00:24 INFO Signed narinfos id=20 count=1 worker2 # [ 85.682700] nix-grpc-daemon[880]: time=2026-09-23T07:00:24.868Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.679882] niks3-server[991]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 85.684539] niks3-server[991]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls lb # [ 85.694553] niks3-server[991]: 2026/09/23 07:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 85.698866] niks3-server[991]: 2026/09/23 07:00:24 INFO Completed upload id=20 lb # [ 85.701467] niks3-server[991]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo worker2 # [ 85.708127] nix-grpc-daemon[880]: time=2026-09-23T07:00:24.893Z level=INFO msg="Upload complete. (218ms)" worker2 # [ 85.714832] nix-grpc-daemon[783]: ts=2026-09-23T07:00:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker2 # [ 85.721261] nix-grpc-daemon[783]: ts=2026-09-23T07:00:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker2 # [ 85.734387] nix-daemon[855]: reaped child process 1040, status = succeeded client # [ 85.796307] systemd[1]: share2.service: Deactivated successfully. client # [ 85.800221] systemd[1]: share2.service: Consumed 120ms CPU time over 3.879s wall clock time, 35.4M memory peak, 6.5K incoming IP traffic, 6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds) (finished: subtest: the first client leaving does not fail the second, in 5.66 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 86.557351] systemd[1]: Started Nix Daemon. lb # [ 86.609680] envoy[912]: conn peer=[2001:db8:1::1]:50002 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=933 ms=6347 lb # [ 86.613531] envoy[912]: conn peer=[2001:db8:1::1]:50008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=796 ms=3111 lb # [ 86.617090] envoy[912]: conn peer=[2001:db8:1::1]:50016 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1602 tx=1598 ms=3697 client # [ 86.649943] nix-daemon[1112]: accepted connection from pid 1109, user root (trusted) worker2 # [ 86.759572] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 86.764880] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 86.772810] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=0 worker1 # [ 86.778495] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 86.785241] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 86.796492] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 86.808161] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 86.819867] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 86.919592] nix-daemon[869]: accepted connection from pid 1169, user nix-grpc-daemon (trusted) worker1 # [ 86.931690] nix-daemon[869]: reaped child process 1172, status = succeeded lb # [ 86.936812] niks3-server[991]: 2026/09/23 07:00:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 86.951322] nix-grpc-daemon[883]: time=2026-09-23T07:00:25.230Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 86.953354] nix-grpc-daemon[883]: time=2026-09-23T07:00:25.232Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)" lb # [ 86.972443] niks3-server[991]: 2026/09/23 07:00:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 86.982637] nix-grpc-daemon[883]: time=2026-09-23T07:00:25.262Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.978760] niks3-server[991]: 2026/09/23 07:00:26 INFO Signed narinfos id=21 count=1 lb # [ 86.983090] niks3-server[991]: 2026/09/23 07:00:26 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls lb # [ 86.994025] niks3-server[991]: 2026/09/23 07:00:26 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 86.996952] niks3-server[991]: 2026/09/23 07:00:26 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst lb # [ 87.002771] niks3-server[991]: 2026/09/23 07:00:26 INFO Completed upload id=21 worker1 # [ 87.007078] nix-grpc-daemon[883]: time=2026-09-23T07:00:25.286Z level=INFO msg="Upload complete. (175ms)" lb # [ 87.004387] niks3-server[991]: 2026/09/23 07:00:26 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo worker1 # [ 87.010974] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 87.015782] nix-grpc-daemon[783]: ts=2026-09-23T07:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 87.026992] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 87.030383] nix-daemon[869]: reaped child process 1167, status = succeeded worker1 # [ 87.035225] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 87.874375] nix-daemon[869]: reaped child process 1176, status = succeeded worker1 # [ 87.951073] nix-daemon[869]: accepted connection from pid 1192, user nix-grpc-daemon (trusted) worker1 # [ 87.964079] nix-daemon[869]: reaped child process 1195, status = succeeded lb # [ 87.967703] niks3-server[991]: 2026/09/23 07:00:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 87.982206] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.261Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 87.983786] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.263Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)" lb # [ 88.006682] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv lb # [ 88.022173] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls lb # [ 88.024959] niks3-server[991]: 2026/09/23 07:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 88.030863] niks3-server[991]: 2026/09/23 07:00:27 INFO Signed narinfos id=22 count=1 worker1 # [ 88.035391] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.314Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.037478] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 88.045113] niks3-server[991]: 2026/09/23 07:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 88.050289] niks3-server[991]: 2026/09/23 07:00:27 INFO Completed upload id=22 worker1 # [ 88.054427] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.333Z level=INFO msg="Upload complete. (212ms)" lb # [ 88.052154] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo worker1 # [ 88.061679] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=1 worker1 # [ 88.070380] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.080778] nix-daemon[869]: reaped child process 1174, status = succeeded worker1 # [ 88.087934] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.091885] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 88.299331] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:43700 worker2 # [ 88.306921] nix-grpc-daemon[783]: ts=2026-09-23T07:00:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 88.324592] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=0 worker1 # [ 88.329889] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 88.335541] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.346931] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.359644] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.639Z level=INFO msg="All 1 paths already cached" worker1 # [ 88.367285] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.375558] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 88.475638] nix-daemon[869]: accepted connection from pid 1201, user nix-grpc-daemon (trusted) worker1 # [ 88.488712] nix-daemon[869]: reaped child process 1204, status = succeeded lb # [ 88.491646] niks3-server[991]: 2026/09/23 07:00:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 88.504714] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.784Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 88.506144] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.785Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)" lb # [ 88.526793] niks3-server[991]: 2026/09/23 07:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker1 # [ 88.537845] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.817Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.533689] niks3-server[991]: 2026/09/23 07:00:27 INFO Signed narinfos id=23 count=1 lb # [ 88.542178] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls lb # [ 88.547662] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst lb # [ 88.558307] niks3-server[991]: 2026/09/23 07:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 88.564068] niks3-server[991]: 2026/09/23 07:00:27 INFO Completed upload id=23 worker1 # [ 88.570619] nix-grpc-daemon[883]: time=2026-09-23T07:00:26.850Z level=INFO msg="Upload complete. (181ms)" lb # [ 88.571933] niks3-server[991]: 2026/09/23 07:00:27 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo worker1 # [ 88.575285] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 88.581517] nix-grpc-daemon[783]: ts=2026-09-23T07:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 88.592259] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 88.601880] nix-daemon[869]: reaped child process 1199, status = succeeded worker1 # [ 88.606220] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 90.044536] nix-daemon[869]: reaped child process 1208, status = succeeded worker1 # [ 90.128502] nix-daemon[869]: accepted connection from pid 1224, user nix-grpc-daemon (trusted) worker1 # [ 90.142580] nix-daemon[869]: reaped child process 1227, status = succeeded lb # [ 90.144686] niks3-server[991]: 2026/09/23 07:00:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.157862] nix-grpc-daemon[883]: time=2026-09-23T07:00:28.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 90.160209] nix-grpc-daemon[883]: time=2026-09-23T07:00:28.439Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)" lb # [ 90.190179] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst lb # [ 90.195636] niks3-server[991]: 2026/09/23 07:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 90.200312] niks3-server[991]: 2026/09/23 07:00:29 INFO Signed narinfos id=24 count=1 worker1 # [ 90.208036] nix-grpc-daemon[883]: time=2026-09-23T07:00:28.487Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.206584] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv lb # [ 90.219124] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls lb # [ 90.221655] niks3-server[991]: 2026/09/23 07:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 90.227371] niks3-server[991]: 2026/09/23 07:00:29 INFO Completed upload id=24 lb # [ 90.229029] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo worker1 # [ 90.231827] nix-grpc-daemon[883]: time=2026-09-23T07:00:28.511Z level=INFO msg="Upload complete. (217ms)" worker1 # [ 90.238665] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=1 worker2 # [ 90.254491] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 90.254345] nix-daemon[869]: reaped child process 1206, status = succeeded worker2 # [ 90.264205] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'... worker2 # [ 90.274834] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'... worker2 # [ 90.302065] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 90.307462] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 90.518270] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 90.525530] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 90.532560] nix-grpc-daemon[880]: time=2026-09-23T07:00:29.717Z level=INFO msg="All 2 paths already cached" worker1 # [ 90.543582] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=0 worker1 # [ 90.549833] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 90.554959] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 90.565997] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 90.578525] nix-grpc-daemon[883]: time=2026-09-23T07:00:28.858Z level=INFO msg="All 2 paths already cached" worker1 # [ 90.586608] nix-grpc-daemon[783]: ts=2026-09-23T07:00:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 90.594914] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 90.693569] nix-daemon[869]: accepted connection from pid 1233, user nix-grpc-daemon (trusted) worker1 # [ 90.706386] nix-daemon[869]: reaped child process 1236, status = succeeded lb # [ 90.710609] niks3-server[991]: 2026/09/23 07:00:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.725270] nix-grpc-daemon[883]: time=2026-09-23T07:00:29.004Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 90.726749] nix-grpc-daemon[883]: time=2026-09-23T07:00:29.006Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)" lb # [ 90.746810] niks3-server[991]: 2026/09/23 07:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 90.754449] niks3-server[991]: 2026/09/23 07:00:29 INFO Signed narinfos id=25 count=1 worker1 # [ 90.762024] nix-grpc-daemon[883]: time=2026-09-23T07:00:29.041Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.759082] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls lb # [ 90.766705] niks3-server[991]: 2026/09/23 07:00:29 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst lb # [ 90.778893] niks3-server[991]: 2026/09/23 07:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 90.784260] niks3-server[991]: 2026/09/23 07:00:29 INFO Completed upload id=25 worker1 # [ 90.791310] nix-grpc-daemon[883]: time=2026-09-23T07:00:29.070Z level=INFO msg="Upload complete. (182ms)" worker1 # [ 90.796369] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 90.802323] nix-grpc-daemon[783]: ts=2026-09-23T07:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 90.814351] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 90.822137] nix-daemon[869]: reaped child process 1231, status = succeeded worker1 # [ 90.829736] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 92.261705] nix-daemon[869]: reaped child process 1240, status = succeeded worker1 # [ 92.348856] nix-daemon[869]: accepted connection from pid 1256, user nix-grpc-daemon (trusted) worker1 # [ 92.361292] nix-daemon[869]: reaped child process 1259, status = succeeded lb # [ 92.366336] niks3-server[991]: 2026/09/23 07:00:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 92.380708] nix-grpc-daemon[883]: time=2026-09-23T07:00:30.660Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 92.383121] nix-grpc-daemon[883]: time=2026-09-23T07:00:30.662Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)" lb # [ 92.412713] niks3-server[991]: 2026/09/23 07:00:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker1 # [ 92.423176] nix-grpc-daemon[883]: time=2026-09-23T07:00:30.702Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.419371] niks3-server[991]: 2026/09/23 07:00:31 INFO Signed narinfos id=26 count=1 lb # [ 92.428287] niks3-server[991]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv lb # [ 92.434143] niks3-server[991]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst lb # [ 92.438204] niks3-server[991]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls lb # [ 92.443777] niks3-server[991]: 2026/09/23 07:00:31 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 92.448643] niks3-server[991]: 2026/09/23 07:00:31 INFO Completed upload id=26 lb # [ 92.450450] niks3-server[991]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo worker1 # [ 92.453081] nix-grpc-daemon[883]: time=2026-09-23T07:00:30.732Z level=INFO msg="Upload complete. (213ms)" worker1 # [ 92.459960] nix-grpc-daemon[783]: ts=2026-09-23T07:00:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=1 worker1 # [ 92.473505] nix-daemon[869]: reaped child process 1238, status = succeeded worker2 # [ 92.510706] nix-grpc-daemon[783]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 92.522376] nix-grpc-daemon[783]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'... worker1 # [ 92.547255] nix-grpc-daemon[783]: ts=2026-09-23T07:00:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 92.552446] nix-grpc-daemon[783]: ts=2026-09-23T07:00:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_out=232 client # [ 92.632629] nix-daemon[1112]: reaped child process 1114, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.18 seconds) client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.24 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds) client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds) worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 93.317095] nix-daemon[1112]: accepted connection from pid 1194, user root (trusted) client # this derivation will be built: client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker2 # [ 93.433113] nix-grpc-daemon[783]: ts=2026-09-23T07:00:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 93.438321] nix-grpc-daemon[783]: ts=2026-09-23T07:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 1 paths... client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 93.447996] nix-grpc-daemon[783]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 93.458502] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 93.564497] nix-daemon[869]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 93.577475] nix-daemon[869]: reaped child process 1271, status = succeeded lb # [ 93.581205] niks3-server[991]: 2026/09/23 07:00:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.593825] nix-grpc-daemon[883]: time=2026-09-23T07:00:31.873Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 93.595866] nix-grpc-daemon[883]: time=2026-09-23T07:00:31.875Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)" lb # [ 93.612179] niks3-server[991]: 2026/09/23 07:00:32 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 93.619500] niks3-server[991]: 2026/09/23 07:00:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 93.632075] nix-grpc-daemon[883]: time=2026-09-23T07:00:31.911Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.627825] niks3-server[991]: 2026/09/23 07:00:32 INFO Signed narinfos id=27 count=1 lb # [ 93.634896] niks3-server[991]: 2026/09/23 07:00:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls lb # [ 93.646215] niks3-server[991]: 2026/09/23 07:00:32 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 93.654290] niks3-server[991]: 2026/09/23 07:00:32 INFO Completed upload id=27 worker1 # [ 93.658829] nix-grpc-daemon[883]: time=2026-09-23T07:00:31.938Z level=INFO msg="Upload complete. (190ms)" lb # [ 93.656659] niks3-server[991]: 2026/09/23 07:00:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo worker1 # [ 93.665321] nix-grpc-daemon[783]: ts=2026-09-23T07:00:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 93.680240] nix-grpc-daemon[783]: ts=2026-09-23T07:00:31Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=0 worker1 # [ 93.683842] nix-grpc-daemon[783]: ts=2026-09-23T07:00:31Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 93.688226] nix-grpc-daemon[783]: ts=2026-09-23T07:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 93.689263] nix-daemon[869]: reaped child process 1266, status = succeeded worker2 # [ 93.697872] nix-grpc-daemon[783]: ts=2026-09-23T07:00:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 93.712458] nix-grpc-daemon[783]: ts=2026-09-23T07:00:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 93.723601] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 93.826742] nix-daemon[855]: accepted connection from pid 1085, user nix-grpc-daemon (trusted) worker2 # [ 93.839054] nix-daemon[855]: reaped child process 1088, status = succeeded lb # [ 93.838995] niks3-server[991]: 2026/09/23 07:00:33 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 93.848645] niks3-server[991]: 2026/09/23 07:00:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 93.865267] nix-grpc-daemon[880]: time=2026-09-23T07:00:33.049Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 93.866627] nix-grpc-daemon[880]: time=2026-09-23T07:00:33.049Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)" worker2 # [ 93.869688] nix-grpc-daemon[880]: time=2026-09-23T07:00:33.049Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)" lb # [ 93.888549] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst lb # [ 93.894154] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls lb # [ 93.905133] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst lb # [ 93.907237] niks3-server[991]: 2026/09/23 07:00:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 93.909340] niks3-server[991]: 2026/09/23 07:00:33 INFO Signed narinfos id=29 count=2 lb # [ 93.910894] niks3-server[991]: 2026/09/23 07:00:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 93.913493] niks3-server[991]: 2026/09/23 07:00:33 INFO Signed narinfos id=28 count=1 worker2 # [ 93.922307] nix-grpc-daemon[880]: time=2026-09-23T07:00:33.107Z level=INFO msg="Uploading 3 narinfos" lb # [ 93.916676] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls lb # [ 93.931309] niks3-server[991]: 2026/09/23 07:00:33 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 93.934043] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo lb # [ 93.937515] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo lb # [ 93.939869] niks3-server[991]: 2026/09/23 07:00:33 INFO Completed upload id=28 lb # [ 93.941256] niks3-server[991]: 2026/09/23 07:00:33 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 93.943622] niks3-server[991]: 2026/09/23 07:00:33 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo lb # [ 93.946458] niks3-server[991]: 2026/09/23 07:00:33 INFO Completed upload id=29 worker2 # [ 93.954613] nix-grpc-daemon[880]: time=2026-09-23T07:00:33.140Z level=INFO msg="Upload complete. (217ms)" worker2 # [ 93.959854] nix-grpc-daemon[783]: ts=2026-09-23T07:00:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=2 nar_bytes_in=976 worker2 # [ 93.965443] nix-grpc-daemon[783]: ts=2026-09-23T07:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 93.979111] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 93.986193] nix-daemon[855]: reaped child process 1083, status = succeeded worker2 # [ 93.996063] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'... worker2 # [ 94.002788] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 95.416267] nix-daemon[855]: reaped child process 1093, status = succeeded worker2 # [ 95.491188] nix-daemon[855]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker2 # [ 95.503273] nix-daemon[855]: reaped child process 1112, status = succeeded lb # [ 95.504472] niks3-server[991]: 2026/09/23 07:00:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.520913] nix-grpc-daemon[880]: time=2026-09-23T07:00:34.706Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 95.538622] niks3-server[991]: 2026/09/23 07:00:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 95.544083] niks3-server[991]: 2026/09/23 07:00:34 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls worker2 # [ 95.559185] nix-grpc-daemon[880]: time=2026-09-23T07:00:34.744Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.552305] niks3-server[991]: 2026/09/23 07:00:34 INFO Signed narinfos id=30 count=1 lb # [ 95.558863] niks3-server[991]: 2026/09/23 07:00:34 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv lb # [ 95.570485] niks3-server[991]: 2026/09/23 07:00:34 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 95.580153] niks3-server[991]: 2026/09/23 07:00:34 INFO Completed upload id=30 worker2 # [ 95.589344] nix-grpc-daemon[880]: time=2026-09-23T07:00:34.775Z level=INFO msg="Upload complete. (177ms)" lb # [ 95.584685] niks3-server[991]: 2026/09/23 07:00:34 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo worker2 # [ 95.593976] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=1 client # copying 1 paths... worker2 # [ 95.606538] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 95.611582] nix-daemon[855]: reaped child process 1090, status = succeeded client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 95.627575] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 95.631894] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_out=120 client # [ 95.752738] nix-daemon[1112]: reaped child process 1197, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.56 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 95.859884] nix-daemon[1112]: accepted connection from pid 1216, user root (trusted) client # this derivation will be built: client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker2 # [ 95.973852] nix-grpc-daemon[783]: ts=2026-09-23T07:00:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40882 client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 95.975715] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 95.982504] nix-grpc-daemon[883]: time=2026-09-23T07:00:34.261Z level=INFO msg="All 1 paths already cached" worker1 # [ 95.996643] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=0 worker1 # [ 96.002401] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 96.040879] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 96.053852] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43700 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 96.071577] nix-grpc-daemon[783]: ts=2026-09-23T07:00:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 96.083120] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 96.098929] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5) worker1 # [ 96.129385] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5) worker1 # [ 96.283673] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 394 ms (attempt 3/5) lb # [ 96.610373] envoy[912]: conn peer=[2001:db8:1::1]:35414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3405 tx=1923 ms=1390 lb # [ 96.614168] envoy[912]: conn peer=[2001:db8:1::1]:35420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=55 lb # [ 96.617565] envoy[912]: conn peer=[2001:db8:1::1]:35422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2137 ms=2056 lb # [ 96.621118] envoy[912]: conn peer=[2001:db8:1::1]:35424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=58 lb # [ 96.624537] envoy[912]: conn peer=[2001:db8:1::1]:35428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4310 tx=2371 ms=2049 lb # [ 96.628049] envoy[912]: conn peer=[2001:db8:1::1]:35440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=19 lb # [ 96.632360] envoy[912]: conn peer=[2001:db8:1::1]:40164 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=2015 ms=2254 lb # [ 96.635879] envoy[912]: conn peer=[2001:db8:1::1]:40170 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=57 worker1 # [ 96.685255] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 175 ms (attempt 4/5) worker1 # [ 96.862569] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 96.866265] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 96.871773] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 96.985509] nix-daemon[869]: accepted connection from pid 1284, user nix-grpc-daemon (trusted) worker1 # [ 97.001972] nix-daemon[869]: reaped child process 1287, status = succeeded lb # [ 97.005773] niks3-server[991]: 2026/09/23 07:00:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.019406] nix-grpc-daemon[883]: time=2026-09-23T07:00:35.298Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 97.021371] nix-grpc-daemon[883]: time=2026-09-23T07:00:35.300Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)" lb # [ 97.041297] niks3-server[991]: 2026/09/23 07:00:36 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls lb # [ 97.045768] niks3-server[991]: 2026/09/23 07:00:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker1 # [ 97.057629] nix-grpc-daemon[883]: time=2026-09-23T07:00:35.337Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.053452] niks3-server[991]: 2026/09/23 07:00:36 INFO Signed narinfos id=31 count=1 lb # [ 97.059509] niks3-server[991]: 2026/09/23 07:00:36 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst lb # [ 97.069333] niks3-server[991]: 2026/09/23 07:00:36 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 97.075413] niks3-server[991]: 2026/09/23 07:00:36 INFO Completed upload id=31 worker1 # [ 97.079622] nix-grpc-daemon[883]: time=2026-09-23T07:00:35.359Z level=INFO msg="Upload complete. (188ms)" lb # [ 97.078339] niks3-server[991]: 2026/09/23 07:00:36 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo worker1 # [ 97.085391] nix-grpc-daemon[783]: ts=2026-09-23T07:00:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=1 paths=1 nar_bytes_in=560 worker1 # [ 97.091776] nix-grpc-daemon[783]: ts=2026-09-23T07:00:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 97.103879] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker1 # [ 97.114134] nix-daemon[869]: reaped child process 1277, status = succeeded worker1 # [ 97.116705] nix-daemon[869]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 97.743536] nix-daemon[869]: reaped child process 1291, status = succeeded worker1 # [ 97.835417] nix-daemon[869]: accepted connection from pid 1307, user nix-grpc-daemon (trusted) worker1 # [ 97.846884] nix-daemon[869]: reaped child process 1310, status = succeeded lb # [ 97.850787] niks3-server[991]: 2026/09/23 07:00:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.863104] nix-grpc-daemon[883]: time=2026-09-23T07:00:36.142Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 97.880892] niks3-server[991]: 2026/09/23 07:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker1 # [ 97.890327] nix-grpc-daemon[883]: time=2026-09-23T07:00:36.169Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.886739] niks3-server[991]: 2026/09/23 07:00:37 INFO Signed narinfos id=32 count=1 lb # [ 97.893773] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls lb # [ 97.901822] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv lb # [ 97.910885] niks3-server[991]: 2026/09/23 07:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 97.918285] niks3-server[991]: 2026/09/23 07:00:37 INFO Completed upload id=32 worker1 # [ 97.925697] nix-grpc-daemon[883]: time=2026-09-23T07:00:36.205Z level=INFO msg="Upload complete. (202ms)" lb # [ 97.922567] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo worker1 # [ 97.931136] nix-grpc-daemon[783]: ts=2026-09-23T07:00:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker2 # [ 97.946529] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 97.945567] nix-daemon[869]: reaped child process 1289, status = succeeded worker2 # [ 97.954252] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 97.977566] nix-grpc-daemon[783]: ts=2026-09-23T07:00:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 97.982599] nix-grpc-daemon[783]: ts=2026-09-23T07:00:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43700 duration_s=0 paths=1 nar_bytes_out=120 client # [ 98.062821] nix-daemon[1112]: reaped child process 1219, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.31 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 98.169303] nix-daemon[1112]: accepted connection from pid 1238, user root (trusted) client # this derivation will be built: client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker2 # [ 98.281152] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40882 client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker2 # [ 98.287935] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 98.298384] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.483Z level=INFO msg="All 1 paths already cached" worker1 # [ 98.309895] nix-grpc-daemon[783]: ts=2026-09-23T07:00:36Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=0 worker1 # [ 98.314769] nix-grpc-daemon[783]: ts=2026-09-23T07:00:36Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 98.362954] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 98.375890] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 98.389111] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.574Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 98.397295] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 98.407879] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 98.504293] nix-daemon[855]: accepted connection from pid 1117, user nix-grpc-daemon (trusted) worker2 # [ 98.516454] nix-daemon[855]: reaped child process 1120, status = succeeded lb # [ 98.516849] niks3-server[991]: 2026/09/23 07:00:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 98.533371] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 98.534896] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.720Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)" lb # [ 98.548616] niks3-server[991]: 2026/09/23 07:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 98.554414] niks3-server[991]: 2026/09/23 07:00:37 INFO Signed narinfos id=33 count=1 worker2 # [ 98.565370] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.750Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.564564] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls lb # [ 98.571870] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst lb # [ 98.581744] niks3-server[991]: 2026/09/23 07:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 98.589127] niks3-server[991]: 2026/09/23 07:00:37 INFO Completed upload id=33 worker2 # [ 98.597242] nix-grpc-daemon[880]: time=2026-09-23T07:00:37.782Z level=INFO msg="Upload complete. (178ms)" worker2 # [ 98.601542] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=560 lb # [ 98.598343] niks3-server[991]: 2026/09/23 07:00:37 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo worker2 # [ 98.607430] nix-grpc-daemon[783]: ts=2026-09-23T07:00:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 98.617747] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 98.626343] nix-daemon[855]: reaped child process 1115, status = succeeded worker2 # [ 98.631438] nix-daemon[855]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 99.863650] nix-daemon[855]: reaped child process 1124, status = succeeded worker2 # [ 99.948080] nix-daemon[855]: accepted connection from pid 1140, user nix-grpc-daemon (trusted) worker2 # [ 99.960775] nix-daemon[855]: reaped child process 1143, status = succeeded lb # [ 99.960521] niks3-server[991]: 2026/09/23 07:00:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 99.977573] nix-grpc-daemon[880]: time=2026-09-23T07:00:39.163Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 99.995595] niks3-server[991]: 2026/09/23 07:00:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 100.003921] niks3-server[991]: 2026/09/23 07:00:39 INFO Signed narinfos id=34 count=1 worker2 # [ 100.015294] nix-grpc-daemon[880]: time=2026-09-23T07:00:39.201Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.009315] niks3-server[991]: 2026/09/23 07:00:39 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv lb # [ 100.017491] niks3-server[991]: 2026/09/23 07:00:39 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls lb # [ 100.029819] niks3-server[991]: 2026/09/23 07:00:39 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 100.034679] niks3-server[991]: 2026/09/23 07:00:39 INFO Completed upload id=34 worker2 # [ 100.044405] nix-grpc-daemon[880]: time=2026-09-23T07:00:39.230Z level=INFO msg="Upload complete. (205ms)" lb # [ 100.038721] niks3-server[991]: 2026/09/23 07:00:39 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo worker2 # [ 100.050789] nix-grpc-daemon[783]: ts=2026-09-23T07:00:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 100.064402] nix-daemon[855]: reaped child process 1122, status = succeeded client # copying 1 paths... worker2 # [ 100.096226] nix-grpc-daemon[783]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 100.118233] nix-grpc-daemon[783]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 100.122056] nix-grpc-daemon[783]: ts=2026-09-23T07:00:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_out=120 client # [ 100.235327] nix-daemon[1112]: reaped child process 1241, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.17 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 7.54 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds) client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds) client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds) worker2 # [ 100.995465] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 101.862630] nix-grpc-daemon[783]: ts=2026-09-23T07:00:41Z level=info event=signal_exit worker1 # [ 103.390751] nix-grpc-daemon[783]: ts=2026-09-23T07:00:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53052 worker1 # [ 103.397268] nix-grpc-daemon[783]: ts=2026-09-23T07:00:41Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 103.397214] nix-grpc-daemon[783]: ts=2026-09-23T07:00:42Z level=info event=scheduler_restarting worker2 # [ 103.411450] nix-grpc-daemon[783]: ts=2026-09-23T07:00:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 103.431475] nix-daemon[855]: reaped child process 857, status = succeeded worker2 # [ 103.742913] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 103.745681] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 103.749551] systemd[1]: nix-grpc-daemon.service: Consumed 1.848s CPU time over 1min 27.042s wall clock time, 60.9M memory peak, 68.2K incoming IP traffic, 118.8K outgoing IP traffic. worker2 # [ 103.761356] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 103.765401] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 103.768745] systemd[1]: nix-grpc-daemon.socket: Consumed 311us CPU time over 1min 35.168s wall clock time, 256K memory peak, 87K incoming IP traffic, 78.7K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2 worker1 # [ 103.965763] nix-grpc-daemon[783]: ts=2026-09-23T07:00:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43700 worker1 # [ 103.968071] nix-grpc-daemon[783]: ts=2026-09-23T07:00:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43700 client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds) worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds) worker2 # [ 104.052843] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 104.072668] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 104.822279] nix-grpc-daemon[783]: ts=2026-09-23T07:00:43Z level=info event=signal_exit worker2 # [ 105.043070] systemd[1]: Starting scheduler-token.service... worker2 # [ 105.104553] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 105.108378] systemd[1]: Finished scheduler-token.service. worker2 # [ 105.110353] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 64ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 105.117752] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 105.195759] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 105.197875] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 105.201075] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 105.205700] nix-grpc-daemon[1167]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 105.207065] nix-grpc-daemon[1167]: E0000 00:00:1790146844.391203 1167 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 105.212760] nix-grpc-daemon[1167]: E0000 00:00:1790146844.396749 1167 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 105.216693] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 105.218320] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 105.224065] nix-grpc-daemon[1167]: E0000 00:00:1790146844.409170 1209 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 105.235759] nix-grpc-daemon[1167]: E0000 00:00:1790146844.421455 1209 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 105.240054] nix-grpc-daemon[1167]: E0000 00:00:1790146844.423718 1209 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 105.249937] nix-grpc-daemon[1167]: E0000 00:00:1790146844.435649 1209 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 105.258812] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.260404] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 105.385913] niks3-server[991]: 2026/09/23 07:00:44 INFO lead: released remote=192.168.1.3:56008 worker2 # [ 105.461689] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.465700] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.666798] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.670756] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.871795] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.875711] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.076842] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.080748] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.283203] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.286489] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 106.390485] nix-grpc-daemon[783]: ts=2026-09-23T07:00:44Z level=info event=scheduler_restarting lb # [ 106.387143] envoy[912]: [2026-09-23 07:00:45.592][912][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 106.395603] nix-grpc-daemon[783]: ts=2026-09-23T07:00:44Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 106.401818] nix-grpc-daemon[783]: ts=2026-09-23T07:00:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53052 worker1 # [ 106.405359] nix-grpc-daemon[783]: ts=2026-09-23T07:00:44Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 106.429603] nix-daemon[869]: reaped child process 871, status = succeeded worker2 # [ 106.484476] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.488492] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 106.609718] nix-grpc-daemon[783]: ts=2026-09-23T07:00:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" lb # [ 106.609940] envoy[912]: conn peer=[2001:db8:1::1]:40184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=16 lb # [ 106.613692] envoy[912]: conn peer=[2001:db8:1::1]:40174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3710 tx=2105 ms=2024 lb # [ 106.617230] envoy[912]: conn peer=[2001:db8:1::1]:40204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=54 lb # [ 106.620596] envoy[912]: conn peer=[2001:db8:1::1]:40196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3768 tx=2099 ms=1890 lb # [ 106.624268] envoy[912]: conn peer=[2001:db8:1::4]:54106 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2607 tx=851 ms=70745 lb # [ 106.627907] envoy[912]: conn peer=[2001:db8:1::1]:33916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=11 lb # [ 106.631390] envoy[912]: conn peer=[2001:db8:1::3]:33612 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2581 tx=1124 ms=74048 worker2 # [ 106.689747] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.693677] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 106.894841] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 106.898862] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.023457] nix-grpc-daemon[783]: E0000 00:00:1790146845.302811 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker2 # [ 107.100187] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.104169] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.273924] nix-grpc-daemon[783]: E0000 00:00:1790146845.553306 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 107.305950] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.309832] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 107.511233] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.515178] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.700339] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 107.703332] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 107.707400] systemd[1]: nix-grpc-daemon.service: Consumed 2.673s CPU time over 1min 31.085s wall clock time, 67.7M memory peak, 102K incoming IP traffic, 218.8K outgoing IP traffic. worker2 # [ 107.716306] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.720212] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 107.719208] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 107.723451] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 107.727458] systemd[1]: nix-grpc-daemon.socket: Consumed 312us CPU time over 1min 40.301s wall clock time, 256K memory peak, 299.1K incoming IP traffic, 108.8K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 107.921466] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 107.925516] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 108.126708] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.130714] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 108.332594] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.336409] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 108.537874] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.541879] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 108.742994] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.747064] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 108.948268] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.952242] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.153242] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.157210] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.359116] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.362869] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.564321] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.568342] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.769407] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.773339] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 109.974474] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.978433] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 110.179612] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.183542] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 110.217229] niks3-server[991]: 2026/09/23 07:00:49 INFO lead: acquired remote=192.168.1.4:54074 worker2 # [ 110.228261] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_take_over worker2 # [ 110.385394] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.398141] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 110.402397] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45692 worker2 # [ 110.406353] nix-grpc-daemon[1167]: ts=2026-09-23T07:00:49Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 127.662736] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:06Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:39374 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 19.97 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2 worker2 # [ 127.865798] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 127.877489] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 127.893829] nix-grpc-daemon[1167]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5) worker2 # [ 127.916963] nix-grpc-daemon[1167]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5) worker2 # [ 128.038751] nix-grpc-daemon[1167]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5) worker2 # [ 128.349975] nix-grpc-daemon[1167]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 4/5) worker2 # [ 128.560244] nix-grpc-daemon[1167]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 128.563635] nix-grpc-daemon[1167]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 128.586611] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 128.597540] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 128.621912] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:07Z level=info event=niks3_push_started pid=1228 worker2 # [ 128.725486] nix-grpc-daemon[1228]: time=2026-09-23T07:01:07.910Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 128.811082] nix-daemon[855]: accepted connection from pid 1241, user nix-grpc-daemon (trusted) worker2 # [ 128.822348] nix-daemon[855]: reaped child process 1244, status = succeeded lb # [ 128.823746] niks3-server[991]: 2026/09/23 07:01:08 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 128.835113] niks3-server[991]: 2026/09/23 07:01:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 128.850561] nix-grpc-daemon[1228]: time=2026-09-23T07:01:08.036Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 128.852181] nix-grpc-daemon[1228]: time=2026-09-23T07:01:08.037Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)" worker2 # [ 128.854131] nix-grpc-daemon[1228]: time=2026-09-23T07:01:08.039Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)" lb # [ 128.883830] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls lb # [ 128.898421] niks3-server[991]: 2026/09/23 07:01:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 128.905127] niks3-server[991]: 2026/09/23 07:01:08 INFO Signed narinfos id=36 count=2 lb # [ 128.909131] niks3-server[991]: 2026/09/23 07:01:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 128.911550] niks3-server[991]: 2026/09/23 07:01:08 INFO Signed narinfos id=35 count=1 worker2 # [ 128.922259] nix-grpc-daemon[1228]: time=2026-09-23T07:01:08.106Z level=INFO msg="Uploading 3 narinfos" lb # [ 128.917697] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls lb # [ 128.922561] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst lb # [ 128.936303] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 128.953118] niks3-server[991]: 2026/09/23 07:01:08 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 128.954664] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo lb # [ 128.956171] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo lb # [ 128.958709] niks3-server[991]: 2026/09/23 07:01:08 INFO Completed upload id=36 lb # [ 128.960800] niks3-server[991]: 2026/09/23 07:01:08 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 128.964056] niks3-server[991]: 2026/09/23 07:01:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo lb # [ 128.966596] niks3-server[991]: 2026/09/23 07:01:08 INFO Completed upload id=35 worker2 # [ 128.974636] nix-grpc-daemon[1228]: time=2026-09-23T07:01:08.160Z level=INFO msg="Upload complete. (245ms)" worker2 # [ 128.980597] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 128.995130] nix-daemon[855]: reaped child process 1226, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.30 seconds) worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds) worker1 # [ 129.075807] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 129.083579] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 129.176365] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 129.178544] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 129.181849] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 129.187058] nix-grpc-daemon[1335]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 129.188399] nix-grpc-daemon[1335]: E0000 00:00:1790146867.466114 1335 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 129.193677] nix-grpc-daemon[1335]: E0000 00:00:1790146867.471654 1335 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 129.197593] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 129.200073] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 129.204258] nix-grpc-daemon[1335]: E0000 00:00:1790146867.483929 1378 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 129.217066] nix-grpc-daemon[1335]: E0000 00:00:1790146867.496666 1378 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 129.221139] nix-grpc-daemon[1335]: E0000 00:00:1790146867.498994 1378 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 129.231074] nix-grpc-daemon[1335]: E0000 00:00:1790146867.510744 1378 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 129.241950] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 129.248137] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45692 worker2 # [ 129.251862] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 29.04 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb # [ 129.337335] systemd[1]: Stopping niks3 server... lb # [ 129.338338] niks3-server[991]: 2026/09/23 07:01:08 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 129.341543] niks3-server[991]: 2026/09/23 07:01:08 INFO lead: released remote=192.168.1.4:54074 worker1 # [ 129.345692] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:07Z level=info event=scheduler_lock_lost status=200 worker2 # [ 129.349801] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:08Z level=info event=scheduler_lock_lost status=200 lb # [ 129.359428] systemd[1]: niks3.service: Deactivated successfully. lb # [ 129.360343] systemd[1]: Stopped niks3 server. lb # [ 129.361713] systemd[1]: niks3.service: Consumed 804ms CPU time over 1min 47.197s wall clock time, 28.8M memory peak, 100.5K incoming IP traffic, 143.5K outgoing IP traffic. lb # [ 129.369466] systemd[1]: Starting niks3 server... lb # [ 129.426360] niks3-server[1225]: 2026/09/23 07:01:08 goose: no migrations to run. current version: 20260920000000 lb # [ 129.432201] niks3-server[1225]: 2026/09/23 07:01:08 OK 1_commit_pending_closure.sql (6.68ms) lb # [ 129.436298] niks3-server[1225]: 2026/09/23 07:01:08 OK 2_object_stats_trigger.sql (3.67ms) lb # [ 129.437451] niks3-server[1225]: 2026/09/23 07:01:08 goose: up to current file version: 2 lb # [ 129.439417] niks3-server[1225]: 2026/09/23 07:01:08 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 129.451507] niks3-server[1225]: 2026/09/23 07:01:08 INFO Read proxy enabled — serving cache objects from S3 lb # [ 129.452833] niks3-server[1225]: 2026/09/23 07:01:08 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 129.454651] niks3-server[1225]: 2026/09/23 07:01:08 INFO systemd watchdog enabled interval=15s lb # [ 129.457084] systemd[1]: Started niks3 server. lb # [ 129.458433] niks3-server[1225]: 2026/09/23 07:01:08 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 129.803062] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 129.814586] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker2 # [ 129.833858] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 129.861174] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=0 worker2 # [ 129.866168] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 129.906435] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 129.919780] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39396 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'... worker2 # [ 129.932826] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 129.942344] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 130.045214] nix-daemon[855]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 130.058025] nix-daemon[855]: reaped child process 1264, status = succeeded lb # [ 130.056630] niks3-server[1225]: 2026/09/23 07:01:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 130.076712] nix-grpc-daemon[1228]: time=2026-09-23T07:01:09.262Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 130.078913] nix-grpc-daemon[1228]: time=2026-09-23T07:01:09.264Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)" lb # [ 130.096240] niks3-server[1225]: 2026/09/23 07:01:09 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst lb # [ 130.102979] niks3-server[1225]: 2026/09/23 07:01:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 130.118131] niks3-server[1225]: 2026/09/23 07:01:09 INFO Signed narinfos id=37 count=1 worker2 # [ 130.129315] nix-grpc-daemon[1228]: time=2026-09-23T07:01:09.314Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.134990] niks3-server[1225]: 2026/09/23 07:01:09 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 130.144494] niks3-server[1225]: 2026/09/23 07:01:09 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo lb # [ 130.147393] niks3-server[1225]: 2026/09/23 07:01:09 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls lb # [ 130.153657] niks3-server[1225]: 2026/09/23 07:01:09 INFO Completed upload id=37 worker2 # [ 130.161629] nix-grpc-daemon[1228]: time=2026-09-23T07:01:09.347Z level=INFO msg="Upload complete. (206ms)" worker2 # [ 130.166876] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=0 paths=1 nar_bytes_in=536 worker2 # [ 130.172951] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 130.185108] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 130.196243] nix-daemon[855]: reaped child process 1259, status = succeeded client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051... worker2 # [ 130.201712] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 130.349327] niks3-server[1225]: 2026/09/23 07:01:09 INFO lead: acquired remote=192.168.1.4:56210 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 131.216330] nix-daemon[855]: reaped child process 1268, status = succeeded worker2 # [ 131.293700] nix-daemon[855]: accepted connection from pid 1286, user nix-grpc-daemon (trusted) worker2 # [ 131.306869] nix-daemon[855]: reaped child process 1289, status = succeeded lb # [ 131.305939] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.327275] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.512Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 131.328630] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.512Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)" lb # [ 131.352443] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls lb # [ 131.360317] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 131.366902] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker2 # [ 131.379576] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.565Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.373249] niks3-server[1225]: 2026/09/23 07:01:10 INFO Signed narinfos id=38 count=1 lb # [ 131.381521] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 131.383512] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 131.390822] niks3-server[1225]: 2026/09/23 07:01:10 INFO Completed upload id=38 worker2 # [ 131.398757] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.584Z level=INFO msg="Upload complete. (186ms)" lb # [ 131.394572] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo worker2 # [ 131.406512] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 worker2 # [ 131.418469] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=0 worker2 # [ 131.423199] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 131.433242] nix-daemon[855]: reaped child process 1266, status = succeeded worker2 # [ 131.462706] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 131.476314] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 131.491804] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.677Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'... worker2 # [ 131.500918] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 131.511179] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 131.605984] nix-daemon[855]: accepted connection from pid 1295, user nix-grpc-daemon (trusted) worker2 # [ 131.619927] nix-daemon[855]: reaped child process 1298, status = succeeded lb # [ 131.616874] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.636391] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.822Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 131.637885] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.823Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)" lb # [ 131.655176] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign lb # [ 131.661677] niks3-server[1225]: 2026/09/23 07:01:10 INFO Signed narinfos id=39 count=1 worker2 # [ 131.672247] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.858Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.673181] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst lb # [ 131.677836] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls lb # [ 131.693718] niks3-server[1225]: 2026/09/23 07:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 131.696673] niks3-server[1225]: 2026/09/23 07:01:10 INFO Completed upload id=39 worker2 # [ 131.704660] nix-grpc-daemon[1228]: time=2026-09-23T07:01:10.890Z level=INFO msg="Upload complete. (182ms)" lb # [ 131.702081] niks3-server[1225]: 2026/09/23 07:01:10 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo worker2 # [ 131.709452] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 131.715376] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 131.725755] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 131.734451] nix-daemon[855]: reaped child process 1293, status = succeeded client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051... worker2 # [ 131.740895] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 132.555123] nix-daemon[855]: reaped child process 1302, status = succeeded worker2 # [ 132.630040] nix-daemon[855]: accepted connection from pid 1318, user nix-grpc-daemon (trusted) worker2 # [ 132.642243] nix-daemon[855]: reaped child process 1321, status = succeeded lb # [ 132.640655] niks3-server[1225]: 2026/09/23 07:01:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 132.658413] nix-grpc-daemon[1228]: time=2026-09-23T07:01:11.843Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 132.659891] nix-grpc-daemon[1228]: time=2026-09-23T07:01:11.845Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)" lb # [ 132.687228] niks3-server[1225]: 2026/09/23 07:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker2 # [ 132.705128] nix-grpc-daemon[1228]: time=2026-09-23T07:01:11.890Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.697266] niks3-server[1225]: 2026/09/23 07:01:11 INFO Signed narinfos id=40 count=1 lb # [ 132.703610] niks3-server[1225]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst lb # [ 132.712252] niks3-server[1225]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv lb # [ 132.720570] niks3-server[1225]: 2026/09/23 07:01:11 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 132.728168] niks3-server[1225]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls lb # [ 132.733398] niks3-server[1225]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo lb # [ 132.737527] niks3-server[1225]: 2026/09/23 07:01:11 INFO Completed upload id=40 worker2 # [ 132.745536] nix-grpc-daemon[1228]: time=2026-09-23T07:01:11.931Z level=INFO msg="Upload complete. (199ms)" worker2 # [ 132.755106] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 worker2 # [ 132.763962] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:11Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=0 worker2 # [ 132.769403] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:11Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 132.779213] nix-daemon[855]: reaped child process 1300, status = succeeded worker2 # [ 132.808761] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 132.821774] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 132.835740] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.021Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'... worker2 # [ 132.845144] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 132.853563] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 132.949830] nix-daemon[855]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker2 # [ 132.962710] nix-daemon[855]: reaped child process 1330, status = succeeded lb # [ 132.962148] niks3-server[1225]: 2026/09/23 07:01:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 132.981480] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.167Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 132.982998] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.168Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)" lb # [ 133.002794] niks3-server[1225]: 2026/09/23 07:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 133.007499] niks3-server[1225]: 2026/09/23 07:01:12 INFO Signed narinfos id=41 count=1 worker2 # [ 133.018315] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.204Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.013793] niks3-server[1225]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst lb # [ 133.026806] niks3-server[1225]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls lb # [ 133.035143] niks3-server[1225]: 2026/09/23 07:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 133.040821] niks3-server[1225]: 2026/09/23 07:01:12 INFO Completed upload id=41 worker2 # [ 133.050330] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.235Z level=INFO msg="Upload complete. (183ms)" lb # [ 133.044826] niks3-server[1225]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo worker2 # [ 133.055413] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 133.061176] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 worker2 # [ 133.071626] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) worker2 # [ 133.081300] nix-daemon[855]: reaped child process 1325, status = succeeded worker2 # [ 133.088990] nix-daemon[855]: accepted connection from pid 1167, user nix-grpc-daemon (trusted) client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051... client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 133.703915] nix-daemon[855]: reaped child process 1334, status = succeeded worker2 # [ 133.776499] nix-daemon[855]: accepted connection from pid 1350, user nix-grpc-daemon (trusted) worker2 # [ 133.788963] nix-daemon[855]: reaped child process 1353, status = succeeded lb # [ 133.789314] niks3-server[1225]: 2026/09/23 07:01:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.807096] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.992Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 133.808543] nix-grpc-daemon[1228]: time=2026-09-23T07:01:12.994Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)" lb # [ 133.836184] niks3-server[1225]: 2026/09/23 07:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign worker2 # [ 133.855739] nix-grpc-daemon[1228]: time=2026-09-23T07:01:13.041Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.847724] niks3-server[1225]: 2026/09/23 07:01:13 INFO Signed narinfos id=42 count=1 lb # [ 133.852089] niks3-server[1225]: 2026/09/23 07:01:13 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv lb # [ 133.856342] niks3-server[1225]: 2026/09/23 07:01:13 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls lb # [ 133.864560] niks3-server[1225]: 2026/09/23 07:01:13 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst lb # [ 133.868680] niks3-server[1225]: 2026/09/23 07:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 133.873191] niks3-server[1225]: 2026/09/23 07:01:13 INFO Completed upload id=42 worker2 # [ 133.881256] nix-grpc-daemon[1228]: time=2026-09-23T07:01:13.066Z level=INFO msg="Upload complete. (186ms)" lb # [ 133.875527] niks3-server[1225]: 2026/09/23 07:01:13 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo worker2 # [ 133.888769] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39396 duration_s=0 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 worker2 # [ 133.903336] nix-daemon[855]: reaped child process 1332, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.27 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 4.78 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2 # [ 134.216495] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 134.222939] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:13Z level=info event=signal_exit lb # [ 134.360262] niks3-server[1225]: 2026/09/23 07:01:13 INFO lead: released remote=192.168.1.4:56210 worker2 # [ 135.369096] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=scheduler_restarting lb # [ 135.360567] envoy[912]: [2026-09-23 07:01:14.566][912][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 135.375691] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 135.372917] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:13Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 135.382191] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45692 worker2 # [ 135.386253] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 135.389495] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45692 worker2 # [ 135.393196] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 135.414408] nix-daemon[855]: reaped child process 1220, status = succeeded worker1 # [ 135.577376] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 135.581480] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 135.588230] nix-grpc-daemon[1167]: ts=2026-09-23T07:01:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 135.706324] nix-grpc-daemon[1167]: E0000 00:00:1790146874.891738 1213 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 135.782478] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 135.786494] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 135.987374] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 135.991415] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.192288] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.196723] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 136.286082] nix-grpc-daemon[1167]: E0000 00:00:1790146875.471510 1213 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 136.398589] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.402536] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.603352] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.607417] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 136.612309] envoy[912]: conn peer=[2001:db8:1::1]:36534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1002 tx=281 ms=1128 lb # [ 136.616231] envoy[912]: conn peer=[2001:db8:1::1]:36550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=4362 ms=4101 lb # [ 136.619715] envoy[912]: conn peer=[2001:db8:1::4]:44882 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=22440 tx=1311 ms=30325 worker1 # [ 136.808232] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.812289] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 137.013312] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.017313] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 137.217914] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.221898] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 137.423572] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.427492] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 137.590577] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 137.594584] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 137.598201] systemd[1]: nix-grpc-daemon.service: Consumed 971ms CPU time over 32.476s wall clock time, 38.7M memory peak, 50.5K incoming IP traffic, 106.2K outgoing IP traffic. worker2 # [ 137.609960] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 137.614353] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 137.618679] systemd[1]: nix-grpc-daemon.socket: Consumed 286us CPU time over 33.564s wall clock time, 256K memory peak, 218.7K incoming IP traffic, 34.6K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 137.628438] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.633465] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker1 # [ 137.831204] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.835206] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 138.035946] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 138.040067] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 138.241430] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 138.245473] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 138.447183] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 138.451118] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 138.652138] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 138.656135] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 138.856915] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 138.861106] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 139.061729] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 139.065713] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 139.266583] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 139.270576] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 139.472287] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 139.476174] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 139.676844] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 139.680927] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1 # [ 139.881792] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 139.885809] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 140.086476] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.090440] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 140.291463] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.296535] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 140.355139] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_take_over lb # [ 140.351245] niks3-server[1225]: 2026/09/23 07:01:19 INFO lead: acquired remote=192.168.1.3:47722 worker1 # [ 140.498209] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.502068] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 140.702910] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.706940] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 140.907960] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:19Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 140.915127] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:49526 worker1 # [ 140.919717] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2 worker1 # [ 142.003746] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44558 client # don't know how to build these paths: client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker1 # [ 142.012600] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 142.022931] nix-grpc-daemon[1335]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5) worker1 # [ 142.113913] nix-grpc-daemon[1335]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 2/5) worker1 # [ 142.270844] nix-grpc-daemon[1335]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 3/5) worker1 # [ 142.294500] nix-grpc-daemon[1335]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 4/5) worker1 # [ 142.532193] nix-grpc-daemon[1335]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 142.536109] nix-grpc-daemon[1335]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 142.562522] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=0 worker1 # [ 142.568666] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 142.611939] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 142.624603] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44558 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'... worker1 # [ 142.636181] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 142.647204] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 142.666194] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:20Z level=info event=niks3_push_started pid=1421 worker1 # [ 142.767844] nix-grpc-daemon[1421]: time=2026-09-23T07:01:21.047Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 142.855578] nix-daemon[869]: accepted connection from pid 1434, user nix-grpc-daemon (trusted) worker1 # [ 142.866798] nix-daemon[869]: reaped child process 1437, status = succeeded lb # [ 142.872489] niks3-server[1225]: 2026/09/23 07:01:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 142.885861] nix-grpc-daemon[1421]: time=2026-09-23T07:01:21.165Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 142.887369] nix-grpc-daemon[1421]: time=2026-09-23T07:01:21.167Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)" lb # [ 142.913719] niks3-server[1225]: 2026/09/23 07:01:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 142.919988] niks3-server[1225]: 2026/09/23 07:01:22 INFO Signed narinfos id=43 count=1 worker1 # [ 142.927732] nix-grpc-daemon[1421]: time=2026-09-23T07:01:21.207Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.933046] niks3-server[1225]: 2026/09/23 07:01:22 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls lb # [ 142.940866] niks3-server[1225]: 2026/09/23 07:01:22 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst lb # [ 142.950122] niks3-server[1225]: 2026/09/23 07:01:22 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 142.954534] niks3-server[1225]: 2026/09/23 07:01:22 INFO Completed upload id=43 worker1 # [ 142.958983] nix-grpc-daemon[1421]: time=2026-09-23T07:01:21.238Z level=INFO msg="Upload complete. (187ms)" lb # [ 142.957370] niks3-server[1225]: 2026/09/23 07:01:22 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo worker1 # [ 142.963468] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 142.969099] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 142.978268] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 142.983284] nix-daemon[869]: reaped child process 1419, status = succeeded client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051... worker1 # [ 142.988298] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 143.705685] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:21Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:53242 client # farm-a> LOG-farm-a-on-standby worker1 # [ 143.819616] nix-daemon[869]: reaped child process 1441, status = succeeded worker1 # [ 143.890689] nix-daemon[869]: accepted connection from pid 1457, user nix-grpc-daemon (trusted) worker1 # [ 143.901524] nix-daemon[869]: reaped child process 1460, status = succeeded lb # [ 143.907446] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 143.921547] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 143.923196] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.202Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)" lb # [ 143.939094] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls lb # [ 143.941469] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 143.944754] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 143.947177] niks3-server[1225]: 2026/09/23 07:01:23 INFO Signed narinfos id=44 count=1 worker1 # [ 143.952877] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.232Z level=INFO msg="Uploading 1 narinfos" lb # [ 143.950742] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv lb # [ 143.956148] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 143.960151] niks3-server[1225]: 2026/09/23 07:01:23 INFO Completed upload id=44 worker1 # [ 143.964292] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.243Z level=INFO msg="Upload complete. (169ms)" lb # [ 143.962055] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo worker1 # [ 143.966650] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 worker1 # [ 143.974485] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=0 worker1 # [ 143.976498] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 143.980802] nix-daemon[869]: reaped child process 1439, status = succeeded worker1 # [ 144.018809] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 144.029778] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 144.042132] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.321Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'... worker1 # [ 144.049525] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 144.058128] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 144.150243] nix-daemon[869]: accepted connection from pid 1466, user nix-grpc-daemon (trusted) worker1 # [ 144.162459] nix-daemon[869]: reaped child process 1469, status = succeeded lb # [ 144.167169] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 144.179708] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.459Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 144.181162] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.460Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)" lb # [ 144.203279] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 144.211762] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.491Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.207951] niks3-server[1225]: 2026/09/23 07:01:23 INFO Signed narinfos id=45 count=1 lb # [ 144.213765] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst lb # [ 144.219091] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls lb # [ 144.229325] niks3-server[1225]: 2026/09/23 07:01:23 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete worker1 # [ 144.240379] nix-grpc-daemon[1421]: time=2026-09-23T07:01:22.519Z level=INFO msg="Upload complete. (172ms)" lb # [ 144.236133] niks3-server[1225]: 2026/09/23 07:01:23 INFO Completed upload id=45 lb # [ 144.241844] niks3-server[1225]: 2026/09/23 07:01:23 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo worker1 # [ 144.243442] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 144.249284] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 144.258249] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 144.263854] nix-daemon[869]: reaped child process 1464, status = succeeded worker1 # [ 144.268655] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051... client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 145.092660] nix-daemon[869]: reaped child process 1473, status = succeeded worker1 # [ 145.166201] nix-daemon[869]: accepted connection from pid 1489, user nix-grpc-daemon (trusted) worker1 # [ 145.179347] nix-daemon[869]: reaped child process 1492, status = succeeded lb # [ 145.182896] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 145.195906] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.475Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 145.197405] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.477Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)" lb # [ 145.226268] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst lb # [ 145.231906] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv lb # [ 145.238897] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 145.243943] niks3-server[1225]: 2026/09/23 07:01:24 INFO Signed narinfos id=46 count=1 worker1 # [ 145.251883] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.531Z level=INFO msg="Uploading 1 narinfos" lb # [ 145.256661] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls lb # [ 145.259805] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 145.263700] niks3-server[1225]: 2026/09/23 07:01:24 INFO Completed upload id=46 worker1 # [ 145.269312] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.548Z level=INFO msg="Upload complete. (194ms)" lb # [ 145.266956] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo worker1 # [ 145.276993] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 worker1 # [ 145.288962] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=0 worker1 # [ 145.293211] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 145.301633] nix-daemon[869]: reaped child process 1471, status = succeeded worker1 # [ 145.333508] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 145.346219] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 145.360581] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.640Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'... worker1 # [ 145.369605] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 145.377665] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 145.475949] nix-daemon[869]: accepted connection from pid 1498, user nix-grpc-daemon (trusted) worker1 # [ 145.488376] nix-daemon[869]: reaped child process 1501, status = succeeded lb # [ 145.492946] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 145.505593] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.785Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 145.507080] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.786Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)" lb # [ 145.525689] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 145.533716] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls lb # [ 145.544643] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst lb # [ 145.555864] niks3-server[1225]: 2026/09/23 07:01:24 INFO Signed narinfos id=47 count=1 worker1 # [ 145.560138] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.839Z level=INFO msg="Uploading 1 narinfos" lb # [ 145.565959] niks3-server[1225]: 2026/09/23 07:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 145.574424] niks3-server[1225]: 2026/09/23 07:01:24 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo lb # [ 145.577898] niks3-server[1225]: 2026/09/23 07:01:24 INFO Completed upload id=47 worker1 # [ 145.582379] nix-grpc-daemon[1421]: time=2026-09-23T07:01:23.861Z level=INFO msg="Upload complete. (191ms)" worker1 # [ 145.586915] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 145.592812] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 145.602658] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 145.614411] nix-daemon[869]: reaped child process 1496, status = succeeded client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051... worker1 # [ 145.622391] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 146.839423] nix-daemon[869]: reaped child process 1505, status = succeeded worker1 # [ 146.916802] nix-daemon[869]: accepted connection from pid 1521, user nix-grpc-daemon (trusted) worker1 # [ 146.928181] nix-daemon[869]: reaped child process 1524, status = succeeded lb # [ 146.933234] niks3-server[1225]: 2026/09/23 07:01:26 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 146.948071] nix-grpc-daemon[1421]: time=2026-09-23T07:01:25.227Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 146.949822] nix-grpc-daemon[1421]: time=2026-09-23T07:01:25.229Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)" lb # [ 146.980624] niks3-server[1225]: 2026/09/23 07:01:26 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst lb # [ 146.987538] niks3-server[1225]: 2026/09/23 07:01:26 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv lb # [ 146.991734] niks3-server[1225]: 2026/09/23 07:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 146.995611] niks3-server[1225]: 2026/09/23 07:01:26 INFO Signed narinfos id=48 count=1 worker1 # [ 147.002234] nix-grpc-daemon[1421]: time=2026-09-23T07:01:25.281Z level=INFO msg="Uploading 1 narinfos" lb # [ 147.001367] niks3-server[1225]: 2026/09/23 07:01:26 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls lb # [ 147.010218] niks3-server[1225]: 2026/09/23 07:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 147.013154] niks3-server[1225]: 2026/09/23 07:01:26 INFO Completed upload id=48 worker1 # [ 147.017587] nix-grpc-daemon[1421]: time=2026-09-23T07:01:25.297Z level=INFO msg="Upload complete. (185ms)" lb # [ 147.016240] niks3-server[1225]: 2026/09/23 07:01:26 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo worker1 # [ 147.023933] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 worker1 # [ 147.037100] nix-daemon[869]: reaped child process 1503, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.18 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds) worker2 # [ 147.239780] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 147.246556] systemd[1]: Starting scheduler-token.service... worker2 # [ 147.292830] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 147.294539] systemd[1]: Finished scheduler-token.service. worker2 # [ 147.295794] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 147.300044] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 147.375632] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 147.377653] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 147.380782] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 147.385054] nix-grpc-daemon[1374]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 147.386866] nix-grpc-daemon[1374]: E0000 00:00:1790146886.570467 1374 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 147.393040] nix-grpc-daemon[1374]: E0000 00:00:1790146886.575919 1374 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.396671] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 147.399609] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 147.403082] nix-grpc-daemon[1374]: E0000 00:00:1790146886.587599 1417 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 147.414853] nix-grpc-daemon[1374]: E0000 00:00:1790146886.600555 1417 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.418129] nix-grpc-daemon[1374]: E0000 00:00:1790146886.602782 1417 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 147.428983] nix-grpc-daemon[1374]: E0000 00:00:1790146886.614693 1417 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.438831] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 147.443180] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:25Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 147.447515] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:49526 worker1 # [ 147.454267] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.57 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs client # [ 147.673780] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 303da3cf3d464e6a86b70c8a4975d326 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds) worker2 # [ 147.808203] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50456 worker2 # [ 147.819194] nix-daemon[855]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) client # [ 147.839908] nix[1365]: don't know how to build these paths: client # [ 147.840947] nix[1365]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker1 # [ 147.835108] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=0 worker1 # [ 147.840559] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 147.879945] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 147.893124] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44558 client # [ 147.915194] nix[1365]: copying 1 paths to 'lb:50051'... worker1 # [ 147.904710] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 client # [ 147.918278] nix[1365]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'... worker1 # [ 147.917740] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 148.019476] nix-daemon[869]: accepted connection from pid 1537, user nix-grpc-daemon (trusted) worker1 # [ 148.032453] nix-daemon[869]: reaped child process 1540, status = succeeded lb # [ 148.036438] niks3-server[1225]: 2026/09/23 07:01:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 148.049723] nix-grpc-daemon[1421]: time=2026-09-23T07:01:26.328Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 148.051101] nix-grpc-daemon[1421]: time=2026-09-23T07:01:26.328Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)" lb # [ 148.066776] niks3-server[1225]: 2026/09/23 07:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker1 # [ 148.078077] nix-grpc-daemon[1421]: time=2026-09-23T07:01:26.357Z level=INFO msg="Uploading 1 narinfos" lb # [ 148.073717] niks3-server[1225]: 2026/09/23 07:01:27 INFO Signed narinfos id=49 count=1 lb # [ 148.084561] niks3-server[1225]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls lb # [ 148.089112] niks3-server[1225]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst lb # [ 148.098224] niks3-server[1225]: 2026/09/23 07:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 148.104748] niks3-server[1225]: 2026/09/23 07:01:27 INFO Completed upload id=49 worker1 # [ 148.108975] nix-grpc-daemon[1421]: time=2026-09-23T07:01:26.388Z level=INFO msg="Upload complete. (180ms)" lb # [ 148.108248] niks3-server[1225]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo worker1 # [ 148.113499] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 148.118929] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 worker1 # [ 148.128122] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) worker1 # [ 148.135449] nix-daemon[869]: reaped child process 1535, status = succeeded worker1 # [ 148.140033] nix-daemon[869]: accepted connection from pid 1335, user nix-grpc-daemon (trusted) client # [ 148.151576] nix[1365]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 150.320639] systemd-journald[284]: Received client request to rotate journal, rotating. worker2 # [ 150.342462] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/60cc630c2c0b4ac48804561462d92acc. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 150.248667] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds) client: must succeed: sleep 5.0 worker1 # [ 151.202491] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:29Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.03 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 155.581515] nix-daemon[869]: reaped child process 1544, status = succeeded worker1 # [ 155.666669] nix-daemon[869]: accepted connection from pid 1585, user nix-grpc-daemon (trusted) worker1 # [ 155.679228] nix-daemon[869]: reaped child process 1588, status = succeeded lb # [ 155.681302] niks3-server[1225]: 2026/09/23 07:01:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 155.695444] nix-grpc-daemon[1421]: time=2026-09-23T07:01:33.974Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 155.696993] nix-grpc-daemon[1421]: time=2026-09-23T07:01:33.976Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)" lb # [ 155.727168] niks3-server[1225]: 2026/09/23 07:01:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 155.738320] nix-grpc-daemon[1421]: time=2026-09-23T07:01:34.017Z level=INFO msg="Uploading 1 narinfos" lb # [ 155.733920] niks3-server[1225]: 2026/09/23 07:01:34 INFO Signed narinfos id=50 count=1 lb # [ 155.742869] niks3-server[1225]: 2026/09/23 07:01:34 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv lb # [ 155.747646] niks3-server[1225]: 2026/09/23 07:01:34 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls lb # [ 155.756424] niks3-server[1225]: 2026/09/23 07:01:34 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 155.762071] niks3-server[1225]: 2026/09/23 07:01:34 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 155.767128] niks3-server[1225]: 2026/09/23 07:01:34 INFO Completed upload id=50 worker1 # [ 155.771572] nix-grpc-daemon[1421]: time=2026-09-23T07:01:34.051Z level=INFO msg="Upload complete. (217ms)" lb # [ 155.769629] niks3-server[1225]: 2026/09/23 07:01:34 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo worker1 # [ 155.780584] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44558 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 155.793474] nix-daemon[869]: reaped child process 1542, status = succeeded client # [ 155.821991] systemd[1]: rs.service: Deactivated successfully. client # [ 155.825971] systemd[1]: rs.service: Consumed 131ms CPU time over 8.148s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 156.203692] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:34Z level=info event=signal_exit lb # [ 156.353311] niks3-server[1225]: 2026/09/23 07:01:35 INFO lead: released remote=192.168.1.3:47722 lb # [ 156.613786] envoy[912]: conn peer=[2001:db8:1::1]:55008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9560 tx=4394 ms=5037 lb # [ 156.617867] envoy[912]: conn peer=[2001:db8:1::1]:51202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2943 tx=1498 ms=7987 worker1 # [ 157.359761] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=scheduler_restarting worker2 # [ 157.363594] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 157.354204] envoy[912]: [2026-09-23 07:01:36.561][912][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 157.362717] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 157.369650] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49526 worker1 # [ 157.373647] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 157.377211] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49526 worker1 # [ 157.381171] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 157.401855] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_take_over worker1 # [ 157.399257] nix-daemon[869]: reaped child process 1388, status = succeeded lb # [ 157.394992] niks3-server[1225]: 2026/09/23 07:01:36 INFO lead: acquired remote=192.168.1.4:47770 worker2 # [ 157.564485] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.568542] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 157.572504] nix-grpc-daemon[1335]: ts=2026-09-23T07:01:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 157.769230] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.772525] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 157.781595] nix-grpc-daemon[1335]: E0000 00:00:1790146896.060961 1381 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true worker2 # [ 157.973699] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.977635] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 158.178830] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 158.182768] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 158.328398] nix-grpc-daemon[1335]: E0000 00:00:1790146896.607823 1381 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 158.384658] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 158.388730] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 158.590121] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 158.593905] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 158.795155] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 158.809136] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=oidc_jwks provider=mock keys=1 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 158.813313] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52860 worker2 # [ 158.817288] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 159.309621] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:52866 worker1 # [ 159.541956] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 159.544642] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 159.547566] systemd[1]: nix-grpc-daemon.service: Consumed 1.075s CPU time over 30.458s wall clock time, 37.6M memory peak, 52.3K incoming IP traffic, 107.5K outgoing IP traffic. worker1 # [ 159.563280] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 159.654734] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 159.659186] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 159.663924] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 159.668025] nix-grpc-daemon[1595]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 159.671367] nix-grpc-daemon[1595]: E0000 00:00:1790146897.941327 1595 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 159.674394] nix-grpc-daemon[1595]: E0000 00:00:1790146897.943483 1595 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 159.676859] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 159.679212] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 159.682126] nix-grpc-daemon[1595]: E0000 00:00:1790146897.958294 1637 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 159.691587] nix-grpc-daemon[1595]: E0000 00:00:1790146897.971195 1637 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 159.696054] nix-grpc-daemon[1595]: E0000 00:00:1790146897.973398 1637 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 159.705406] nix-grpc-daemon[1595]: E0000 00:00:1790146897.985032 1637 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 159.714773] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 159.720652] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52860 worker2 # [ 159.725139] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.33 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid client # [ 159.993721] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 4cae723ad45d4bbbbc571f781ef404ba client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.05 seconds) worker2 # [ 160.120060] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50456 client # [ 160.135078] nix[1401]: don't know how to build these paths: client # [ 160.135988] nix[1401]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker2 # [ 160.134242] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:39Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker2 # [ 160.138240] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:39Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 160.188118] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50456 worker2 # [ 160.198824] nix-grpc-daemon[1374]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5) worker2 # [ 160.232214] nix-grpc-daemon[1374]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5) worker2 # [ 160.354486] nix-grpc-daemon[1374]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 224 ms (attempt 3/5) worker2 # [ 160.580649] nix-grpc-daemon[1374]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 680 ms (attempt 4/5) worker2 # [ 161.263885] nix-grpc-daemon[1374]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 161.267759] nix-grpc-daemon[1374]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 161.284906] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50456 client # [ 161.304332] nix[1401]: copying 1 paths to 'lb:50051'... client # [ 161.308298] nix[1401]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'... worker2 # [ 161.300667] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50456 worker2 # [ 161.312164] nix-daemon[855]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) worker2 # [ 161.333184] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:40Z level=info event=niks3_push_started pid=1504 worker2 # [ 161.433944] nix-grpc-daemon[1504]: time=2026-09-23T07:01:40.619Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 161.519136] nix-daemon[855]: accepted connection from pid 1516, user nix-grpc-daemon (trusted) worker2 # [ 161.530146] nix-daemon[855]: reaped child process 1519, status = succeeded lb # [ 161.531289] niks3-server[1225]: 2026/09/23 07:01:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 161.555219] nix-grpc-daemon[1504]: time=2026-09-23T07:01:40.740Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 161.556757] nix-grpc-daemon[1504]: time=2026-09-23T07:01:40.742Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)" lb # [ 161.576549] niks3-server[1225]: 2026/09/23 07:01:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 161.583150] niks3-server[1225]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls lb # [ 161.595573] niks3-server[1225]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst lb # [ 161.600153] niks3-server[1225]: 2026/09/23 07:01:40 INFO Signed narinfos id=51 count=1 worker2 # [ 161.608791] nix-grpc-daemon[1504]: time=2026-09-23T07:01:40.793Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.606058] niks3-server[1225]: 2026/09/23 07:01:40 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 161.610669] niks3-server[1225]: 2026/09/23 07:01:40 INFO Completed upload id=51 worker2 # [ 161.620110] nix-grpc-daemon[1504]: time=2026-09-23T07:01:40.805Z level=INFO msg="Upload complete. (182ms)" lb # [ 161.615825] niks3-server[1225]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo worker2 # [ 161.625175] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50456 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 161.630167] nix-grpc-daemon[1374]: ts=2026-09-23T07:01:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50456 worker2 # [ 161.639803] nix-daemon[855]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) worker2 # [ 161.643766] nix-daemon[855]: reaped child process 1502, status = succeeded worker2 # [ 161.648489] nix-daemon[855]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) client # [ 161.656475] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 163.499506] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1374 (nix-grpc-daemon) on client request. worker2 # [ 163.502561] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 163.505742] niks3-server[1225]: 2026/09/23 07:01:42 INFO lead: released remote=192.168.1.4:47770 worker2 # [ 163.520791] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 163.514718] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:41Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" lb # [ 163.510313] envoy[912]: [2026-09-23 07:01:42.716][912][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} client # [ 163.527257] nix[1401]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 163.522199] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 163.523368] systemd[1]: nix-grpc-daemon.service: Consumed 389ms CPU time over 16.217s wall clock time, 36M memory peak, 16.6K incoming IP traffic, 36.8K outgoing IP traffic. client # [ 163.531736] nix[1401]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 163.534367] systemd[1]: Starting scheduler-token.service... worker2 # [ 163.549127] nix-daemon[855]: reaped child process 1434, status = succeeded worker2 # [ 163.550635] nix-daemon[855]: reaped child process 1522, status = succeeded worker2 # [ 163.561240] nix-daemon[1524]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 163.565812] nix-daemon[855]: reaped child process 1524, status = failed with exit code 1 worker2 # [ 163.585918] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 163.587084] systemd[1]: Finished scheduler-token.service. worker2 # [ 163.588932] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 52ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 163.592086] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 163.594590] systemd[1]: Starting scheduler-token.service... worker2 # [ 163.628049] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 163.628994] systemd[1]: Finished scheduler-token.service. worker2 # [ 163.630384] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 33ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 163.635155] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 163.709967] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 163.711928] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 163.715091] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 163.719463] nix-grpc-daemon[1567]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 163.715248] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.722099] nix-grpc-daemon[1567]: E0000 00:00:1790146902.905076 1567 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 163.727616] nix-grpc-daemon[1567]: E0000 00:00:1790146902.910801 1567 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 163.732093] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 163.735213] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 163.740555] nix-grpc-daemon[1567]: E0000 00:00:1790146902.926314 1609 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 163.734487] niks3-server[1225]: 2026/09/23 07:01:42 INFO lead: acquired remote=192.168.1.4:49846 worker2 # [ 163.745125] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=scheduler_take_over worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.27 seconds) client: must succeed: sleep 5.0 worker2 # [ 163.749791] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:44096 worker2 # [ 163.752120] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 163.760412] nix-grpc-daemon[1567]: E0000 00:00:1790146902.946154 1609 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 163.764053] nix-grpc-daemon[1567]: E0000 00:00:1790146902.948408 1609 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 163.774419] nix-grpc-daemon[1567]: E0000 00:00:1790146902.960131 1609 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 163.783384] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.785051] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 163.986373] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 163.990333] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 164.029925] nix[1401]: scheduler at lb:50051: no healthy upstream, reconnecting worker2 # [ 164.191520] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 164.195501] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 164.396704] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 164.400619] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 164.601919] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 164.606195] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 164.807606] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 164.818144] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:44Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 164.821654] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:44096 worker2 # [ 164.825530] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 165.031262] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:44Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker2 # [ 165.035857] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:44Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 165.080672] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43832 worker1 # [ 165.091855] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) worker1 # [ 165.108114] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5) worker1 # [ 165.149441] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 2/5) worker1 # [ 165.341668] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 3/5) worker1 # [ 165.382330] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 444 ms (attempt 4/5) worker1 # [ 165.828913] nix-grpc-daemon[1595]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 165.832283] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 165.847587] nix-grpc-daemon[1595]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'... worker1 # [ 165.871157] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) worker1 # [ 165.884755] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) client # [ 165.895961] nix[1401]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... worker2 # [ 165.912618] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:45Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:44116 lb # [ 166.614308] envoy[912]: conn peer=[2001:db8:1::3]:38378 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=6804 tx=1494 ms=28326 lb # [ 166.618689] envoy[912]: conn peer=[2001:db8:1::4]:53868 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7013 tx=556 ms=16084 client: (finished: must succeed: sleep 5.0, in 5.04 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 169.110575] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:47Z level=info event=niks3_push_started pid=1702 worker1 # [ 169.145360] nix-daemon[869]: reaped child process 1677, status = succeeded worker1 # [ 169.231735] nix-grpc-daemon[1702]: time=2026-09-23T07:01:47.511Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 169.318273] nix-daemon[869]: accepted connection from pid 1715, user nix-grpc-daemon (trusted) worker1 # [ 169.330861] nix-daemon[869]: reaped child process 1718, status = succeeded lb # [ 169.336448] niks3-server[1225]: 2026/09/23 07:01:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.349721] nix-grpc-daemon[1702]: time=2026-09-23T07:01:47.629Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 169.351241] nix-grpc-daemon[1702]: time=2026-09-23T07:01:47.630Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)" lb # [ 169.385516] niks3-server[1225]: 2026/09/23 07:01:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 169.390114] niks3-server[1225]: 2026/09/23 07:01:48 INFO Signed narinfos id=52 count=1 worker1 # [ 169.397144] nix-grpc-daemon[1702]: time=2026-09-23T07:01:47.676Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.399662] niks3-server[1225]: 2026/09/23 07:01:48 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls lb # [ 169.404115] niks3-server[1225]: 2026/09/23 07:01:48 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv lb # [ 169.414559] niks3-server[1225]: 2026/09/23 07:01:48 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 169.418316] niks3-server[1225]: 2026/09/23 07:01:48 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 169.422063] niks3-server[1225]: 2026/09/23 07:01:48 INFO Completed upload id=52 worker1 # [ 169.426618] nix-grpc-daemon[1702]: time=2026-09-23T07:01:47.706Z level=INFO msg="Upload complete. (191ms)" lb # [ 169.424819] niks3-server[1225]: 2026/09/23 07:01:48 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo worker1 # [ 169.431479] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43832 duration_s=4 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 169.443843] nix-daemon[869]: reaped child process 1675, status = succeeded client # [ 169.477603] systemd[1]: mid.service: Deactivated successfully. client # [ 169.481341] systemd[1]: mid.service: Consumed 149ms CPU time over 9.486s wall clock time, 41.4M memory peak, 9K incoming IP traffic, 10.8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.13 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr client # [ 170.102948] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: e0cf5ac9c89148f0995b6dde377acaa6 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds) worker2 # [ 170.226764] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 170.237798] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # [ 170.258876] nix[1437]: don't know how to build these paths: client # [ 170.259842] nix[1437]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker2 # [ 170.256699] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=0 worker2 # [ 170.260786] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 170.305138] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 170.316340] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5) worker2 # [ 170.345546] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5) worker2 # [ 170.508761] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 3/5) worker2 # [ 170.551527] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 4/5) worker2 # [ 170.682538] nix-grpc-daemon[1567]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 170.685985] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 170.703947] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 170.723337] nix[1437]: copying 1 paths to 'lb:50051'... worker2 # [ 170.716169] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 170.726127] nix[1437]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'... worker2 # [ 170.728455] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 170.749460] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:49Z level=info event=niks3_push_started pid=1644 worker2 # [ 170.849726] nix-grpc-daemon[1644]: time=2026-09-23T07:01:50.035Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 170.934780] nix-daemon[855]: accepted connection from pid 1657, user nix-grpc-daemon (trusted) worker2 # [ 170.945905] nix-daemon[855]: reaped child process 1660, status = succeeded lb # [ 170.947584] niks3-server[1225]: 2026/09/23 07:01:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.963680] nix-grpc-daemon[1644]: time=2026-09-23T07:01:50.149Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 170.965202] nix-grpc-daemon[1644]: time=2026-09-23T07:01:50.150Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)" lb # [ 170.986139] niks3-server[1225]: 2026/09/23 07:01:50 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst lb # [ 170.992860] niks3-server[1225]: 2026/09/23 07:01:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 170.999815] niks3-server[1225]: 2026/09/23 07:01:50 INFO Signed narinfos id=53 count=1 worker2 # [ 171.010621] nix-grpc-daemon[1644]: time=2026-09-23T07:01:50.196Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.009381] niks3-server[1225]: 2026/09/23 07:01:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls lb # [ 171.018882] niks3-server[1225]: 2026/09/23 07:01:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 171.024977] niks3-server[1225]: 2026/09/23 07:01:50 INFO Completed upload id=53 worker2 # [ 171.033074] nix-grpc-daemon[1644]: time=2026-09-23T07:01:50.218Z level=INFO msg="Upload complete. (180ms)" lb # [ 171.028480] niks3-server[1225]: 2026/09/23 07:01:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo worker2 # [ 171.035552] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 171.039158] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 171.047252] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 171.052638] nix-daemon[855]: reaped child process 1642, status = succeeded worker2 # [ 171.056689] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # [ 171.064956] nix[1437]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 172.426239] systemd[1]: intr.service: Sent signal SIGINT to main process 1437 (nix) on client request. worker2 # [ 172.420365] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:51Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv client # [ 172.434797] nix[1437]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds) client # [ 172.440578] nix[1437]: error: interrupted by the user client # [ 172.465903] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 172.467192] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 172.468737] systemd[1]: intr.service: Consumed 117ms CPU time over 2.362s wall clock time, 41.4M memory peak, 6.3K incoming IP traffic, 8K outgoing IP traffic. worker2 # [ 172.472934] nix-daemon[1664]: unexpected Nix daemon error: worker2 # [ 172.474342] nix-daemon[855]: reaped child process 1662, status = succeeded worker2 # [ 172.476113] nix-daemon[1664]: error: interrupted by the user worker2 # [ 172.480419] nix-daemon[855]: reaped child process 1664, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 2.42 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw client # [ 172.541473] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: e5b15a62358646bd9f640082f440db99 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.04 seconds) worker2 # [ 172.656138] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 172.669975] nix[1458]: don't know how to build these paths: client # [ 172.670984] nix[1458]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker2 # [ 172.668638] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:51Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=0 worker2 # [ 172.673265] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:51Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.710776] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43832 worker1 # [ 172.725102] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43832 client # [ 172.747262] nix[1458]: copying 1 paths to 'lb:50051'... worker1 # [ 172.736782] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43832 client # [ 172.749853] nix[1458]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'... worker1 # [ 172.749604] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) worker1 # [ 172.853468] nix-daemon[869]: accepted connection from pid 1742, user nix-grpc-daemon (trusted) worker1 # [ 172.866485] nix-daemon[869]: reaped child process 1745, status = succeeded lb # [ 172.870569] niks3-server[1225]: 2026/09/23 07:01:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.883769] nix-grpc-daemon[1702]: time=2026-09-23T07:01:51.163Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 172.886122] nix-grpc-daemon[1702]: time=2026-09-23T07:01:51.164Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)" lb # [ 172.902556] niks3-server[1225]: 2026/09/23 07:01:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 172.915803] nix-grpc-daemon[1702]: time=2026-09-23T07:01:51.195Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.910984] niks3-server[1225]: 2026/09/23 07:01:52 INFO Signed narinfos id=54 count=1 lb # [ 172.917288] niks3-server[1225]: 2026/09/23 07:01:52 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls lb # [ 172.921744] niks3-server[1225]: 2026/09/23 07:01:52 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst lb # [ 172.931273] niks3-server[1225]: 2026/09/23 07:01:52 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 172.935088] niks3-server[1225]: 2026/09/23 07:01:52 INFO Completed upload id=54 worker1 # [ 172.939367] nix-grpc-daemon[1702]: time=2026-09-23T07:01:51.219Z level=INFO msg="Upload complete. (178ms)" lb # [ 172.937944] niks3-server[1225]: 2026/09/23 07:01:52 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo worker1 # [ 172.943162] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43832 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 172.947933] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43832 worker1 # [ 172.958162] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) worker1 # [ 172.966035] nix-daemon[869]: accepted connection from pid 1595, user nix-grpc-daemon (trusted) client # [ 172.977477] nix[1458]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051... worker1 # [ 172.968536] nix-daemon[869]: reaped child process 1740, status = succeeded worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 174.931545] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 174.962660] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker2 # [ 175.157790] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 175.163472] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 175.186712] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=0 worker2 # [ 175.191811] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 175.237750] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 175.250956] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 client # copying 1 paths to 'lb:50051'... worker2 # [ 175.264267] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'... worker2 # [ 175.285178] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 175.385314] nix-daemon[855]: accepted connection from pid 1715, user nix-grpc-daemon (trusted) worker2 # [ 175.397452] nix-daemon[855]: reaped child process 1718, status = succeeded lb # [ 175.400082] niks3-server[1225]: 2026/09/23 07:01:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 175.415272] nix-grpc-daemon[1644]: time=2026-09-23T07:01:54.600Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 175.417564] nix-grpc-daemon[1644]: time=2026-09-23T07:01:54.602Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)" worker2 # [ 175.438789] nix-grpc-daemon[1644]: time=2026-09-23T07:01:54.624Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.430756] niks3-server[1225]: 2026/09/23 07:01:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 175.434692] niks3-server[1225]: 2026/09/23 07:01:54 INFO Signed narinfos id=55 count=1 lb # [ 175.444914] niks3-server[1225]: 2026/09/23 07:01:54 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst lb # [ 175.457822] niks3-server[1225]: 2026/09/23 07:01:54 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls lb # [ 175.463234] niks3-server[1225]: 2026/09/23 07:01:54 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 175.470466] niks3-server[1225]: 2026/09/23 07:01:54 INFO Completed upload id=55 lb # [ 175.473391] niks3-server[1225]: 2026/09/23 07:01:54 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo worker2 # [ 175.480372] nix-grpc-daemon[1644]: time=2026-09-23T07:01:54.666Z level=INFO msg="Upload complete. (183ms)" worker2 # [ 175.484478] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 175.489273] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 175.498727] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 175.506849] nix-daemon[855]: reaped child process 1713, status = succeeded worker2 # [ 175.511786] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051... client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker1 # [ 175.672201] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:53Z level=info event=draining client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 176.354180] nix-daemon[855]: reaped child process 1722, status = succeeded worker2 # [ 176.429537] nix-daemon[855]: accepted connection from pid 1738, user nix-grpc-daemon (trusted) worker2 # [ 176.441162] nix-daemon[855]: reaped child process 1741, status = succeeded lb # [ 176.440494] niks3-server[1225]: 2026/09/23 07:01:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 176.457406] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.642Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 176.458915] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.644Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)" lb # [ 176.475164] niks3-server[1225]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv lb # [ 176.491190] niks3-server[1225]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls lb # [ 176.498861] niks3-server[1225]: 2026/09/23 07:01:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 176.504780] niks3-server[1225]: 2026/09/23 07:01:55 INFO Signed narinfos id=56 count=1 worker2 # [ 176.513428] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.698Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.508357] niks3-server[1225]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 176.518582] niks3-server[1225]: 2026/09/23 07:01:55 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 176.522758] niks3-server[1225]: 2026/09/23 07:01:55 INFO Completed upload id=56 worker2 # [ 176.530465] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.716Z level=INFO msg="Upload complete. (211ms)" lb # [ 176.525262] niks3-server[1225]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo worker2 # [ 176.536808] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=1 worker2 # [ 176.549642] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=0 worker2 # [ 176.553166] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 176.561535] nix-daemon[855]: reaped child process 1720, status = succeeded worker2 # [ 176.594226] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 176.607493] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 176.621761] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.807Z level=INFO msg="All 1 paths already cached" lb # [ 176.614965] envoy[912]: conn peer=[2001:db8:1::1]:41910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4106 tx=2034 ms=9328 client # copying 1 paths to 'lb:50051'... lb # [ 176.619508] envoy[912]: conn peer=[2001:db8:1::1]:33246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=812 ms=2224 worker2 # [ 176.636833] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'... worker2 # [ 176.656230] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 176.757932] nix-daemon[855]: accepted connection from pid 1747, user nix-grpc-daemon (trusted) worker2 # [ 176.770231] nix-daemon[855]: reaped child process 1750, status = succeeded lb # [ 176.770386] niks3-server[1225]: 2026/09/23 07:01:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 176.788082] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.973Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 176.789597] nix-grpc-daemon[1644]: time=2026-09-23T07:01:55.975Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)" lb # [ 176.804898] niks3-server[1225]: 2026/09/23 07:01:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker2 # [ 176.818854] nix-grpc-daemon[1644]: time=2026-09-23T07:01:56.004Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.811223] niks3-server[1225]: 2026/09/23 07:01:56 INFO Signed narinfos id=57 count=1 lb # [ 176.815661] niks3-server[1225]: 2026/09/23 07:01:56 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst lb # [ 176.825192] niks3-server[1225]: 2026/09/23 07:01:56 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls lb # [ 176.834882] niks3-server[1225]: 2026/09/23 07:01:56 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 176.841940] niks3-server[1225]: 2026/09/23 07:01:56 INFO Completed upload id=57 worker2 # [ 176.849781] nix-grpc-daemon[1644]: time=2026-09-23T07:01:56.035Z level=INFO msg="Upload complete. (181ms)" lb # [ 176.847181] niks3-server[1225]: 2026/09/23 07:01:56 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo worker2 # [ 176.853989] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 176.860576] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 176.869936] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 176.880224] nix-daemon[855]: reaped child process 1745, status = succeeded worker2 # [ 176.883145] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051... client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 178.116921] nix-daemon[855]: reaped child process 1754, status = succeeded worker2 # [ 178.196341] nix-daemon[855]: accepted connection from pid 1770, user nix-grpc-daemon (trusted) worker2 # [ 178.207269] nix-daemon[855]: reaped child process 1773, status = succeeded lb # [ 178.207982] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 178.227713] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.413Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 178.229171] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.414Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)" lb # [ 178.253184] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 178.259530] niks3-server[1225]: 2026/09/23 07:01:57 INFO Signed narinfos id=58 count=1 worker2 # [ 178.272074] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.457Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.271552] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls lb # [ 178.277039] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv lb # [ 178.285744] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 178.289863] niks3-server[1225]: 2026/09/23 07:01:57 INFO Completed upload id=58 worker2 # [ 178.297661] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.483Z level=INFO msg="Upload complete. (205ms)" lb # [ 178.292734] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo lb # [ 178.295248] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst worker2 # [ 178.304471] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=1 worker2 # [ 178.316505] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=0 worker2 # [ 178.320427] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 178.329598] nix-daemon[855]: reaped child process 1752, status = succeeded worker2 # [ 178.361232] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 178.374296] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 178.388233] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.573Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'... worker2 # [ 178.397843] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 178.405982] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 178.500936] nix-daemon[855]: accepted connection from pid 1779, user nix-grpc-daemon (trusted) worker2 # [ 178.513864] nix-daemon[855]: reaped child process 1782, status = succeeded lb # [ 178.512960] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 178.529141] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.714Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 178.530604] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.716Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)" lb # [ 178.547774] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst lb # [ 178.558154] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 178.563940] niks3-server[1225]: 2026/09/23 07:01:57 INFO Signed narinfos id=59 count=1 worker2 # [ 178.575028] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.760Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.569743] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls lb # [ 178.583440] niks3-server[1225]: 2026/09/23 07:01:57 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 178.589940] niks3-server[1225]: 2026/09/23 07:01:57 INFO Completed upload id=59 worker2 # [ 178.597704] nix-grpc-daemon[1644]: time=2026-09-23T07:01:57.783Z level=INFO msg="Upload complete. (178ms)" lb # [ 178.593126] niks3-server[1225]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo worker2 # [ 178.602402] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 178.609336] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 178.620085] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 178.629287] nix-daemon[855]: reaped child process 1777, status = succeeded client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051... worker2 # [ 178.635593] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 179.872752] nix-daemon[855]: reaped child process 1786, status = succeeded worker2 # [ 179.948144] nix-daemon[855]: accepted connection from pid 1802, user nix-grpc-daemon (trusted) worker2 # [ 179.960760] nix-daemon[855]: reaped child process 1805, status = succeeded lb # [ 179.961276] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 179.978069] nix-grpc-daemon[1644]: time=2026-09-23T07:01:59.163Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 179.979522] nix-grpc-daemon[1644]: time=2026-09-23T07:01:59.165Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)" lb # [ 180.002443] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv lb # [ 180.016065] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker2 # [ 180.033083] nix-grpc-daemon[1644]: time=2026-09-23T07:01:59.218Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.025458] niks3-server[1225]: 2026/09/23 07:01:59 INFO Signed narinfos id=60 count=1 lb # [ 180.030057] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst lb # [ 180.045451] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 180.049682] niks3-server[1225]: 2026/09/23 07:01:59 INFO Completed upload id=60 worker2 # [ 180.057378] nix-grpc-daemon[1644]: time=2026-09-23T07:01:59.243Z level=INFO msg="Upload complete. (215ms)" lb # [ 180.051766] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo lb # [ 180.057288] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls worker2 # [ 180.063621] nix-grpc-daemon[1567]: ts=2026-09-23T07:01:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=1 worker2 # [ 180.080425] nix-daemon[855]: reaped child process 1784, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.14 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 180.388296] nix-daemon[869]: reaped child process 1749, status = succeeded worker1 # [ 180.466297] nix-daemon[869]: accepted connection from pid 1790, user nix-grpc-daemon (trusted) worker1 # [ 180.478145] nix-daemon[869]: reaped child process 1793, status = succeeded lb # [ 180.481155] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 180.497331] nix-grpc-daemon[1702]: time=2026-09-23T07:01:58.776Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 180.499065] nix-grpc-daemon[1702]: time=2026-09-23T07:01:58.778Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)" lb # [ 180.527359] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 180.533277] niks3-server[1225]: 2026/09/23 07:01:59 INFO Signed narinfos id=61 count=1 worker1 # [ 180.540233] nix-grpc-daemon[1702]: time=2026-09-23T07:01:58.819Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.538644] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls lb # [ 180.543091] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv lb # [ 180.550551] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 180.556060] niks3-server[1225]: 2026/09/23 07:01:59 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 180.559364] niks3-server[1225]: 2026/09/23 07:01:59 INFO Completed upload id=61 worker1 # [ 180.563539] nix-grpc-daemon[1702]: time=2026-09-23T07:01:58.843Z level=INFO msg="Upload complete. (183ms)" lb # [ 180.562797] niks3-server[1225]: 2026/09/23 07:01:59 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo worker1 # [ 180.570695] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43832 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 180.584582] nix-daemon[869]: reaped child process 1747, status = succeeded client # [ 180.615188] systemd[1]: sw.service: Deactivated successfully. client # [ 180.618377] systemd[1]: sw.service: Consumed 120ms CPU time over 8.072s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 180.674756] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:58Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 181.688774] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:44096 worker2 # [ 181.695734] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:00Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 181.680998] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:59Z level=info event=scheduler_restarting worker1 # [ 181.699931] nix-grpc-daemon[1595]: ts=2026-09-23T07:01:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 181.719444] nix-daemon[869]: reaped child process 1668, status = succeeded worker1 # [ 182.597847] nix-grpc-daemon[1595]: E0000 00:00:1790146920.877342 1641 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 183.709295] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 183.712371] systemd[1]: nix-grpc-daemon.service: Consumed 566ms CPU time over 24.146s wall clock time, 35M memory peak, 21.4K incoming IP traffic, 28.5K outgoing IP traffic. worker1 # [ 183.728188] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 183.819157] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 183.821219] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 183.825309] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 183.830066] nix-grpc-daemon[1807]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 183.831434] nix-grpc-daemon[1807]: E0000 00:00:1790146922.108253 1807 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 183.837061] nix-grpc-daemon[1807]: E0000 00:00:1790146922.113601 1807 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 183.839203] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 183.845081] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 183.847134] nix-grpc-daemon[1807]: E0000 00:00:1790146922.123865 1850 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 183.859501] nix-grpc-daemon[1807]: E0000 00:00:1790146922.138429 1850 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 183.863138] nix-grpc-daemon[1807]: E0000 00:00:1790146922.138547 1850 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.47 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 183.874062] nix-grpc-daemon[1807]: E0000 00:00:1790146922.153674 1850 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 183.882060] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 183.887457] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:44096 worker2 # [ 183.892780] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.47 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 184.110717] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 970769757c51462f8dd720ec3195b976 client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 184.245168] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 184.256722] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) client # [ 184.280660] nix[1514]: don't know how to build these paths: client # [ 184.281535] nix[1514]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv client # [ 184.282507] nix[1514]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv client # [ 184.284100] nix[1514]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv client # [ 184.285291] nix[1514]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker2 # [ 184.283337] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=0 worker2 # [ 184.289219] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 184.293643] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=0 worker2 # [ 184.297493] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 184.301777] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=0 worker2 # [ 184.305762] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=0 worker1 # [ 184.328699] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 184.332508] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker1 # [ 184.339810] nix-grpc-daemon[1807]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5) worker2 # [ 184.346333] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 184.365348] nix[1514]: copying 1 paths to 'lb:50051'... worker2 # [ 184.358140] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 184.368171] nix[1514]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'... worker2 # [ 184.370950] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker1 # [ 184.429964] nix-grpc-daemon[1807]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 2/5) worker2 # [ 184.466429] nix-daemon[855]: accepted connection from pid 1820, user nix-grpc-daemon (trusted) worker2 # [ 184.479348] nix-daemon[855]: reaped child process 1823, status = succeeded lb # [ 184.478612] niks3-server[1225]: 2026/09/23 07:02:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 184.495267] nix-grpc-daemon[1644]: time=2026-09-23T07:02:03.680Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 184.497264] nix-grpc-daemon[1644]: time=2026-09-23T07:02:03.682Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)" lb # [ 184.509270] niks3-server[1225]: 2026/09/23 07:02:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker2 # [ 184.522163] nix-grpc-daemon[1644]: time=2026-09-23T07:02:03.707Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.514190] niks3-server[1225]: 2026/09/23 07:02:03 INFO Signed narinfos id=62 count=1 lb # [ 184.528379] niks3-server[1225]: 2026/09/23 07:02:03 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls lb # [ 184.533297] niks3-server[1225]: 2026/09/23 07:02:03 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst lb # [ 184.539905] niks3-server[1225]: 2026/09/23 07:02:03 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 184.547819] niks3-server[1225]: 2026/09/23 07:02:03 INFO Completed upload id=62 worker2 # [ 184.555529] nix-grpc-daemon[1644]: time=2026-09-23T07:02:03.741Z level=INFO msg="Upload complete. (173ms)" lb # [ 184.550351] niks3-server[1225]: 2026/09/23 07:02:03 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo worker2 # [ 184.559725] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 184.565267] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker1 # [ 184.566924] nix-grpc-daemon[1807]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 318 ms (attempt 3/5) worker2 # [ 184.575471] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 184.581127] nix-daemon[855]: reaped child process 1818, status = succeeded worker2 # [ 184.583941] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # [ 184.592127] nix[1514]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051... worker1 # [ 184.887770] nix-grpc-daemon[1807]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 426 ms (attempt 4/5) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 185.315483] nix-grpc-daemon[1807]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 185.319382] nix-grpc-daemon[1807]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 185.335271] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 client # [ 185.356738] nix[1514]: copying 1 paths to 'lb:50051'... client # [ 185.360631] nix[1514]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'... worker1 # [ 185.349330] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 185.361901] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 185.388989] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:03Z level=info event=niks3_push_started pid=1872 worker1 # [ 185.491570] nix-grpc-daemon[1872]: time=2026-09-23T07:02:03.770Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 185.580199] nix-daemon[869]: accepted connection from pid 1885, user nix-grpc-daemon (trusted) worker1 # [ 185.593210] nix-daemon[869]: reaped child process 1888, status = succeeded lb # [ 185.596610] niks3-server[1225]: 2026/09/23 07:02:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.609627] nix-grpc-daemon[1872]: time=2026-09-23T07:02:03.889Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 185.611291] nix-grpc-daemon[1872]: time=2026-09-23T07:02:03.890Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)" lb # [ 185.635514] niks3-server[1225]: 2026/09/23 07:02:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 185.645871] niks3-server[1225]: 2026/09/23 07:02:04 INFO Signed narinfos id=63 count=1 worker1 # [ 185.654081] nix-grpc-daemon[1872]: time=2026-09-23T07:02:03.933Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.649821] niks3-server[1225]: 2026/09/23 07:02:04 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls lb # [ 185.656996] niks3-server[1225]: 2026/09/23 07:02:04 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst lb # [ 185.672943] niks3-server[1225]: 2026/09/23 07:02:04 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 185.678028] niks3-server[1225]: 2026/09/23 07:02:04 INFO Completed upload id=63 worker1 # [ 185.682218] nix-grpc-daemon[1872]: time=2026-09-23T07:02:03.961Z level=INFO msg="Upload complete. (187ms)" lb # [ 185.680206] niks3-server[1225]: 2026/09/23 07:02:04 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo worker1 # [ 185.685211] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 185.690407] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 185.701106] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 185.705158] nix-daemon[869]: reaped child process 1868, status = succeeded worker1 # [ 185.710821] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) client # [ 185.722552] nix[1514]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051... lb # [ 186.615869] envoy[912]: conn peer=[2001:db8:1::1]:55506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4382 ms=4975 lb # [ 186.619734] envoy[912]: conn peer=[2001:db8:1::1]:33260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1498 ms=7927 lb # [ 186.623507] envoy[912]: conn peer=[2001:db8:1::3]:37246 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=482 ms=21974 worker2 # [ 186.823516] nix-daemon[855]: reaped child process 1827, status = succeeded worker2 # [ 186.905969] nix-daemon[855]: accepted connection from pid 1855, user nix-grpc-daemon (trusted) worker2 # [ 186.917116] nix-daemon[855]: reaped child process 1858, status = succeeded lb # [ 186.917337] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 186.934264] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.119Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 186.935729] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.121Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)" lb # [ 186.958741] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls lb # [ 186.969997] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 186.977266] niks3-server[1225]: 2026/09/23 07:02:06 INFO Signed narinfos id=64 count=1 worker2 # [ 186.988344] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.173Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.986410] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv lb # [ 186.991812] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst lb # [ 186.999906] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 187.003942] niks3-server[1225]: 2026/09/23 07:02:06 INFO Completed upload id=64 worker2 # [ 187.011938] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.197Z level=INFO msg="Upload complete. (219ms)" lb # [ 187.006256] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo worker2 # [ 187.018598] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=1 worker2 # [ 187.027372] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 187.038801] nix-daemon[855]: reaped child process 1825, status = succeeded worker2 # [ 187.070598] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 187.084183] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 187.104068] nix[1514]: copying 1 paths to 'lb:50051'... worker2 # [ 187.096171] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 187.106387] nix[1514]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'... worker2 # [ 187.106642] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 187.202388] nix-daemon[855]: accepted connection from pid 1862, user nix-grpc-daemon (trusted) worker2 # [ 187.214761] nix-daemon[855]: reaped child process 1865, status = succeeded lb # [ 187.214644] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 187.230918] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.416Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 187.232382] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.418Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)" lb # [ 187.250203] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign worker2 # [ 187.265034] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.450Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.257332] niks3-server[1225]: 2026/09/23 07:02:06 INFO Signed narinfos id=65 count=1 lb # [ 187.264149] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls lb # [ 187.268388] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst lb # [ 187.283448] niks3-server[1225]: 2026/09/23 07:02:06 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 187.288887] niks3-server[1225]: 2026/09/23 07:02:06 INFO Completed upload id=65 worker2 # [ 187.296858] nix-grpc-daemon[1644]: time=2026-09-23T07:02:06.482Z level=INFO msg="Upload complete. (176ms)" lb # [ 187.290666] niks3-server[1225]: 2026/09/23 07:02:06 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo worker2 # [ 187.301216] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 187.306927] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 187.318984] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 187.328905] nix-daemon[855]: reaped child process 1860, status = succeeded worker2 # [ 187.331714] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # [ 187.339777] nix[1514]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051... worker1 # [ 187.727338] nix-daemon[869]: reaped child process 1892, status = succeeded worker1 # [ 187.802914] nix-daemon[869]: accepted connection from pid 1917, user nix-grpc-daemon (trusted) worker1 # [ 187.815608] nix-daemon[869]: reaped child process 1920, status = succeeded lb # [ 187.818968] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.832309] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 187.834030] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.113Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)" lb # [ 187.858549] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv lb # [ 187.871060] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 187.876432] niks3-server[1225]: 2026/09/23 07:02:07 INFO Signed narinfos id=66 count=1 worker1 # [ 187.884420] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.163Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.884417] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls lb # [ 187.897061] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 187.898827] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 187.902944] niks3-server[1225]: 2026/09/23 07:02:07 INFO Completed upload id=66 worker1 # [ 187.907218] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.186Z level=INFO msg="Upload complete. (188ms)" lb # [ 187.904681] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo worker1 # [ 187.912926] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=1 worker2 # [ 187.917275] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:07Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 187.921290] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 187.931293] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 187.934342] nix-daemon[869]: reaped child process 1890, status = succeeded client # [ 187.949179] nix[1514]: copying 1 paths to 'lb:50051'... worker1 # [ 187.940230] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 client # [ 187.951736] nix[1514]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'... worker1 # [ 187.946172] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 188.031504] nix-daemon[869]: accepted connection from pid 1924, user nix-grpc-daemon (trusted) worker1 # [ 188.044567] nix-daemon[869]: reaped child process 1927, status = succeeded lb # [ 188.048379] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.060981] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.340Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 188.062461] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.342Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)" lb # [ 188.082622] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 188.088379] niks3-server[1225]: 2026/09/23 07:02:07 INFO Signed narinfos id=67 count=1 worker1 # [ 188.096649] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.376Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.098204] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls lb # [ 188.102581] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst lb # [ 188.113388] niks3-server[1225]: 2026/09/23 07:02:07 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 188.119315] niks3-server[1225]: 2026/09/23 07:02:07 INFO Completed upload id=67 worker1 # [ 188.123694] nix-grpc-daemon[1872]: time=2026-09-23T07:02:06.403Z level=INFO msg="Upload complete. (171ms)" lb # [ 188.121489] niks3-server[1225]: 2026/09/23 07:02:07 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo worker1 # [ 188.127849] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 188.133096] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 188.142468] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 188.151118] nix-daemon[869]: reaped child process 1922, status = succeeded worker1 # [ 188.154395] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) client # [ 188.166825] nix[1514]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051... worker2 # [ 188.957484] nix-daemon[855]: reaped child process 1869, status = succeeded worker2 # [ 189.036039] nix-daemon[855]: accepted connection from pid 1894, user nix-grpc-daemon (trusted) worker2 # [ 189.046967] nix-daemon[855]: reaped child process 1897, status = succeeded lb # [ 189.046860] niks3-server[1225]: 2026/09/23 07:02:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 189.063263] nix-grpc-daemon[1644]: time=2026-09-23T07:02:08.248Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 189.064747] nix-grpc-daemon[1644]: time=2026-09-23T07:02:08.250Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)" lb # [ 189.094301] niks3-server[1225]: 2026/09/23 07:02:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker2 # [ 189.106873] nix-grpc-daemon[1644]: time=2026-09-23T07:02:08.292Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.099565] niks3-server[1225]: 2026/09/23 07:02:08 INFO Signed narinfos id=68 count=1 lb # [ 189.107953] niks3-server[1225]: 2026/09/23 07:02:08 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv lb # [ 189.113448] niks3-server[1225]: 2026/09/23 07:02:08 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst lb # [ 189.118563] niks3-server[1225]: 2026/09/23 07:02:08 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls lb # [ 189.128245] niks3-server[1225]: 2026/09/23 07:02:08 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 189.131850] niks3-server[1225]: 2026/09/23 07:02:08 INFO Completed upload id=68 lb # [ 189.135727] niks3-server[1225]: 2026/09/23 07:02:08 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo worker2 # [ 189.142055] nix-grpc-daemon[1644]: time=2026-09-23T07:02:08.327Z level=INFO msg="Upload complete. (202ms)" worker2 # [ 189.148590] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=1 worker2 # [ 189.163155] nix-daemon[855]: reaped child process 1867, status = succeeded worker1 # [ 189.971306] nix-daemon[869]: reaped child process 1931, status = succeeded worker1 # [ 190.046712] nix-daemon[869]: accepted connection from pid 1956, user nix-grpc-daemon (trusted) worker1 # [ 190.059321] nix-daemon[869]: reaped child process 1959, status = succeeded lb # [ 190.063290] niks3-server[1225]: 2026/09/23 07:02:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.076833] nix-grpc-daemon[1872]: time=2026-09-23T07:02:08.356Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 190.078521] nix-grpc-daemon[1872]: time=2026-09-23T07:02:08.358Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)" lb # [ 190.107981] niks3-server[1225]: 2026/09/23 07:02:09 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls lb # [ 190.112564] niks3-server[1225]: 2026/09/23 07:02:09 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst lb # [ 190.122768] niks3-server[1225]: 2026/09/23 07:02:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 190.127729] niks3-server[1225]: 2026/09/23 07:02:09 INFO Signed narinfos id=69 count=1 worker1 # [ 190.132043] nix-grpc-daemon[1872]: time=2026-09-23T07:02:08.411Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.134117] niks3-server[1225]: 2026/09/23 07:02:09 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv lb # [ 190.144159] niks3-server[1225]: 2026/09/23 07:02:09 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 190.148652] niks3-server[1225]: 2026/09/23 07:02:09 INFO Completed upload id=69 lb # [ 190.150906] niks3-server[1225]: 2026/09/23 07:02:09 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo worker1 # [ 190.154351] nix-grpc-daemon[1872]: time=2026-09-23T07:02:08.433Z level=INFO msg="Upload complete. (192ms)" worker1 # [ 190.160741] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=1 worker1 # [ 190.175461] nix-daemon[869]: reaped child process 1929, status = succeeded client # [ 190.205984] systemd[1]: burst.service: Deactivated successfully. client # [ 190.210712] systemd[1]: burst.service: Consumed 162ms CPU time over 6.096s wall clock time, 53.9M memory peak, 14K incoming IP traffic, 18.8K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 7.35 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 191.462682] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 191.470073] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 191.513787] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:10Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=0 worker2 # [ 191.518267] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:10Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 191.522705] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:10Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=0 worker2 # [ 191.526662] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:10Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=0 worker1 # [ 191.554720] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 191.567830] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 191.579808] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 191.590946] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 191.692819] nix-daemon[869]: accepted connection from pid 1967, user nix-grpc-daemon (trusted) worker1 # [ 191.705340] nix-daemon[869]: reaped child process 1970, status = succeeded lb # [ 191.710820] niks3-server[1225]: 2026/09/23 07:02:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.727407] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.006Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.728881] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.006Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)" lb # [ 191.744796] niks3-server[1225]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst lb # [ 191.753913] niks3-server[1225]: 2026/09/23 07:02:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker1 # [ 191.765713] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.045Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.761425] niks3-server[1225]: 2026/09/23 07:02:10 INFO Signed narinfos id=70 count=1 lb # [ 191.768367] niks3-server[1225]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls lb # [ 191.776396] niks3-server[1225]: 2026/09/23 07:02:10 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 191.785124] niks3-server[1225]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo lb # [ 191.788584] niks3-server[1225]: 2026/09/23 07:02:10 INFO Completed upload id=70 worker1 # [ 191.792995] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.072Z level=INFO msg="Upload complete. (190ms)" worker1 # [ 191.797888] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 191.803569] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 191.813847] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 191.825746] nix-daemon[869]: reaped child process 1965, status = succeeded worker1 # [ 191.829355] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 192.454962] nix-daemon[869]: reaped child process 1974, status = succeeded worker1 # [ 192.531451] nix-daemon[869]: accepted connection from pid 1990, user nix-grpc-daemon (trusted) worker1 # [ 192.544075] nix-daemon[869]: reaped child process 1993, status = succeeded lb # [ 192.548319] niks3-server[1225]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.563365] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.843Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 192.564970] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.844Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)" lb # [ 192.593097] niks3-server[1225]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls lb # [ 192.597530] niks3-server[1225]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv lb # [ 192.604666] niks3-server[1225]: 2026/09/23 07:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 192.607946] niks3-server[1225]: 2026/09/23 07:02:11 INFO Signed narinfos id=71 count=1 worker1 # [ 192.612162] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.891Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.619592] niks3-server[1225]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 192.623270] niks3-server[1225]: 2026/09/23 07:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 192.627674] niks3-server[1225]: 2026/09/23 07:02:11 INFO Completed upload id=71 lb # [ 192.629557] niks3-server[1225]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo worker1 # [ 192.633398] nix-grpc-daemon[1872]: time=2026-09-23T07:02:10.912Z level=INFO msg="Upload complete. (198ms)" worker1 # [ 192.638679] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=1 worker2 # [ 192.643432] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:11Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 192.646270] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 192.653510] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:11Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=0 worker1 # [ 192.657527] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 192.664718] nix-daemon[869]: reaped child process 1972, status = succeeded worker1 # [ 192.666209] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 192.670893] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 192.756178] nix-daemon[869]: accepted connection from pid 1997, user nix-grpc-daemon (trusted) worker1 # [ 192.768635] nix-daemon[869]: reaped child process 2000, status = succeeded lb # [ 192.773445] niks3-server[1225]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.786337] nix-grpc-daemon[1872]: time=2026-09-23T07:02:11.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 192.788361] nix-grpc-daemon[1872]: time=2026-09-23T07:02:11.067Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)" lb # [ 192.807464] niks3-server[1225]: 2026/09/23 07:02:12 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst lb # [ 192.815322] niks3-server[1225]: 2026/09/23 07:02:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 192.825431] nix-grpc-daemon[1872]: time=2026-09-23T07:02:11.104Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.821428] niks3-server[1225]: 2026/09/23 07:02:12 INFO Signed narinfos id=72 count=1 lb # [ 192.832503] niks3-server[1225]: 2026/09/23 07:02:12 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls lb # [ 192.842648] niks3-server[1225]: 2026/09/23 07:02:12 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 192.845811] niks3-server[1225]: 2026/09/23 07:02:12 INFO Completed upload id=72 worker1 # [ 192.851567] nix-grpc-daemon[1872]: time=2026-09-23T07:02:11.131Z level=INFO msg="Upload complete. (176ms)" lb # [ 192.848564] niks3-server[1225]: 2026/09/23 07:02:12 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo worker1 # [ 192.855603] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 192.860777] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 192.870106] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 192.877781] nix-daemon[869]: reaped child process 1995, status = succeeded worker1 # [ 192.882490] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 193.898821] nix-daemon[869]: reaped child process 2004, status = succeeded worker1 # [ 193.978091] nix-daemon[869]: accepted connection from pid 2021, user nix-grpc-daemon (trusted) worker1 # [ 193.989427] nix-daemon[869]: reaped child process 2024, status = succeeded lb # [ 193.995559] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.008334] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.287Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 194.009903] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.289Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)" lb # [ 194.039916] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls lb # [ 194.042937] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 194.050227] niks3-server[1225]: 2026/09/23 07:02:13 INFO Signed narinfos id=73 count=1 worker1 # [ 194.057770] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.337Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.057158] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv lb # [ 194.059919] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 194.065723] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 194.068832] niks3-server[1225]: 2026/09/23 07:02:13 INFO Completed upload id=73 worker1 # [ 194.074613] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.354Z level=INFO msg="Upload complete. (185ms)" lb # [ 194.071303] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo worker1 # [ 194.079449] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=1 worker1 # [ 194.086611] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 194.087975] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:13Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 194.093929] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:13Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=0 worker1 # [ 194.097285] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 194.099812] nix-daemon[869]: reaped child process 2002, status = succeeded worker1 # [ 194.103462] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 194.107401] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 194.192331] nix-daemon[869]: accepted connection from pid 2028, user nix-grpc-daemon (trusted) worker1 # [ 194.205786] nix-daemon[869]: reaped child process 2031, status = succeeded lb # [ 194.208078] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.222136] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 194.223588] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.503Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)" lb # [ 194.240139] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign worker1 # [ 194.249067] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.528Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.244841] niks3-server[1225]: 2026/09/23 07:02:13 INFO Signed narinfos id=74 count=1 lb # [ 194.254231] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls lb # [ 194.257836] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst lb # [ 194.266684] niks3-server[1225]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 194.269463] niks3-server[1225]: 2026/09/23 07:02:13 INFO Completed upload id=74 worker1 # [ 194.273862] nix-grpc-daemon[1872]: time=2026-09-23T07:02:12.553Z level=INFO msg="Upload complete. (160ms)" lb # [ 194.271410] niks3-server[1225]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo worker1 # [ 194.276615] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 194.280904] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 194.287924] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 194.290257] nix-daemon[869]: reaped child process 2026, status = succeeded worker1 # [ 194.295346] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 195.333750] nix-daemon[869]: reaped child process 2035, status = succeeded worker1 # [ 195.412498] nix-daemon[869]: accepted connection from pid 2051, user nix-grpc-daemon (trusted) worker1 # [ 195.423403] nix-daemon[869]: reaped child process 2054, status = succeeded lb # [ 195.429427] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.442266] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.721Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 195.445057] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.723Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)" lb # [ 195.474607] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 195.482808] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls lb # [ 195.489502] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 195.495521] niks3-server[1225]: 2026/09/23 07:02:14 INFO Signed narinfos id=75 count=1 worker1 # [ 195.503069] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.782Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.509492] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 195.512507] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv lb # [ 195.515112] niks3-server[1225]: 2026/09/23 07:02:14 INFO Completed upload id=75 worker1 # [ 195.519588] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.799Z level=INFO msg="Upload complete. (217ms)" lb # [ 195.518272] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo worker1 # [ 195.525674] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=1 worker2 # [ 195.533924] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:14Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 195.534729] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 195.539641] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:14Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=0 worker1 # [ 195.546523] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 195.552792] nix-daemon[869]: reaped child process 2033, status = succeeded worker1 # [ 195.558396] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.837Z level=INFO msg="All 1 paths already cached" worker1 # [ 195.564384] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 195.572167] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 195.656459] nix-daemon[869]: accepted connection from pid 2058, user nix-grpc-daemon (trusted) worker1 # [ 195.669243] nix-daemon[869]: reaped child process 2061, status = succeeded lb # [ 195.672815] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.685309] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.964Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 195.687357] nix-grpc-daemon[1872]: time=2026-09-23T07:02:13.966Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)" lb # [ 195.705175] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls lb # [ 195.712394] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 195.718232] niks3-server[1225]: 2026/09/23 07:02:14 INFO Signed narinfos id=76 count=1 worker1 # [ 195.725384] nix-grpc-daemon[1872]: time=2026-09-23T07:02:14.004Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.732436] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst lb # [ 195.739856] niks3-server[1225]: 2026/09/23 07:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 195.745389] niks3-server[1225]: 2026/09/23 07:02:14 INFO Completed upload id=76 lb # [ 195.747656] niks3-server[1225]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo worker1 # [ 195.749793] nix-grpc-daemon[1872]: time=2026-09-23T07:02:14.029Z level=INFO msg="Upload complete. (172ms)" worker1 # [ 195.753850] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 195.758577] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 195.768076] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 195.774156] nix-daemon[869]: reaped child process 2056, status = succeeded worker1 # [ 195.777931] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) lb # [ 196.616480] envoy[912]: conn peer=[2001:db8:1::1]:39848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9604 tx=5186 ms=5930 worker1 # [ 197.210774] nix-daemon[869]: reaped child process 2065, status = succeeded worker1 # [ 197.303453] nix-daemon[869]: accepted connection from pid 2081, user nix-grpc-daemon (trusted) worker1 # [ 197.314990] nix-daemon[869]: reaped child process 2084, status = succeeded lb # [ 197.320470] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 197.332505] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.611Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 197.334156] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.613Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)" lb # [ 197.360688] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst lb # [ 197.368868] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv lb # [ 197.372609] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker1 # [ 197.384708] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.664Z level=INFO msg="Uploading 1 narinfos" lb # [ 197.380187] niks3-server[1225]: 2026/09/23 07:02:16 INFO Signed narinfos id=77 count=1 lb # [ 197.388132] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls lb # [ 197.397203] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 197.402983] niks3-server[1225]: 2026/09/23 07:02:16 INFO Completed upload id=77 worker1 # [ 197.407336] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.686Z level=INFO msg="Upload complete. (220ms)" lb # [ 197.406164] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo worker1 # [ 197.413820] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=1 worker2 # [ 197.419653] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:16Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 197.422355] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 197.427771] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:16Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=0 worker1 # [ 197.432893] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 197.441783] nix-daemon[869]: reaped child process 2063, status = succeeded worker1 # [ 197.444996] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.723Z level=INFO msg="All 1 paths already cached" worker1 # [ 197.448952] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 197.454372] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 197.539558] nix-daemon[869]: accepted connection from pid 2088, user nix-grpc-daemon (trusted) worker1 # [ 197.551909] nix-daemon[869]: reaped child process 2091, status = succeeded lb # [ 197.556379] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 197.568803] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.848Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 197.570775] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.850Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)" lb # [ 197.586799] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst lb # [ 197.595889] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 197.600493] niks3-server[1225]: 2026/09/23 07:02:16 INFO Signed narinfos id=78 count=1 worker1 # [ 197.607709] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.887Z level=INFO msg="Uploading 1 narinfos" lb # [ 197.612290] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls lb # [ 197.622057] niks3-server[1225]: 2026/09/23 07:02:16 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 197.628343] niks3-server[1225]: 2026/09/23 07:02:16 INFO Completed upload id=78 worker1 # [ 197.632128] nix-grpc-daemon[1872]: time=2026-09-23T07:02:15.911Z level=INFO msg="Upload complete. (172ms)" lb # [ 197.632078] niks3-server[1225]: 2026/09/23 07:02:16 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo worker1 # [ 197.635229] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 197.640503] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 197.651139] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 197.656164] nix-daemon[869]: reaped child process 2086, status = succeeded worker1 # [ 197.660850] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 198.691948] nix-daemon[869]: reaped child process 2095, status = succeeded worker1 # [ 198.782649] nix-daemon[869]: accepted connection from pid 2111, user nix-grpc-daemon (trusted) worker1 # [ 198.793750] nix-daemon[869]: reaped child process 2114, status = succeeded lb # [ 198.798887] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.812637] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.092Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 198.816608] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.093Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)" lb # [ 198.848417] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign worker1 # [ 198.858735] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.138Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.854658] niks3-server[1225]: 2026/09/23 07:02:18 INFO Signed narinfos id=79 count=1 lb # [ 198.856699] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv lb # [ 198.863319] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls lb # [ 198.868910] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst lb # [ 198.878492] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 198.881437] niks3-server[1225]: 2026/09/23 07:02:18 INFO Completed upload id=79 lb # [ 198.883922] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo worker1 # [ 198.887104] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.166Z level=INFO msg="Upload complete. (216ms)" worker2 # [ 198.898253] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:18Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 198.894262] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=1 worker1 # [ 198.908381] nix-daemon[869]: reaped child process 2093, status = succeeded worker1 # [ 198.937765] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 198.944926] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:18Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=0 worker1 # [ 198.950506] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 198.964605] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.244Z level=INFO msg="All 1 paths already cached" worker1 # [ 198.972310] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 198.980417] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 199.077869] nix-daemon[869]: accepted connection from pid 2118, user nix-grpc-daemon (trusted) worker1 # [ 199.090925] nix-daemon[869]: reaped child process 2121, status = succeeded lb # [ 199.093953] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 199.106691] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.386Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 199.108272] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.387Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)" lb # [ 199.129529] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 199.134346] niks3-server[1225]: 2026/09/23 07:02:18 INFO Signed narinfos id=80 count=1 worker1 # [ 199.141330] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.420Z level=INFO msg="Uploading 1 narinfos" lb # [ 199.140085] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst lb # [ 199.148675] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls lb # [ 199.160946] niks3-server[1225]: 2026/09/23 07:02:18 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 199.168188] niks3-server[1225]: 2026/09/23 07:02:18 INFO Completed upload id=80 worker1 # [ 199.172587] nix-grpc-daemon[1872]: time=2026-09-23T07:02:17.452Z level=INFO msg="Upload complete. (179ms)" lb # [ 199.171554] niks3-server[1225]: 2026/09/23 07:02:18 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo worker1 # [ 199.177341] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 199.185256] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 199.194534] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 199.203390] nix-daemon[869]: reaped child process 2116, status = succeeded worker1 # [ 199.208489] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 200.223481] nix-daemon[869]: reaped child process 2125, status = succeeded worker1 # [ 200.299251] nix-daemon[869]: accepted connection from pid 2141, user nix-grpc-daemon (trusted) worker1 # [ 200.312551] nix-daemon[869]: reaped child process 2144, status = succeeded lb # [ 200.315738] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.329254] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.608Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 200.331050] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.610Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)" lb # [ 200.359773] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 200.364970] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls worker1 # [ 200.374767] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.654Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.370252] niks3-server[1225]: 2026/09/23 07:02:19 INFO Signed narinfos id=81 count=1 lb # [ 200.376832] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst lb # [ 200.382215] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv lb # [ 200.391715] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 200.395097] niks3-server[1225]: 2026/09/23 07:02:19 INFO Completed upload id=81 worker1 # [ 200.399515] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.679Z level=INFO msg="Upload complete. (184ms)" lb # [ 200.398330] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo worker1 # [ 200.405801] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=1 worker2 # [ 200.413868] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:19Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 200.414627] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2 # [ 200.419552] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:19Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=0 worker1 # [ 200.425541] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 200.433366] nix-daemon[869]: reaped child process 2123, status = succeeded worker1 # [ 200.436528] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.715Z level=INFO msg="All 2 paths already cached" worker1 # [ 200.441553] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 200.446377] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 200.530494] nix-daemon[869]: accepted connection from pid 2148, user nix-grpc-daemon (trusted) worker1 # [ 200.543460] nix-daemon[869]: reaped child process 2151, status = succeeded lb # [ 200.547072] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.560031] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.839Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 200.562064] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.841Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)" lb # [ 200.579774] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign worker1 # [ 200.593360] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.872Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.588994] niks3-server[1225]: 2026/09/23 07:02:19 INFO Signed narinfos id=82 count=1 lb # [ 200.593787] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls lb # [ 200.599587] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst lb # [ 200.613251] niks3-server[1225]: 2026/09/23 07:02:19 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 200.618660] niks3-server[1225]: 2026/09/23 07:02:19 INFO Completed upload id=82 worker1 # [ 200.622713] nix-grpc-daemon[1872]: time=2026-09-23T07:02:18.902Z level=INFO msg="Upload complete. (171ms)" lb # [ 200.621333] niks3-server[1225]: 2026/09/23 07:02:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo worker1 # [ 200.626624] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 200.631594] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 200.641109] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 200.648332] nix-daemon[869]: reaped child process 2146, status = succeeded worker1 # [ 200.653310] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 201.488630] nix-daemon[869]: reaped child process 2155, status = succeeded worker1 # [ 201.564763] nix-daemon[869]: accepted connection from pid 2171, user nix-grpc-daemon (trusted) worker1 # [ 201.577589] nix-daemon[869]: reaped child process 2174, status = succeeded lb # [ 201.581713] niks3-server[1225]: 2026/09/23 07:02:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 201.595041] nix-grpc-daemon[1872]: time=2026-09-23T07:02:19.874Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 201.598071] nix-grpc-daemon[1872]: time=2026-09-23T07:02:19.876Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)" lb # [ 201.624235] niks3-server[1225]: 2026/09/23 07:02:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign worker1 # [ 201.636027] nix-grpc-daemon[1872]: time=2026-09-23T07:02:19.915Z level=INFO msg="Uploading 1 narinfos" lb # [ 201.632156] niks3-server[1225]: 2026/09/23 07:02:20 INFO Signed narinfos id=83 count=1 lb # [ 201.641347] niks3-server[1225]: 2026/09/23 07:02:20 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv lb # [ 201.645632] niks3-server[1225]: 2026/09/23 07:02:20 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst lb # [ 201.652199] niks3-server[1225]: 2026/09/23 07:02:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls lb # [ 201.659947] niks3-server[1225]: 2026/09/23 07:02:20 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 201.663727] niks3-server[1225]: 2026/09/23 07:02:20 INFO Completed upload id=83 worker1 # [ 201.668062] nix-grpc-daemon[1872]: time=2026-09-23T07:02:19.947Z level=INFO msg="Upload complete. (207ms)" lb # [ 201.665890] niks3-server[1225]: 2026/09/23 07:02:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo worker2 # [ 201.678819] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:20Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 201.674946] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=1 worker1 # [ 201.689295] nix-daemon[869]: reaped child process 2153, status = succeeded worker1 # [ 201.719379] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 201.732768] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 201.746283] nix-grpc-daemon[1872]: time=2026-09-23T07:02:20.025Z level=INFO msg="All 2 paths already cached" worker1 # [ 201.754323] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 201.764739] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 201.864465] nix-daemon[869]: accepted connection from pid 2178, user nix-grpc-daemon (trusted) worker1 # [ 201.879576] nix-daemon[869]: reaped child process 2181, status = succeeded lb # [ 201.881558] niks3-server[1225]: 2026/09/23 07:02:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 201.894954] nix-grpc-daemon[1872]: time=2026-09-23T07:02:20.174Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 201.896456] nix-grpc-daemon[1872]: time=2026-09-23T07:02:20.176Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)" lb # [ 201.914322] niks3-server[1225]: 2026/09/23 07:02:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign worker1 # [ 201.926120] nix-grpc-daemon[1872]: time=2026-09-23T07:02:20.205Z level=INFO msg="Uploading 1 narinfos" lb # [ 201.921807] niks3-server[1225]: 2026/09/23 07:02:21 INFO Signed narinfos id=84 count=1 lb # [ 201.929131] niks3-server[1225]: 2026/09/23 07:02:21 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls lb # [ 201.941452] niks3-server[1225]: 2026/09/23 07:02:21 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst lb # [ 201.947627] niks3-server[1225]: 2026/09/23 07:02:21 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 201.955131] niks3-server[1225]: 2026/09/23 07:02:21 INFO Completed upload id=84 worker1 # [ 201.959225] nix-grpc-daemon[1872]: time=2026-09-23T07:02:20.238Z level=INFO msg="Upload complete. (181ms)" lb # [ 201.957276] niks3-server[1225]: 2026/09/23 07:02:21 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo worker1 # [ 201.963133] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 201.969139] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 201.977488] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 201.986457] nix-daemon[869]: reaped child process 2176, status = succeeded worker1 # [ 201.991546] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 202.805582] nix-daemon[869]: reaped child process 2185, status = succeeded worker1 # [ 202.882262] nix-daemon[869]: accepted connection from pid 2201, user nix-grpc-daemon (trusted) worker1 # [ 202.894081] nix-daemon[869]: reaped child process 2204, status = succeeded lb # [ 202.898299] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.912257] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.191Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.913961] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.193Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)" lb # [ 202.943443] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 202.948259] niks3-server[1225]: 2026/09/23 07:02:22 INFO Signed narinfos id=85 count=1 worker1 # [ 202.955507] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.235Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.956408] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv lb # [ 202.961339] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst lb # [ 202.971173] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 202.977125] niks3-server[1225]: 2026/09/23 07:02:22 INFO Completed upload id=85 worker1 # [ 202.981435] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.261Z level=INFO msg="Upload complete. (184ms)" lb # [ 202.979414] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo lb # [ 202.981827] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls worker1 # [ 202.987778] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=1 worker2 # [ 202.993887] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:22Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 202.996424] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 203.007538] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 203.014336] nix-daemon[869]: reaped child process 2183, status = succeeded worker1 # [ 203.016603] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.296Z level=INFO msg="All 2 paths already cached" worker1 # [ 203.021100] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 203.025214] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 203.110745] nix-daemon[869]: accepted connection from pid 2208, user nix-grpc-daemon (trusted) worker1 # [ 203.124566] nix-daemon[869]: reaped child process 2211, status = succeeded lb # [ 203.127332] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 203.141193] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.420Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 203.143176] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.422Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)" lb # [ 203.159499] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign lb # [ 203.167162] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst lb # [ 203.171953] niks3-server[1225]: 2026/09/23 07:02:22 INFO Signed narinfos id=86 count=1 worker1 # [ 203.179494] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.459Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.185123] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls lb # [ 203.191696] niks3-server[1225]: 2026/09/23 07:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete lb # [ 203.196327] niks3-server[1225]: 2026/09/23 07:02:22 INFO Completed upload id=86 worker1 # [ 203.202861] nix-grpc-daemon[1872]: time=2026-09-23T07:02:21.482Z level=INFO msg="Upload complete. (172ms)" lb # [ 203.201846] niks3-server[1225]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo worker1 # [ 203.206874] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 203.211514] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 worker1 # [ 203.223117] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 203.227387] nix-daemon[869]: reaped child process 2206, status = succeeded worker1 # [ 203.232487] nix-daemon[869]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker1 # [ 204.248237] nix-daemon[869]: reaped child process 2215, status = succeeded worker1 # [ 204.323182] nix-daemon[869]: accepted connection from pid 2231, user nix-grpc-daemon (trusted) worker1 # [ 204.337142] nix-daemon[869]: reaped child process 2234, status = succeeded lb # [ 204.339736] niks3-server[1225]: 2026/09/23 07:02:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 204.354350] nix-grpc-daemon[1872]: time=2026-09-23T07:02:22.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 204.357064] nix-grpc-daemon[1872]: time=2026-09-23T07:02:22.635Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)" lb # [ 204.383814] niks3-server[1225]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls lb # [ 204.394098] niks3-server[1225]: 2026/09/23 07:02:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign lb # [ 204.403514] niks3-server[1225]: 2026/09/23 07:02:23 INFO Signed narinfos id=87 count=1 worker1 # [ 204.410475] nix-grpc-daemon[1872]: time=2026-09-23T07:02:22.689Z level=INFO msg="Uploading 1 narinfos" lb # [ 204.408532] niks3-server[1225]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst lb # [ 204.413494] niks3-server[1225]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv lb # [ 204.424900] niks3-server[1225]: 2026/09/23 07:02:23 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 204.427449] niks3-server[1225]: 2026/09/23 07:02:23 INFO Completed upload id=87 worker1 # [ 204.431887] nix-grpc-daemon[1872]: time=2026-09-23T07:02:22.711Z level=INFO msg="Upload complete. (192ms)" lb # [ 204.431116] niks3-server[1225]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo worker1 # [ 204.437569] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51372 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=1 worker1 # [ 204.451871] nix-daemon[869]: reaped child process 2213, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.17 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 204.583570] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 86b21e4a4da54ddc8236c6f28317fbad client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 204.708945] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44122 client # [ 204.722696] nix[1588]: don't know how to build these paths: client # [ 204.723614] nix[1588]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv worker1 # [ 204.712371] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51372 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2 # [ 204.736415] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:23Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=0 client # [ 204.745372] nix[1588]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 206.617779] envoy[912]: conn peer=[2001:db8:1::1]:39864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27107 tx=12278 ms=12995 client # [ 234.746615] nix[1588]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip} client # [ 234.756573] nix[1588]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed client # [ 234.789364] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 234.792666] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 234.796608] systemd[1]: gpu.service: Consumed 134ms CPU time over 30.206s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.39 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.06 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' lb # [ 236.620231] envoy[912]: conn peer=[2001:db8:1::1]:60360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=30048 worker1 # [ 236.854113] nix-grpc-daemon[1807]: ts=2026-09-23T07:02:55Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker2 # [ 237.578634] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 237.584380] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 237.605672] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=0 worker2 # [ 237.611137] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 237.650453] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 237.663239] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'... worker2 # [ 237.676079] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 237.687344] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 237.797086] nix-daemon[855]: accepted connection from pid 1939, user nix-grpc-daemon (trusted) worker2 # [ 237.811929] nix-daemon[855]: reaped child process 1942, status = succeeded lb # [ 237.810748] niks3-server[1225]: 2026/09/23 07:02:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 237.829664] nix-grpc-daemon[1644]: time=2026-09-23T07:02:57.015Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 237.831649] nix-grpc-daemon[1644]: time=2026-09-23T07:02:57.016Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)" lb # [ 237.850630] niks3-server[1225]: 2026/09/23 07:02:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls lb # [ 237.859969] niks3-server[1225]: 2026/09/23 07:02:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign worker2 # [ 237.874513] nix-grpc-daemon[1644]: time=2026-09-23T07:02:57.060Z level=INFO msg="Uploading 1 narinfos" lb # [ 237.866621] niks3-server[1225]: 2026/09/23 07:02:57 INFO Signed narinfos id=88 count=1 lb # [ 237.878539] niks3-server[1225]: 2026/09/23 07:02:57 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst lb # [ 237.888573] niks3-server[1225]: 2026/09/23 07:02:57 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 237.891564] niks3-server[1225]: 2026/09/23 07:02:57 INFO Completed upload id=88 worker2 # [ 237.899618] nix-grpc-daemon[1644]: time=2026-09-23T07:02:57.085Z level=INFO msg="Upload complete. (198ms)" worker2 # [ 237.904144] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 237.901112] niks3-server[1225]: 2026/09/23 07:02:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo worker2 # [ 237.910910] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 237.925157] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 237.930208] nix-daemon[855]: reaped child process 1937, status = succeeded worker2 # [ 237.939500] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051... client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 239.177418] nix-daemon[855]: reaped child process 1946, status = succeeded worker2 # [ 239.259325] nix-daemon[855]: accepted connection from pid 1962, user nix-grpc-daemon (trusted) worker2 # [ 239.271041] nix-daemon[855]: reaped child process 1965, status = succeeded lb # [ 239.270388] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.289273] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 239.290758] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.476Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)" lb # [ 239.313180] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign worker2 # [ 239.325090] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.510Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.317556] niks3-server[1225]: 2026/09/23 07:02:58 INFO Signed narinfos id=89 count=1 lb # [ 239.328159] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 239.331609] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls lb # [ 239.336540] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv lb # [ 239.340394] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete lb # [ 239.342758] niks3-server[1225]: 2026/09/23 07:02:58 INFO Completed upload id=89 worker2 # [ 239.352212] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.537Z level=INFO msg="Upload complete. (205ms)" lb # [ 239.346464] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo worker2 # [ 239.358666] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=1 worker2 # [ 239.369513] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=0 worker2 # [ 239.373590] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 239.383228] nix-daemon[855]: reaped child process 1944, status = succeeded worker2 # [ 239.413288] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 239.425704] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 239.438750] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.624Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'... worker2 # [ 239.447231] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 239.455400] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 239.549572] nix-daemon[855]: accepted connection from pid 1971, user nix-grpc-daemon (trusted) worker2 # [ 239.562585] nix-daemon[855]: reaped child process 1974, status = succeeded lb # [ 239.561362] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.578379] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.764Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 239.579944] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.765Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)" lb # [ 239.596266] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign worker2 # [ 239.608240] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.793Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.600436] niks3-server[1225]: 2026/09/23 07:02:58 INFO Signed narinfos id=90 count=1 lb # [ 239.607952] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst lb # [ 239.612733] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls lb # [ 239.622879] niks3-server[1225]: 2026/09/23 07:02:58 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 239.627279] niks3-server[1225]: 2026/09/23 07:02:58 INFO Completed upload id=90 worker2 # [ 239.635215] nix-grpc-daemon[1644]: time=2026-09-23T07:02:58.820Z level=INFO msg="Upload complete. (167ms)" lb # [ 239.629561] niks3-server[1225]: 2026/09/23 07:02:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo worker2 # [ 239.639509] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 239.645128] nix-grpc-daemon[1567]: ts=2026-09-23T07:02:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 239.653746] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 239.659167] nix-daemon[855]: reaped child process 1969, status = succeeded worker2 # [ 239.663884] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051... client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 241.079601] nix-daemon[855]: reaped child process 1978, status = succeeded worker2 # [ 241.153388] nix-daemon[855]: accepted connection from pid 1994, user nix-grpc-daemon (trusted) worker2 # [ 241.165432] nix-daemon[855]: reaped child process 1997, status = succeeded lb # [ 241.166746] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.183620] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.369Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 241.185388] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.370Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)" lb # [ 241.211772] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls lb # [ 241.215595] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst lb # [ 241.222844] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign worker2 # [ 241.236596] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.422Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.229109] niks3-server[1225]: 2026/09/23 07:03:00 INFO Signed narinfos id=91 count=1 lb # [ 241.232174] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv lb # [ 241.243060] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 241.247386] niks3-server[1225]: 2026/09/23 07:03:00 INFO Completed upload id=91 worker2 # [ 241.255301] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.440Z level=INFO msg="Upload complete. (183ms)" lb # [ 241.249753] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo worker2 # [ 241.261935] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=1 worker2 # [ 241.273857] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=0 worker2 # [ 241.278456] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 241.282252] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 241.291048] nix-daemon[855]: reaped child process 1976, status = succeeded worker2 # [ 241.292689] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 241.299492] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.485Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 241.305284] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'... worker2 # [ 241.310091] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 241.394360] nix-daemon[855]: accepted connection from pid 2003, user nix-grpc-daemon (trusted) worker2 # [ 241.407097] nix-daemon[855]: reaped child process 2006, status = succeeded lb # [ 241.406723] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.425096] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.610Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 241.427059] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.612Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)" lb # [ 241.438716] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign worker2 # [ 241.455380] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.640Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.446973] niks3-server[1225]: 2026/09/23 07:03:00 INFO Signed narinfos id=92 count=1 lb # [ 241.454549] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst lb # [ 241.461312] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls lb # [ 241.471885] niks3-server[1225]: 2026/09/23 07:03:00 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 241.478481] niks3-server[1225]: 2026/09/23 07:03:00 INFO Completed upload id=92 lb # [ 241.480503] niks3-server[1225]: 2026/09/23 07:03:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo worker2 # [ 241.485969] nix-grpc-daemon[1644]: time=2026-09-23T07:03:00.671Z level=INFO msg="Upload complete. (170ms)" worker2 # [ 241.490322] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 241.495963] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 worker2 # [ 241.505902] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) worker2 # [ 241.511428] nix-daemon[855]: reaped child process 2001, status = succeeded worker2 # [ 241.516409] nix-daemon[855]: accepted connection from pid 1567, user nix-grpc-daemon (trusted) client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051... client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 242.354809] nix-daemon[855]: reaped child process 2010, status = succeeded worker2 # [ 242.430411] nix-daemon[855]: accepted connection from pid 2026, user nix-grpc-daemon (trusted) worker2 # [ 242.442403] nix-daemon[855]: reaped child process 2029, status = succeeded lb # [ 242.441919] niks3-server[1225]: 2026/09/23 07:03:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 242.459654] nix-grpc-daemon[1644]: time=2026-09-23T07:03:01.645Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 242.461198] nix-grpc-daemon[1644]: time=2026-09-23T07:03:01.646Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)" lb # [ 242.486234] niks3-server[1225]: 2026/09/23 07:03:01 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv lb # [ 242.489844] niks3-server[1225]: 2026/09/23 07:03:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 242.505964] nix-grpc-daemon[1644]: time=2026-09-23T07:03:01.691Z level=INFO msg="Uploading 1 narinfos" lb # [ 242.498447] niks3-server[1225]: 2026/09/23 07:03:01 INFO Signed narinfos id=93 count=1 lb # [ 242.504738] niks3-server[1225]: 2026/09/23 07:03:01 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst lb # [ 242.510719] niks3-server[1225]: 2026/09/23 07:03:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls lb # [ 242.519822] niks3-server[1225]: 2026/09/23 07:03:01 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 242.523267] niks3-server[1225]: 2026/09/23 07:03:01 INFO Completed upload id=93 lb # [ 242.525209] niks3-server[1225]: 2026/09/23 07:03:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo worker2 # [ 242.532520] nix-grpc-daemon[1644]: time=2026-09-23T07:03:01.718Z level=INFO msg="Upload complete. (209ms)" worker2 # [ 242.539372] nix-grpc-daemon[1567]: ts=2026-09-23T07:03:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44122 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=1 worker2 # [ 242.554377] nix-daemon[855]: reaped child process 2008, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.15 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 243.856509] nix-grpc-daemon[1807]: ts=2026-09-23T07:03:02Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.57 seconds) (finished: run the VM test script, in 244.85 seconds) test script finished in 245.04s cleanup kill QemuMachine (pid 49) client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) client # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) lb # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T07:03:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T07:03:04Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T07:03:04Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T07:03:04Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.07 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh