tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker2 # Disk image does not exist, creating the virtualisation disk image... worker2 # Formatting '/build/vm-state-worker2/tmp.7ND1ZAinQ2', fmt=raw size=1073741824 worker2 # mke2fs 1.47.4 (6-Mar-2025) worker2: QEMU running (pid 52) worker2 # Discarding device blocks: 0/262144 done worker1 # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 50) worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Formatting '/build/vm-state-worker1/tmp.knK14uUqmZ', fmt=raw size=1073741824 worker1: QEMU running (pid 51) client # Disk image does not exist, creating the virtualisation disk image... worker2 # Filesystem UUID: c78d41dc-5684-49b7-adc2-e6a090d67ada lb # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 49) worker2 # Superblock backups stored on blocks: lb # Formatting '/build/vm-state-lb/tmp.4qabcTta6p', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.ctckxly6Dv', fmt=raw size=1073741824 worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # 32768, 98304, 163840, 229376 lb # mke2fs 1.47.4 (6-Mar-2025) worker1 # Discarding device blocks: 0/262144 done worker2 # lb # Discarding device blocks: 0/262144 done worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Allocating group tables: 0/8 done lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Filesystem UUID: 70146035-bf5b-448d-8907-857e8c6e25a3 worker2 # Writing inode tables: 0/8 done lb # Filesystem UUID: f6a17f58-4013-4783-9a94-b59379a60662 worker1 # Superblock backups stored on blocks: worker2 # Creating journal (8192 blocks): done lb # Superblock backups stored on blocks: worker1 # 32768, 98304, 163840, 229376 worker2 # Writing superblocks and filesystem accounting information: 0/8 done lb # 32768, 98304, 163840, 229376 worker1 # worker2 # lb # worker1 # Allocating group tables: 0/8 done worker2 # Virtualisation disk image created. lb # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker2 # Starting virtiofs daemons... lb # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # worker1 # Virtualisation disk image created. worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Virtualisation disk image created. worker1 # Starting virtiofs daemons... worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Starting virtiofs daemons... worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether (finished: start all VMs, in 0.24 seconds) client # mke2fs 1.47.4 (6-Mar-2025) lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... client: waiting for unit network-addresses-eth1.service client # Discarding device blocks: 0/262144 done lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client: waiting for the VM to finish booting client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # Filesystem UUID: d8967c73-c804-4787-9148-a44ac641fad0 lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # Superblock backups stored on blocks: lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # 32768, 98304, 163840, 229376 worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # Allocating group tables: 0/8 done client # Writing inode tables: 0/8 done client # Creating journal (8192 blocks): done client # Writing superblocks and filesystem accounting information: 0/8 done client # client # Virtualisation disk image created. client # Starting virtiofs daemons... client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) lb # lb # client # client # lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 worker1 # worker1 # worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)... lb # lb # lb # lb # client # Press Ctrl-B to configure iPXE (PCI 00:02.0)... client # client # client # client # lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker1 # worker1 # worker1 # worker1 # worker2 # worker2 # worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00 lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)... lb # lb # client # Press Ctrl-B to configure iPXE (PCI 00:05.0)... client # client # worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)... worker2 # worker2 # worker2 # worker2 # worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker1 # worker1 # worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00 worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)... worker2 # worker2 # lb # Booting from ROM... client # Booting from ROM... worker1 # Booting from ROM... worker2 # Booting from ROM... client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test/init regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n0idzb2a5795zcdcs33p9240n19lall0-nixos-system-lb-test/init regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgp1w89psmkzqxph18mrrd4l45zq8icp-nixos-system-worker1-test/init regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.000000] BIOS-provided physical RAM map: worker1 # [ 0.000000] BIOS-provided physical RAM map: client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved client # [ 0.000000] NX (Execute Disable) protection: active worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved client # [ 0.000000] APIC: Static calls initialized client # [ 0.000000] SMBIOS 2.8 present. worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kknbsz59pyihvz42vvc4xnin6z1984g8-nixos-system-worker2-test/init regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved worker2 # [ 0.000000] BIOS-provided physical RAM map: worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved lb # [ 0.000000] NX (Execute Disable) protection: active client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable worker1 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.000000] APIC: Static calls initialized client # [ 0.000000] DMI: Memory slots populated: 1/1 worker1 # [ 0.000000] APIC: Static calls initialized lb # [ 0.000000] SMBIOS 2.8 present. client # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] SMBIOS 2.8 present. worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 lb # [ 0.000000] DMI: Memory slots populated: 1/1 worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable worker1 # [ 0.000000] DMI: Memory slots populated: 1/1 client # [ 0.000000] kvm-clock: using sched offset of 481706363 cycles lb # [ 0.000000] Hypervisor detected: KVM worker1 # [ 0.000000] Hypervisor detected: KVM worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 client # [ 0.000005] tsc: Detected 2400.012 MHz processor lb # [ 0.000001] kvm-clock: using sched offset of 476503539 cycles worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved worker1 # [ 0.000000] kvm-clock: using sched offset of 486640087 cycles client # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved lb # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000005] tsc: Detected 2400.012 MHz processor worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor client # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT lb # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved client # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker2 # [ 0.000000] NX (Execute Disable) protection: active lb # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs client # [ 0.002752] Using GB pages for direct mapping worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker2 # [ 0.000000] APIC: Static calls initialized client # [ 0.002822] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker2 # [ 0.000000] SMBIOS 2.8 present. lb # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT worker1 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT client # [ 0.002831] ACPI: Early table checksum verification disabled lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f] worker1 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f] client # [ 0.002834] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) lb # [ 0.002740] Using GB pages for direct mapping worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 worker1 # [ 0.002741] Using GB pages for direct mapping worker2 # [ 0.000000] DMI: Memory slots populated: 1/1 lb # [ 0.002885] RAMDISK: [mem 0x5e36e000-0x5ffcffff] worker1 # [ 0.002868] RAMDISK: [mem 0x5e370000-0x5ffcffff] worker2 # [ 0.000000] Hypervisor detected: KVM client # [ 0.002838] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002894] ACPI: Early table checksum verification disabled worker1 # [ 0.002876] ACPI: Early table checksum verification disabled worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 client # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002896] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker1 # [ 0.002879] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 lb # [ 0.002900] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000000] kvm-clock: using sched offset of 494355129 cycles client # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002883] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040 lb # [ 0.002904] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002887] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.002853] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor lb # [ 0.002911] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002893] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000 lb # [ 0.002913] ACPI: FACS 0x000000005FFE0000 000040 worker1 # [ 0.002895] ACPI: FACS 0x000000005FFE0000 000040 lb # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002856] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000833] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs worker1 # [ 0.002897] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT client # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002916] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.002898] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002721] found SMP MP-table at [mem 0x000f5450-0x000f545f] client # [ 0.002859] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] lb # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002732] Using GB pages for direct mapping worker1 # [ 0.002900] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002825] RAMDISK: [mem 0x5e36f000-0x5ffcffff] client # [ 0.002860] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] lb # [ 0.002919] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker2 # [ 0.002834] ACPI: Early table checksum verification disabled worker1 # [ 0.002901] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] lb # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] worker2 # [ 0.002837] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) worker1 # [ 0.002903] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] client # [ 0.002861] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] lb # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.002904] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker2 # [ 0.002841] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] lb # [ 0.002922] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] worker1 # [ 0.002904] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] client # [ 0.002862] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker2 # [ 0.002845] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker1 # [ 0.002905] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] client # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] worker2 # [ 0.002851] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.003095] No NUMA configuration found lb # [ 0.002923] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker1 # [ 0.002905] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] worker2 # [ 0.002853] ACPI: FACS 0x000000005FFE0000 000040 client # [ 0.003096] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] lb # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker1 # [ 0.002906] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] worker2 # [ 0.002855] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.003099] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.005254] Zone ranges: worker1 # [ 0.002906] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] lb # [ 0.003160] No NUMA configuration found worker2 # [ 0.002856] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) worker1 # [ 0.003136] No NUMA configuration found client # [ 0.005254] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.003161] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] worker1 # [ 0.003137] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] client # [ 0.005256] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] worker2 # [ 0.002858] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) client # [ 0.005257] Normal empty lb # [ 0.003163] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] worker1 # [ 0.003140] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.005360] Zone ranges: client # [ 0.005258] Device empty worker1 # [ 0.005373] Zone ranges: client # [ 0.005258] Movable zone start for each node worker2 # [ 0.002859] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) lb # [ 0.005361] DMA [mem 0x0000000000001000-0x0000000000ffffff] client # [ 0.005259] Early memory node ranges worker1 # [ 0.005374] DMA [mem 0x0000000000001000-0x0000000000ffffff] worker2 # [ 0.002861] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d] lb # [ 0.005363] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.005259] node 0: [mem 0x0000000000001000-0x000000000009efff] worker1 # [ 0.005375] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] lb # [ 0.005364] Normal empty worker1 # [ 0.005376] Normal empty lb # [ 0.005365] Device empty worker2 # [ 0.002862] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219] worker1 # [ 0.005377] Device empty client # [ 0.005261] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] lb # [ 0.005365] Movable zone start for each node worker1 # [ 0.005377] Movable zone start for each node worker2 # [ 0.002862] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f] lb # [ 0.005366] Early memory node ranges worker1 # [ 0.005378] Early memory node ranges client # [ 0.005262] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] lb # [ 0.005366] node 0: [mem 0x0000000000001000-0x000000000009efff] client # [ 0.005282] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.002863] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385] worker1 # [ 0.005379] node 0: [mem 0x0000000000001000-0x000000000009efff] lb # [ 0.005367] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] client # [ 0.005547] On node 0, zone DMA: 97 pages in unavailable ranges worker1 # [ 0.005380] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] worker2 # [ 0.002863] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd] client # [ 0.030819] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.005368] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.005381] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker2 # [ 0.002864] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9] client # [ 0.031284] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.005392] On node 0, zone DMA: 1 pages in unavailable ranges worker1 # [ 0.005399] On node 0, zone DMA: 1 pages in unavailable ranges worker2 # [ 0.002864] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421] client # [ 0.031296] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) lb # [ 0.005653] On node 0, zone DMA: 97 pages in unavailable ranges worker2 # [ 0.003101] No NUMA configuration found worker1 # [ 0.005653] On node 0, zone DMA: 97 pages in unavailable ranges client # [ 0.031325] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.031191] On node 0, zone DMA32: 40 pages in unavailable ranges worker1 # [ 0.031082] On node 0, zone DMA32: 40 pages in unavailable ranges worker2 # [ 0.003102] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff] lb # [ 0.031651] ACPI: PM-Timer IO Port: 0x608 client # [ 0.031328] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker1 # [ 0.031551] ACPI: PM-Timer IO Port: 0x608 worker2 # [ 0.003104] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff] lb # [ 0.031663] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.005302] Zone ranges: client # [ 0.031329] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker1 # [ 0.031563] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker2 # [ 0.005303] DMA [mem 0x0000000000001000-0x0000000000ffffff] lb # [ 0.031692] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 client # [ 0.031331] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) worker1 # [ 0.031594] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.031694] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker2 # [ 0.005305] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff] client # [ 0.031331] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker1 # [ 0.031596] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) worker2 # [ 0.005306] Normal empty worker2 # [ 0.005307] Device empty lb # [ 0.031696] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) client # [ 0.031332] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) worker1 # [ 0.031598] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) worker2 # [ 0.005308] Movable zone start for each node lb # [ 0.031697] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) client # [ 0.031334] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.005308] Early memory node ranges worker1 # [ 0.031599] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) client # [ 0.031335] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker2 # [ 0.005309] node 0: [mem 0x0000000000001000-0x000000000009efff] lb # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) worker1 # [ 0.031600] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) client # [ 0.031338] TSC deadline timer available worker2 # [ 0.005310] node 0: [mem 0x0000000000100000-0x000000005ffd7fff] lb # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) client # [ 0.031342] CPU topo: Max. logical packages: 1 worker1 # [ 0.031601] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) client # [ 0.031343] CPU topo: Max. logical dies: 1 lb # [ 0.031701] ACPI: Using ACPI (MADT) for SMP configuration information worker2 # [ 0.005311] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff] worker1 # [ 0.031603] ACPI: Using ACPI (MADT) for SMP configuration information client # [ 0.031343] CPU topo: Max. dies per package: 1 lb # [ 0.031702] ACPI: HPET id: 0x8086a201 base: 0xfed00000 worker2 # [ 0.005332] On node 0, zone DMA: 1 pages in unavailable ranges worker1 # [ 0.031604] ACPI: HPET id: 0x8086a201 base: 0xfed00000 client # [ 0.031347] CPU topo: Max. threads per core: 1 lb # [ 0.031704] TSC deadline timer available worker1 # [ 0.031607] TSC deadline timer available worker2 # [ 0.005584] On node 0, zone DMA: 97 pages in unavailable ranges lb # [ 0.031709] CPU topo: Max. logical packages: 1 worker1 # [ 0.031611] CPU topo: Max. logical packages: 1 client # [ 0.031347] CPU topo: Num. cores per package: 1 worker2 # [ 0.030842] On node 0, zone DMA32: 40 pages in unavailable ranges lb # [ 0.031709] CPU topo: Max. logical dies: 1 client # [ 0.031348] CPU topo: Num. threads per package: 1 worker1 # [ 0.031612] CPU topo: Max. logical dies: 1 worker2 # [ 0.031311] ACPI: PM-Timer IO Port: 0x608 lb # [ 0.031709] CPU topo: Max. dies per package: 1 worker1 # [ 0.031612] CPU topo: Max. dies per package: 1 client # [ 0.031348] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs lb # [ 0.031713] CPU topo: Max. threads per core: 1 worker2 # [ 0.031322] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) worker1 # [ 0.031616] CPU topo: Max. threads per core: 1 client # [ 0.031365] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() lb # [ 0.031713] CPU topo: Num. cores per package: 1 worker1 # [ 0.031616] CPU topo: Num. cores per package: 1 worker2 # [ 0.031350] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 lb # [ 0.031714] CPU topo: Num. threads per package: 1 client # [ 0.031396] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.031617] CPU topo: Num. threads per package: 1 worker2 # [ 0.031353] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) lb # [ 0.031714] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker1 # [ 0.031617] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs client # [ 0.031397] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker2 # [ 0.031354] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) lb # [ 0.031732] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() worker1 # [ 0.031634] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.031399] [mem 0x60000000-0xafffffff] available for PCI devices worker2 # [ 0.031355] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) client # [ 0.031400] Booting paravirtualized kernel on KVM lb # [ 0.031763] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker1 # [ 0.031665] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] worker2 # [ 0.031356] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) lb # [ 0.031765] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] client # [ 0.031403] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.031667] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker2 # [ 0.031357] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) lb # [ 0.031766] [mem 0x60000000-0xafffffff] available for PCI devices worker1 # [ 0.031668] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.035866] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker2 # [ 0.031360] ACPI: Using ACPI (MADT) for SMP configuration information lb # [ 0.031768] Booting paravirtualized kernel on KVM worker1 # [ 0.031669] Booting paravirtualized kernel on KVM worker2 # [ 0.031361] ACPI: HPET id: 0x8086a201 base: 0xfed00000 client # [ 0.037949] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker2 # [ 0.031364] TSC deadline timer available client # [ 0.037996] kvm-guest: PV spinlocks disabled, single CPU lb # [ 0.031770] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker1 # [ 0.031672] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns worker2 # [ 0.031368] CPU topo: Max. logical packages: 1 worker2 # [ 0.031369] CPU topo: Max. logical dies: 1 lb # [ 0.036244] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker1 # [ 0.036147] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 worker2 # [ 0.031369] CPU topo: Max. dies per package: 1 lb # [ 0.038333] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker1 # [ 0.038229] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker2 # [ 0.031372] CPU topo: Max. threads per core: 1 lb # [ 0.038381] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.038281] kvm-guest: PV spinlocks disabled, single CPU worker2 # [ 0.031373] CPU topo: Num. cores per package: 1 worker2 # [ 0.031373] CPU topo: Num. threads per package: 1 client # [ 0.037997] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test/init regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration console=ttyS0,115200n8 console=tty0 worker2 # [ 0.031374] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs worker2 # [ 0.031393] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() client # [ 0.038099] Unknown kernel command line parameters "regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration", will be passed to user space. worker2 # [ 0.031426] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] client # [ 0.038112] random: crng init done lb # [ 0.038382] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n0idzb2a5795zcdcs33p9240n19lall0-nixos-system-lb-test/init regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration console=ttyS0,115200n8 console=tty0 worker1 # [ 0.038282] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgp1w89psmkzqxph18mrrd4l45zq8icp-nixos-system-worker1-test/init regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration console=ttyS0,115200n8 console=tty0 client # [ 0.038113] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker2 # [ 0.031427] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] worker2 # [ 0.031429] [mem 0x60000000-0xafffffff] available for PCI devices client # [ 0.040164] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.038474] Unknown kernel command line parameters "regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration", will be passed to user space. worker2 # [ 0.031430] Booting paravirtualized kernel on KVM worker1 # [ 0.038375] Unknown kernel command line parameters "regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration", will be passed to user space. lb # [ 0.038486] random: crng init done worker1 # [ 0.038387] random: crng init done client # [ 0.041337] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.038487] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes client # [ 0.041368] Fallback order for Node 0: 0 worker1 # [ 0.038388] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes worker2 # [ 0.031433] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns client # [ 0.041371] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.040614] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.035916] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 client # [ 0.041371] Policy zone: DMA32 worker1 # [ 0.040453] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.041801] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.043776] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.038171] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152 worker1 # [ 0.041498] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) lb # [ 0.041830] Fallback order for Node 0: 0 worker2 # [ 0.038215] kvm-guest: PV spinlocks disabled, single CPU worker1 # [ 0.041528] Fallback order for Node 0: 0 client # [ 0.046749] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.041832] Built 1 zonelists, mobility grouping on. Total pages: 393078 client # [ 0.049947] allocated 3145728 bytes of page_ext worker1 # [ 0.041531] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.041833] Policy zone: DMA32 worker1 # [ 0.041532] Policy zone: DMA32 client # [ 0.059788] ftrace: allocating 48787 entries in 192 pages lb # [ 0.044268] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.059789] ftrace: allocated 192 pages with 2 groups worker1 # [ 0.044081] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.060596] Dynamic Preempt: lazy lb # [ 0.047305] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.047092] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.060759] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.050535] allocated 3145728 bytes of page_ext worker1 # [ 0.050299] allocated 3145728 bytes of page_ext client # [ 0.060759] rcu: RCU event tracing is enabled. worker2 # [ 0.038217] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kknbsz59pyihvz42vvc4xnin6z1984g8-nixos-system-worker2-test/init regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration console=ttyS0,115200n8 console=tty0 lb # [ 0.060370] ftrace: allocating 48787 entries in 192 pages worker1 # [ 0.060241] ftrace: allocating 48787 entries in 192 pages lb # [ 0.060372] ftrace: allocated 192 pages with 2 groups client # [ 0.060760] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.060243] ftrace: allocated 192 pages with 2 groups lb # [ 0.061217] Dynamic Preempt: lazy client # [ 0.060761] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.061056] Dynamic Preempt: lazy worker2 # [ 0.038310] Unknown kernel command line parameters "regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration", will be passed to user space. client # [ 0.060762] Rude variant of Tasks RCU enabled. worker1 # [ 0.061215] rcu: Preemptible hierarchical RCU implementation. lb # [ 0.061368] rcu: Preemptible hierarchical RCU implementation. worker2 # [ 0.038327] random: crng init done client # [ 0.060762] Tracing variant of Tasks RCU enabled. lb # [ 0.061369] rcu: RCU event tracing is enabled. worker1 # [ 0.061216] rcu: RCU event tracing is enabled. worker2 # [ 0.038327] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes lb # [ 0.061370] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.060763] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker1 # [ 0.061217] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.061371] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.061218] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.040391] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.060763] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.061372] Rude variant of Tasks RCU enabled. worker1 # [ 0.061219] Rude variant of Tasks RCU enabled. lb # [ 0.061372] Tracing variant of Tasks RCU enabled. worker1 # [ 0.061219] Tracing variant of Tasks RCU enabled. worker2 # [ 0.041413] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.060783] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.041443] Fallback order for Node 0: 0 lb # [ 0.061373] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. worker1 # [ 0.061219] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. client # [ 0.060784] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.041447] Built 1 zonelists, mobility grouping on. Total pages: 393078 lb # [ 0.061373] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.041448] Policy zone: DMA32 worker1 # [ 0.061220] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.060785] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.043835] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.061389] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.061241] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.065109] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker2 # [ 0.046773] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.061390] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.065550] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.050111] allocated 3145728 bytes of page_ext worker1 # [ 0.061242] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.059754] ftrace: allocating 48787 entries in 192 pages lb # [ 0.061391] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.061243] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.065556] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.059756] ftrace: allocated 192 pages with 2 groups lb # [ 0.065722] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker2 # [ 0.060563] Dynamic Preempt: lazy worker1 # [ 0.065627] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker2 # [ 0.060683] rcu: Preemptible hierarchical RCU implementation. client # [ 0.065657] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.066208] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.065901] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.069222] Console: colour VGA+ 80x25 worker2 # [ 0.060684] rcu: RCU event tracing is enabled. client # [ 0.069225] printk: legacy console [tty0] enabled lb # [ 0.066215] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.060685] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.098659] printk: legacy console [ttyS0] enabled worker1 # [ 0.065908] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker2 # [ 0.060686] Trampoline variant of Tasks RCU enabled. client # [ 0.193319] ACPI: Core revision 20250807 lb # [ 0.066314] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.060686] Rude variant of Tasks RCU enabled. worker1 # [ 0.066010] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.069838] Console: colour VGA+ 80x25 worker1 # [ 0.069531] Console: colour VGA+ 80x25 worker2 # [ 0.060687] Tracing variant of Tasks RCU enabled. lb # [ 0.069841] printk: legacy console [tty0] enabled worker1 # [ 0.069533] printk: legacy console [tty0] enabled client # [ 0.194204] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker2 # [ 0.060687] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. lb # [ 0.099400] printk: legacy console [ttyS0] enabled worker1 # [ 0.098955] printk: legacy console [ttyS0] enabled client # [ 0.195695] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.060688] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.194168] ACPI: Core revision 20250807 lb # [ 0.194785] ACPI: Core revision 20250807 client # [ 0.196624] x2apic enabled worker2 # [ 0.060743] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.195653] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns worker1 # [ 0.195037] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns client # [ 0.197367] APIC: Switched APIC routing to: physical x2apic worker2 # [ 0.060745] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.197160] APIC: Switch to symmetric I/O mode setup worker1 # [ 0.196524] APIC: Switch to symmetric I/O mode setup worker2 # [ 0.060745] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.198154] x2apic enabled worker1 # [ 0.197469] x2apic enabled client # [ 0.199175] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker2 # [ 0.065045] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16 worker1 # [ 0.198198] APIC: Switched APIC routing to: physical x2apic lb # [ 0.198926] APIC: Switched APIC routing to: physical x2apic worker2 # [ 0.065314] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.200125] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.065321] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns worker1 # [ 0.200020] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 lb # [ 0.200745] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 client # [ 0.201718] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker2 # [ 0.065418] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.068920] Console: colour VGA+ 80x25 client # [ 0.202813] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker1 # [ 0.200954] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.201708] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.068923] printk: legacy console [tty0] enabled client # [ 0.203859] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker2 # [ 0.098340] printk: legacy console [ttyS0] enabled worker1 # [ 0.202563] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) lb # [ 0.203351] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) worker2 # [ 0.195157] ACPI: Core revision 20250807 client # [ 0.204715] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 worker1 # [ 0.203648] x86/cpu: User Mode Instruction Prevention (UMIP) activated lb # [ 0.204436] x86/cpu: User Mode Instruction Prevention (UMIP) activated worker2 # [ 0.196066] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns client # [ 0.205718] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker1 # [ 0.204736] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 lb # [ 0.205523] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 worker2 # [ 0.197586] APIC: Switch to symmetric I/O mode setup worker1 # [ 0.205561] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 lb # [ 0.207014] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 client # [ 0.206715] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.198538] x2apic enabled client # [ 0.207715] Transient Scheduler Attacks: Vulnerable: No microcode worker2 # [ 0.199307] APIC: Switched APIC routing to: physical x2apic worker1 # [ 0.206564] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto lb # [ 0.207351] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto client # [ 0.208715] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker1 # [ 0.207560] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl client # [ 0.209715] Speculative Return Stack Overflow: Mitigation: Safe RET lb # [ 0.209348] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker2 # [ 0.201097] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 worker1 # [ 0.208560] Transient Scheduler Attacks: Vulnerable: No microcode lb # [ 0.210348] Transient Scheduler Attacks: Vulnerable: No microcode worker1 # [ 0.209560] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker2 # [ 0.202057] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns lb # [ 0.211347] Spectre V2 : Mitigation: Enhanced / Automatic IBRS worker1 # [ 0.210560] Speculative Return Stack Overflow: Mitigation: Safe RET lb # [ 0.212348] Speculative Return Stack Overflow: Mitigation: Safe RET worker2 # [ 0.203705] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) client # [ 0.210715] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker2 # [ 0.204789] x86/cpu: User Mode Instruction Prevention (UMIP) activated client # [ 0.212722] Spectre V2 : Enabling IBPB for BPF worker2 # [ 0.205873] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 client # [ 0.213440] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.207422] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 client # [ 0.214716] active return thunk: srso_alias_return_thunk worker1 # [ 0.211560] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization lb # [ 0.213347] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization worker1 # [ 0.213568] Spectre V2 : Enabling IBPB for BPF lb # [ 0.215355] Spectre V2 : Enabling IBPB for BPF client # [ 0.215546] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.207705] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto worker1 # [ 0.214282] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier client # [ 0.216715] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' lb # [ 0.216348] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier worker2 # [ 0.209702] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl worker1 # [ 0.215561] active return thunk: srso_alias_return_thunk client # [ 0.217715] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' lb # [ 0.217349] active return thunk: srso_alias_return_thunk worker2 # [ 0.210702] Transient Scheduler Attacks: Vulnerable: No microcode worker1 # [ 0.216581] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' client # [ 0.218641] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' lb # [ 0.218369] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker2 # [ 0.211702] Spectre V2 : Mitigation: Enhanced / Automatic IBRS client # [ 0.219439] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker1 # [ 0.217560] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' lb # [ 0.219348] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker2 # [ 0.212702] Speculative Return Stack Overflow: Mitigation: Safe RET worker1 # [ 0.218560] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' client # [ 0.220430] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' lb # [ 0.220347] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker1 # [ 0.219560] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' client # [ 0.221462] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' lb # [ 0.221347] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' worker1 # [ 0.220560] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' client # [ 0.222715] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 lb # [ 0.222347] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker1 # [ 0.221560] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' client # [ 0.223624] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 lb # [ 0.224095] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.213702] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization client # [ 0.224414] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker1 # [ 0.222560] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.215709] Spectre V2 : Enabling IBPB for BPF lb # [ 0.225110] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' client # [ 0.225411] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker1 # [ 0.223560] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 lb # [ 0.226348] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker2 # [ 0.216703] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier client # [ 0.225715] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker1 # [ 0.224560] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 lb # [ 0.227257] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.217702] active return thunk: srso_alias_return_thunk worker1 # [ 0.225560] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 client # [ 0.226715] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. lb # [ 0.228036] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker2 # [ 0.218723] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' worker1 # [ 0.226561] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 lb # [ 0.229041] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker2 # [ 0.219702] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' worker1 # [ 0.227560] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 lb # [ 0.230065] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.220702] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' worker1 # [ 0.228560] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.221702] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' lb # [ 0.231067] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. worker2 # [ 0.222702] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' worker2 # [ 0.223702] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' worker2 # [ 0.224702] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' worker2 # [ 0.225702] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 worker2 # [ 0.227463] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 worker2 # [ 0.228458] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 worker2 # [ 0.229463] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 worker2 # [ 0.230460] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 worker2 # [ 0.231469] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. client # [ 0.260428] Freeing SMP alternatives memory: 44K client # [ 0.260718] pid_max: default: 32768 minimum: 301 client # [ 0.261872] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.262135] Freeing SMP alternatives memory: 44K client # [ 0.262821] landlock: Up and running. client # [ 0.263716] Yama: becoming mindful. worker1 # [ 0.262563] pid_max: default: 32768 minimum: 301 lb # [ 0.265380] Freeing SMP alternatives memory: 44K client # [ 0.264499] LSM support for eBPF active worker1 # [ 0.263876] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.266176] pid_max: default: 32768 minimum: 301 worker1 # [ 0.264657] landlock: Up and running. client # [ 0.265263] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.266971] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.265561] Yama: becoming mindful. lb # [ 0.267448] landlock: Up and running. client # [ 0.265756] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.266569] LSM support for eBPF active lb # [ 0.268349] Yama: becoming mindful. lb # [ 0.269564] LSM support for eBPF active worker1 # [ 0.267301] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.268627] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) lb # [ 0.270422] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.268495] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.269202] Performance Events: Fam17h+ core perfctr, AMD PMU driver. lb # [ 0.271383] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.269719] ... version: 2 worker1 # [ 0.270747] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.270392] ... bit width: 48 client # [ 0.270717] ... generic counters: 6 worker2 # [ 0.265396] Freeing SMP alternatives memory: 44K lb # [ 0.273900] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) client # [ 0.271394] ... generic bitmap: 000000000000003f worker1 # [ 0.272102] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker2 # [ 0.265705] pid_max: default: 32768 minimum: 301 worker1 # [ 0.272570] ... version: 2 client # [ 0.271760] ... fixed-purpose counters: 0 lb # [ 0.274879] Performance Events: Fam17h+ core perfctr, AMD PMU driver. worker1 # [ 0.273258] ... bit width: 48 worker2 # [ 0.266815] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.272448] ... fixed-purpose bitmap: 0000000000000000 lb # [ 0.275352] ... version: 2 worker1 # [ 0.273562] ... generic counters: 6 worker2 # [ 0.267806] landlock: Up and running. client # [ 0.272718] ... value mask: 0000ffffffffffff lb # [ 0.276061] ... bit width: 48 worker2 # [ 0.268703] Yama: becoming mindful. worker1 # [ 0.274259] ... generic bitmap: 000000000000003f client # [ 0.273568] ... max period: 00007fffffffffff lb # [ 0.276350] ... generic counters: 6 worker1 # [ 0.274563] ... fixed-purpose counters: 0 worker2 # [ 0.269496] LSM support for eBPF active client # [ 0.274353] ... global_ctrl mask: 000000000000003f lb # [ 0.277017] ... generic bitmap: 000000000000003f worker1 # [ 0.275232] ... fixed-purpose bitmap: 0000000000000000 client # [ 0.274840] signal: max sigframe size: 3376 worker2 # [ 0.270266] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.277362] ... fixed-purpose counters: 0 worker1 # [ 0.275563] ... value mask: 0000ffffffffffff client # [ 0.275573] rcu: Hierarchical SRCU implementation. lb # [ 0.278068] ... fixed-purpose bitmap: 0000000000000000 worker1 # [ 0.276426] ... max period: 00007fffffffffff worker2 # [ 0.270738] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.276289] rcu: Max phase no-delay instances is 400. lb # [ 0.278350] ... value mask: 0000ffffffffffff worker1 # [ 0.277214] ... global_ctrl mask: 000000000000003f lb # [ 0.279232] ... max period: 00007fffffffffff worker1 # [ 0.277674] signal: max sigframe size: 3376 lb # [ 0.280071] ... global_ctrl mask: 000000000000003f worker2 # [ 0.273666] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) worker1 # [ 0.278464] rcu: Hierarchical SRCU implementation. lb # [ 0.280463] signal: max sigframe size: 3376 worker1 # [ 0.279114] rcu: Max phase no-delay instances is 400. worker2 # [ 0.274203] Performance Events: Fam17h+ core perfctr, AMD PMU driver. lb # [ 0.281263] rcu: Hierarchical SRCU implementation. worker2 # [ 0.274707] ... version: 2 lb # [ 0.281928] rcu: Max phase no-delay instances is 400. worker2 # [ 0.275414] ... bit width: 48 client # [ 0.281177] smp: Bringing up secondary CPUs ... client # [ 0.281731] smp: Brought up 1 node, 1 CPU worker2 # [ 0.275705] ... generic counters: 6 worker2 # [ 0.276425] ... generic bitmap: 000000000000003f client # [ 0.282387] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.276752] ... fixed-purpose counters: 0 worker1 # [ 0.284075] smp: Bringing up secondary CPUs ... worker2 # [ 0.277469] ... fixed-purpose bitmap: 0000000000000000 worker1 # [ 0.284576] smp: Brought up 1 node, 1 CPU client # [ 0.282913] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker2 # [ 0.277705] ... value mask: 0000ffffffffffff lb # [ 0.286868] smp: Bringing up secondary CPUs ... client # [ 0.283967] devtmpfs: initialized worker1 # [ 0.285229] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.278587] ... max period: 00007fffffffffff lb # [ 0.287364] smp: Brought up 1 node, 1 CPU client # [ 0.284821] x86/mm: Memory block size: 128MB worker2 # [ 0.279362] ... global_ctrl mask: 000000000000003f lb # [ 0.288060] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) worker2 # [ 0.279815] signal: max sigframe size: 3376 worker1 # [ 0.285768] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved) worker2 # [ 0.280592] rcu: Hierarchical SRCU implementation. client # [ 0.286509] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.286820] devtmpfs: initialized worker2 # [ 0.281287] rcu: Max phase no-delay instances is 400. lb # [ 0.288547] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved) worker1 # [ 0.287651] x86/mm: Memory block size: 128MB client # [ 0.287592] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.289611] devtmpfs: initialized client # [ 0.288794] pinctrl core: initialized pinctrl subsystem lb # [ 0.290466] x86/mm: Memory block size: 128MB client # [ 0.289868] PM: RTC time: 05:32:52, date: 2026-09-23 worker1 # [ 0.289371] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.290461] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.292202] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.291649] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.286213] smp: Bringing up secondary CPUs ... lb # [ 0.293302] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.286717] smp: Brought up 1 node, 1 CPU worker1 # [ 0.292711] PM: RTC time: 05:32:52, date: 2026-09-23 client # [ 0.293325] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.294431] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.287371] smpboot: Total of 1 processors activated (4800.02 BogoMIPS) lb # [ 0.295530] PM: RTC time: 05:32:52, date: 2026-09-23 client # [ 0.294201] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.294738] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.287828] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved) worker1 # [ 0.296078] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.288958] devtmpfs: initialized client # [ 0.295986] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.289820] x86/mm: Memory block size: 128MB worker1 # [ 0.297027] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.299019] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.296727] audit: initializing netlink subsys (disabled) worker1 # [ 0.297583] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.297719] audit: type=2000 audit(1790141573.148:1): state=initialized audit_enabled=0 res=1 lb # [ 0.299846] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.291554] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.298840] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.298831] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.300378] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.292614] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.299573] audit: initializing netlink subsys (disabled) client # [ 0.298842] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.293793] pinctrl core: initialized pinctrl subsystem client # [ 0.299718] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.301625] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.300715] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.294878] PM: RTC time: 05:32:52, date: 2026-09-23 lb # [ 0.302360] audit: initializing netlink subsys (disabled) client # [ 0.300618] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.300717] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.301395] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.303467] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.301564] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.301730] cpuidle: using governor menu lb # [ 0.303469] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.302478] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.304351] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.303735] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.303283] audit: type=2000 audit(1790141573.152:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.298492] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.305326] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.304565] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.304943] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker1 # [ 0.304579] cpuidle: using governor menu worker2 # [ 0.299926] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.306096] audit: type=2000 audit(1790141573.153:1): state=initialized audit_enabled=0 res=1 client # [ 0.305720] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry lb # [ 0.307353] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.300723] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.306729] PCI: Using configuration type 1 for base access worker1 # [ 0.306756] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 lb # [ 0.307366] cpuidle: using governor menu worker2 # [ 0.301974] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.307796] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker1 # [ 0.307806] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] lb # [ 0.309599] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker2 # [ 0.302715] audit: initializing netlink subsys (disabled) worker1 # [ 0.308566] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker1 # [ 0.309575] PCI: Using configuration type 1 for base access worker2 # [ 0.303706] audit: type=2000 audit(1790141573.161:1): state=initialized audit_enabled=0 res=1 lb # [ 0.310601] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] lb # [ 0.311353] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.304878] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.310635] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. lb # [ 0.312361] PCI: Using configuration type 1 for base access worker2 # [ 0.304880] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.305705] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.313487] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker2 # [ 0.306663] thermal_sys: Registered thermal governor 'user_space' client # [ 0.314732] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.307463] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.307717] cpuidle: using governor menu client # [ 0.315717] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.315783] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.316703] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.309935] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 worker1 # [ 0.316564] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.310943] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] worker2 # [ 0.311707] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry worker2 # [ 0.312716] PCI: Using configuration type 1 for base access lb # [ 0.320612] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.321718] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.321351] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.313809] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. worker1 # [ 0.321564] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.322562] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.326352] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.327351] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.320719] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.330046] ACPI: Added _OSI(Module Device) worker2 # [ 0.321705] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.330720] ACPI: Added _OSI(Processor Device) client # [ 0.331422] ACPI: Added _OSI(Processor Aggregator Device) worker1 # [ 0.332914] ACPI: Added _OSI(Module Device) worker1 # [ 0.333565] ACPI: Added _OSI(Processor Device) worker1 # [ 0.334330] ACPI: Added _OSI(Processor Aggregator Device) worker2 # [ 0.326707] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.335948] ACPI: Added _OSI(Module Device) worker2 # [ 0.327705] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.336352] ACPI: Added _OSI(Processor Device) lb # [ 0.337105] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.339247] ACPI: 1 ACPI AML tables successfully acquired and loaded client # [ 0.343130] ACPI: Interpreter enabled client # [ 0.343734] ACPI: PM: (supports S0 S3 S4 S5) worker1 # [ 0.343103] ACPI: 1 ACPI AML tables successfully acquired and loaded client # [ 0.344431] ACPI: Using IOAPIC for interrupt routing lb # [ 0.343860] ACPI: 1 ACPI AML tables successfully acquired and loaded worker1 # [ 0.344452] ACPI: Interpreter enabled worker1 # [ 0.345069] ACPI: PM: (supports S0 S3 S4 S5) worker2 # [ 0.338946] ACPI: Added _OSI(Module Device) worker2 # [ 0.339665] ACPI: Added _OSI(Processor Device) client # [ 0.346757] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug lb # [ 0.347253] ACPI: Interpreter enabled worker2 # [ 0.340250] ACPI: Added _OSI(Processor Aggregator Device) client # [ 0.347718] PCI: Using E820 reservations for host bridge windows lb # [ 0.347812] ACPI: PM: (supports S0 S3 S4 S5) worker1 # [ 0.347563] ACPI: Using IOAPIC for interrupt routing lb # [ 0.348351] ACPI: Using IOAPIC for interrupt routing worker1 # [ 0.348448] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug lb # [ 0.349258] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.350800] ACPI: Enabled 2 GPEs in block 00 to 3F worker1 # [ 0.351563] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.352683] ACPI: Enabled 2 GPEs in block 00 to 3F worker2 # [ 0.346069] ACPI: 1 ACPI AML tables successfully acquired and loaded lb # [ 0.354351] PCI: Using E820 reservations for host bridge windows lb # [ 0.355490] ACPI: Enabled 2 GPEs in block 00 to 3F worker2 # [ 0.350118] ACPI: Interpreter enabled worker2 # [ 0.350721] ACPI: PM: (supports S0 S3 S4 S5) client # [ 0.358474] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) worker2 # [ 0.351431] ACPI: Using IOAPIC for interrupt routing client # [ 0.358724] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.359797] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.361294] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.360839] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker2 # [ 0.353780] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug client # [ 0.362076] PCI host bridge to bus 0000:00 worker2 # [ 0.354705] PCI: Using E820 reservations for host bridge windows worker1 # [ 0.362130] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.362722] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.364249] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) client # [ 0.363718] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker1 # [ 0.362642] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker2 # [ 0.357801] ACPI: Enabled 2 GPEs in block 00 to 3F client # [ 0.364705] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] worker1 # [ 0.363685] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] lb # [ 0.365465] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] client # [ 0.365588] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker1 # [ 0.364944] PCI host bridge to bus 0000:00 lb # [ 0.366431] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.365567] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] client # [ 0.366587] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] lb # [ 0.367474] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] worker1 # [ 0.366563] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.367587] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] lb # [ 0.368733] PCI host bridge to bus 0000:00 client # [ 0.368598] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.367563] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] lb # [ 0.369355] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] worker1 # [ 0.368563] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] client # [ 0.369453] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.370351] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] worker1 # [ 0.369563] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.364947] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) lb # [ 0.371351] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.371112] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker1 # [ 0.370563] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] lb # [ 0.372350] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker2 # [ 0.365711] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] worker1 # [ 0.371563] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.373350] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.366793] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] worker1 # [ 0.372544] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.374350] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker2 # [ 0.367827] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] lb # [ 0.375351] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.374010] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker2 # [ 0.369063] PCI host bridge to bus 0000:00 worker2 # [ 0.369709] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] lb # [ 0.376390] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.370705] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] client # [ 0.373887] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] lb # [ 0.377792] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint client # [ 0.374731] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] worker2 # [ 0.371705] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] client # [ 0.375657] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker2 # [ 0.372704] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window] worker2 # [ 0.373705] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] client # [ 0.376821] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] worker2 # [ 0.374705] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.376623] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker2 # [ 0.375705] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.377575] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] client # [ 0.378444] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.378536] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] worker2 # [ 0.376667] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.380436] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] client # [ 0.380737] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] worker2 # [ 0.378146] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint worker1 # [ 0.379626] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] lb # [ 0.381363] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] client # [ 0.381569] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] lb # [ 0.382330] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] client # [ 0.382443] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.381303] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.382724] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] lb # [ 0.383451] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] client # [ 0.384223] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.384581] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.385267] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.385445] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] worker1 # [ 0.386342] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker2 # [ 0.380769] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] worker1 # [ 0.387442] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] client # [ 0.386735] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker2 # [ 0.381718] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] lb # [ 0.388393] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] client # [ 0.387544] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.382668] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] lb # [ 0.389215] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] client # [ 0.388423] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.390044] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] worker1 # [ 0.388858] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.383608] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] lb # [ 0.390357] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] client # [ 0.389279] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.391509] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker2 # [ 0.385651] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.391917] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.392204] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] client # [ 0.391742] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker1 # [ 0.392584] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.392561] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker2 # [ 0.388724] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f] lb # [ 0.393390] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] worker1 # [ 0.394138] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.394206] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker2 # [ 0.389580] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] client # [ 0.393443] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker2 # [ 0.390438] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] lb # [ 0.395048] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] client # [ 0.394270] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.390711] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] lb # [ 0.395936] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.395726] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.396582] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker2 # [ 0.392272] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.396542] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] worker1 # [ 0.397570] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] client # [ 0.397440] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker1 # [ 0.398552] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] client # [ 0.397723] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] worker2 # [ 0.394603] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f] lb # [ 0.398393] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] worker2 # [ 0.395335] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] worker1 # [ 0.399982] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.400003] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] client # [ 0.399221] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.395726] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] lb # [ 0.400374] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker1 # [ 0.402434] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker2 # [ 0.397259] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.401900] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.400742] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker1 # [ 0.403174] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.401676] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] worker1 # [ 0.403584] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] lb # [ 0.403357] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] worker1 # [ 0.404569] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] lb # [ 0.404189] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.403097] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.405119] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] worker2 # [ 0.399722] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f] client # [ 0.404724] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.400712] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] worker1 # [ 0.406123] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.406246] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] client # [ 0.405640] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] worker2 # [ 0.401640] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref] worker1 # [ 0.407586] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] lb # [ 0.407925] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.407098] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.403141] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.408510] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] lb # [ 0.409363] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] worker1 # [ 0.409942] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.405441] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f] lb # [ 0.410372] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] client # [ 0.409074] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.405712] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] client # [ 0.410438] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker1 # [ 0.411588] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] worker2 # [ 0.406656] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref] lb # [ 0.411945] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.407508] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref] client # [ 0.411259] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.412531] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] lb # [ 0.413363] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] client # [ 0.412740] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] worker2 # [ 0.409681] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.414184] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.414372] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] client # [ 0.413664] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker1 # [ 0.415589] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker2 # [ 0.411717] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] lb # [ 0.415933] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.415062] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.416532] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] worker2 # [ 0.412652] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref] lb # [ 0.417363] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] worker1 # [ 0.417961] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.414113] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.418372] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] client # [ 0.417411] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] client # [ 0.418353] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker1 # [ 0.419587] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] lb # [ 0.419893] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.415751] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] client # [ 0.418743] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.420514] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.416727] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref] lb # [ 0.421363] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] client # [ 0.420268] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.422108] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.422320] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker2 # [ 0.418293] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.421726] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] client # [ 0.422606] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.423894] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.419753] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] client # [ 0.423444] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker1 # [ 0.424452] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker1 # [ 0.425173] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker2 # [ 0.420727] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref] lb # [ 0.425370] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker1 # [ 0.425584] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] client # [ 0.424287] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.426197] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] worker2 # [ 0.422273] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.427051] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.427117] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.425432] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.423717] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] client # [ 0.425913] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.427935] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.427360] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker2 # [ 0.424715] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref] worker1 # [ 0.429476] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] client # [ 0.427915] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.430210] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker2 # [ 0.427156] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.430351] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] worker1 # [ 0.430585] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] client # [ 0.429379] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] lb # [ 0.431267] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] lb # [ 0.432128] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] worker1 # [ 0.432129] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.429519] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf] worker2 # [ 0.430378] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] lb # [ 0.433848] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.433376] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.430726] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref] worker1 # [ 0.434431] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.435101] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] worker2 # [ 0.432284] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.436282] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] lb # [ 0.435546] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint lb # [ 0.437004] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] worker1 # [ 0.436753] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.434758] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f] lb # [ 0.437552] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint worker1 # [ 0.438355] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker2 # [ 0.435643] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff] worker2 # [ 0.436458] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref] lb # [ 0.440041] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] client # [ 0.429937] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.437258] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint worker2 # [ 0.438453] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f] client # [ 0.432753] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker2 # [ 0.438955] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint client # [ 0.433970] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker2 # [ 0.440433] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf] client # [ 0.435017] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO worker2 # [ 0.440898] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint client # [ 0.435969] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker2 # [ 0.442551] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df] worker1 # [ 0.439395] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.437750] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] lb # [ 0.440570] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.441280] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] client # [ 0.438587] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] lb # [ 0.443005] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] worker1 # [ 0.441849] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.439733] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint lb # [ 0.443633] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint worker1 # [ 0.442851] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.441417] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] lb # [ 0.444673] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO worker1 # [ 0.443824] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint lb # [ 0.445634] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker1 # [ 0.445591] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] client # [ 0.444772] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker1 # [ 0.446448] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.445807] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 lb # [ 0.447378] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker1 # [ 0.447629] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint lb # [ 0.448267] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] client # [ 0.446813] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 client # [ 0.447823] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker1 # [ 0.449225] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] worker2 # [ 0.443635] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.449440] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint client # [ 0.448828] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker2 # [ 0.445436] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff] lb # [ 0.451013] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.449817] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 client # [ 0.450818] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker2 # [ 0.445992] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint client # [ 0.451820] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.453137] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.452755] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.447209] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO client # [ 0.453687] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker1 # [ 0.453673] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 lb # [ 0.455100] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 client # [ 0.454434] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker2 # [ 0.447950] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint worker1 # [ 0.454660] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 lb # [ 0.455458] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 client # [ 0.454729] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.455663] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 client # [ 0.455660] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 lb # [ 0.456450] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker2 # [ 0.449776] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff] worker1 # [ 0.456666] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 client # [ 0.456428] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.457446] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker2 # [ 0.450641] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff] worker1 # [ 0.457667] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 client # [ 0.456748] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.458446] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 worker1 # [ 0.458670] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker2 # [ 0.451759] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint client # [ 0.457658] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 lb # [ 0.459447] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.459668] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 lb # [ 0.460458] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker1 # [ 0.460599] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker2 # [ 0.453413] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] client # [ 0.459567] iommu: Default domain type: Translated lb # [ 0.461450] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.461539] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 client # [ 0.460279] iommu: DMA domain TLB invalidation policy: lazy mode client # [ 0.460992] ACPI: bus type USB registered lb # [ 0.462397] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 worker1 # [ 0.462579] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 client # [ 0.461703] usbcore: registered new interface driver usbfs lb # [ 0.463366] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker1 # [ 0.463579] ACPI: PCI: Interrupt link GSID configured for IRQ 19 client # [ 0.462373] usbcore: registered new interface driver hub lb # [ 0.464364] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker1 # [ 0.464575] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 client # [ 0.462727] usbcore: registered new device driver usb worker2 # [ 0.457503] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 worker1 # [ 0.465574] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.465362] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker2 # [ 0.458561] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 worker1 # [ 0.466526] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.466352] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 client # [ 0.464480] NetLabel: Initializing client # [ 0.464719] NetLabel: domain hash size = 128 worker2 # [ 0.459572] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 worker1 # [ 0.467266] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 lb # [ 0.467333] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 client # [ 0.465417] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.468082] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 worker2 # [ 0.460535] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 worker1 # [ 0.468633] iommu: Default domain type: Translated client # [ 0.465776] NetLabel: unlabeled traffic allowed by default lb # [ 0.468363] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.461564] ACPI: PCI: Interrupt link LNKE configured for IRQ 10 client # [ 0.466731] PCI: Using ACPI for IRQ routing worker1 # [ 0.469382] iommu: DMA domain TLB invalidation policy: lazy mode worker2 # [ 0.462539] ACPI: PCI: Interrupt link LNKF configured for IRQ 10 worker1 # [ 0.470567] ACPI: bus type USB registered lb # [ 0.470319] iommu: Default domain type: Translated worker2 # [ 0.463687] ACPI: PCI: Interrupt link LNKG configured for IRQ 11 worker1 # [ 0.471293] usbcore: registered new interface driver usbfs lb # [ 0.470928] iommu: DMA domain TLB invalidation policy: lazy mode worker1 # [ 0.471577] usbcore: registered new interface driver hub lb # [ 0.471629] ACPI: bus type USB registered worker2 # [ 0.464515] ACPI: PCI: Interrupt link LNKH configured for IRQ 11 worker1 # [ 0.472425] usbcore: registered new device driver usb lb # [ 0.472358] usbcore: registered new interface driver usbfs worker2 # [ 0.465642] ACPI: PCI: Interrupt link GSIA configured for IRQ 16 lb # [ 0.473279] usbcore: registered new interface driver hub worker2 # [ 0.466554] ACPI: PCI: Interrupt link GSIB configured for IRQ 17 worker1 # [ 0.474123] NetLabel: Initializing lb # [ 0.474048] usbcore: registered new device driver usb worker1 # [ 0.474564] NetLabel: domain hash size = 128 worker2 # [ 0.467500] ACPI: PCI: Interrupt link GSIC configured for IRQ 18 worker1 # [ 0.475270] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO lb # [ 0.475190] NetLabel: Initializing worker2 # [ 0.468497] ACPI: PCI: Interrupt link GSID configured for IRQ 19 worker1 # [ 0.475618] NetLabel: unlabeled traffic allowed by default lb # [ 0.475787] NetLabel: domain hash size = 128 worker2 # [ 0.469477] ACPI: PCI: Interrupt link GSIE configured for IRQ 20 worker1 # [ 0.476492] PCI: Using ACPI for IRQ routing lb # [ 0.476351] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker2 # [ 0.469721] ACPI: PCI: Interrupt link GSIF configured for IRQ 21 lb # [ 0.477335] NetLabel: unlabeled traffic allowed by default worker2 # [ 0.470711] ACPI: PCI: Interrupt link GSIG configured for IRQ 22 lb # [ 0.478084] PCI: Using ACPI for IRQ routing worker2 # [ 0.471717] ACPI: PCI: Interrupt link GSIH configured for IRQ 23 worker2 # [ 0.473740] iommu: Default domain type: Translated worker2 # [ 0.474641] iommu: DMA domain TLB invalidation policy: lazy mode worker2 # [ 0.475762] ACPI: bus type USB registered worker2 # [ 0.476560] usbcore: registered new interface driver usbfs worker2 # [ 0.477440] usbcore: registered new interface driver hub worker2 # [ 0.477713] usbcore: registered new device driver usb worker2 # [ 0.479445] NetLabel: Initializing worker2 # [ 0.479710] NetLabel: domain hash size = 128 worker2 # [ 0.480484] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO worker2 # [ 0.480767] NetLabel: unlabeled traffic allowed by default worker2 # [ 0.481716] PCI: Using ACPI for IRQ routing client # [ 0.510163] pci 0000:00:01.0: vgaarb: setting as boot VGA device client # [ 0.510714] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.510714] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none client # [ 0.510722] vgaarb: loaded client # [ 0.511449] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.511717] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.514810] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.519910] pci 0000:00:01.0: vgaarb: setting as boot VGA device lb # [ 0.521402] pci 0000:00:01.0: vgaarb: setting as boot VGA device worker1 # [ 0.520559] pci 0000:00:01.0: vgaarb: bridge control possible lb # [ 0.522346] pci 0000:00:01.0: vgaarb: bridge control possible worker1 # [ 0.520559] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker1 # [ 0.520568] vgaarb: loaded client # [ 0.518267] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.521322] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.518947] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.522346] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker1 # [ 0.521565] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.520272] pnp: PnP ACPI init lb # [ 0.522356] vgaarb: loaded client # [ 0.521001] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.523169] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.521928] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.523996] hpet0: 3 comparators, 64-bit 100.000000 MHz counter client # [ 0.523439] pnp: PnP ACPI: found 5 devices worker1 # [ 0.524669] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.526397] VFS: Disk quotas dquot_6.6.0 lb # [ 0.527439] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.528943] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.530305] pnp: PnP ACPI init lb # [ 0.531015] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.531176] ACPI: IRQ 4 override to edge(!), high(!) worker2 # [ 0.526335] pci 0000:00:01.0: vgaarb: setting as boot VGA device lb # [ 0.531738] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.526701] pci 0000:00:01.0: vgaarb: bridge control possible client # [ 0.531011] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.532100] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved lb # [ 0.533016] pnp: PnP ACPI init worker2 # [ 0.526701] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none worker1 # [ 0.533564] pnp: PnP ACPI: found 5 devices client # [ 0.532396] clocksource: Switched to clocksource acpi_pm lb # [ 0.533776] ACPI: IRQ 4 override to edge(!), high(!) worker2 # [ 0.526710] vgaarb: loaded client # [ 0.533319] NET: Registered PF_INET protocol family lb # [ 0.534658] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker2 # [ 0.527471] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 client # [ 0.534429] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.536222] pnp: PnP ACPI: found 5 devices worker2 # [ 0.527705] hpet0: 3 comparators, 64-bit 100.000000 MHz counter worker2 # [ 0.531794] clocksource: Switched to clocksource kvm-clock worker1 # [ 0.541117] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker1 # [ 0.542616] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.543627] NET: Registered PF_INET protocol family lb # [ 0.543790] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns worker2 # [ 0.535327] VFS: Disk quotas dquot_6.6.0 lb # [ 0.545230] clocksource: Switched to clocksource acpi_pm worker1 # [ 0.544834] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.536061] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.546200] NET: Registered PF_INET protocol family worker2 # [ 0.537444] pnp: PnP ACPI init worker2 # [ 0.538247] ACPI: IRQ 4 override to edge(!), high(!) lb # [ 0.547389] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.539265] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved worker2 # [ 0.540864] pnp: PnP ACPI: found 5 devices client # [ 0.550405] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.551884] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.553141] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.554452] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.555649] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.556808] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.548491] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns client # [ 0.558086] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.550023] clocksource: Switched to clocksource acpi_pm worker2 # [ 0.550970] NET: Registered PF_INET protocol family client # [ 0.559215] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.560354] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.552127] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.561240] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.561266] NET: Registered PF_XDP protocol family client # [ 0.562037] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] worker1 # [ 0.562762] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.563000] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] lb # [ 0.563720] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.564066] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.563957] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.565288] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.565383] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.565001] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] worker1 # [ 0.566603] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.566577] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.566043] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] lb # [ 0.567914] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.567691] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.567082] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker1 # [ 0.568927] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.569175] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.568714] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker1 # [ 0.569997] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.570303] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.571195] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.571589] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.571294] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker1 # [ 0.572157] NET: Registered PF_XDP protocol family lb # [ 0.572705] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.572996] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] lb # [ 0.573912] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.573190] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.574022] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] lb # [ 0.574844] NET: Registered PF_XDP protocol family worker1 # [ 0.575018] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.575684] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] client # [ 0.575005] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.576114] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] lb # [ 0.576689] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] client # [ 0.576638] PCI: CLS 0 bytes, default 64 worker1 # [ 0.577222] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] lb # [ 0.577736] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] worker2 # [ 0.568216] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.578289] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] lb # [ 0.578767] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] client # [ 0.577489] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.569751] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.579788] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker1 # [ 0.579952] ACPI: \_SB_.GSIA: Enabled at IRQ 16 client # [ 0.579126] Trying to unpack rootfs image as initramfs... worker2 # [ 0.571034] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.580824] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker2 # [ 0.572404] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.582453] ACPI: \_SB_.GSIB: Enabled at IRQ 17 lb # [ 0.582746] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker2 # [ 0.573682] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.584388] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.574835] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.585195] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.576120] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.586289] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.577254] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.587093] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker1 # [ 0.587950] PCI: CLS 0 bytes, default 64 worker2 # [ 0.578504] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.579502] NET: Registered PF_XDP protocol family lb # [ 0.589112] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker1 # [ 0.588819] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.580372] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] lb # [ 0.590750] PCI: CLS 0 bytes, default 64 worker1 # [ 0.590487] Trying to unpack rootfs image as initramfs... worker2 # [ 0.581449] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] worker2 # [ 0.582529] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] lb # [ 0.591587] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.583661] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window] lb # [ 0.593387] Trying to unpack rootfs image as initramfs... worker2 # [ 0.584814] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] worker2 # [ 0.585987] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window] worker2 # [ 0.587765] ACPI: \_SB_.GSIA: Enabled at IRQ 16 worker2 # [ 0.590583] ACPI: \_SB_.GSIB: Enabled at IRQ 17 worker2 # [ 0.592519] ACPI: \_SB_.GSIC: Enabled at IRQ 18 worker2 # [ 0.594529] ACPI: \_SB_.GSID: Enabled at IRQ 19 worker2 # [ 0.596178] PCI: CLS 0 bytes, default 64 worker2 # [ 0.597074] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns worker2 # [ 0.598892] Trying to unpack rootfs image as initramfs... client # [ 0.623090] Initialise system trusted keyrings client # [ 0.626237] workingset: timestamp_bits=40 max_order=19 bucket_order=0 worker1 # [ 0.634575] Initialise system trusted keyrings worker1 # [ 0.637728] workingset: timestamp_bits=40 max_order=19 bucket_order=0 lb # [ 0.638255] Initialise system trusted keyrings lb # [ 0.641206] workingset: timestamp_bits=40 max_order=19 bucket_order=0 client # [ 0.645455] Key type asymmetric registered client # [ 0.648193] Asymmetric key parser 'x509' registered client # [ 0.649058] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker2 # [ 0.643790] Initialise system trusted keyrings client # [ 0.652288] io scheduler mq-deadline registered client # [ 0.653033] io scheduler kyber registered worker2 # [ 0.644738] workingset: timestamp_bits=40 max_order=19 bucket_order=0 client # [ 0.655765] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.657056] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.660691] Key type asymmetric registered worker1 # [ 0.661439] Asymmetric key parser 'x509' registered lb # [ 0.662418] Key type asymmetric registered worker1 # [ 0.662338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) client # [ 0.662948] Linux agpgart interface v0.103 client # [ 0.663750] ACPI: bus type drm_connector registered lb # [ 0.665167] Asymmetric key parser 'x509' registered worker1 # [ 0.665814] io scheduler mq-deadline registered lb # [ 0.666052] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) worker1 # [ 0.666546] io scheduler kyber registered client # [ 0.666653] usbcore: registered new interface driver usbserial_generic client # [ 0.667799] usbserial: USB Serial support registered for generic lb # [ 0.669261] io scheduler mq-deadline registered worker1 # [ 0.669248] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.670052] io scheduler kyber registered worker1 # [ 0.670538] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A lb # [ 0.672722] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.671203] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. client # [ 0.671203] Please enable it if your BIOS has the CPPC option. lb # [ 0.673998] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A client # [ 0.673438] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.675960] Linux agpgart interface v0.103 worker1 # [ 0.676751] ACPI: bus type drm_connector registered worker2 # [ 0.667608] Key type asymmetric registered client # [ 0.676301] drop_monitor: Initializing network drop monitor service worker2 # [ 0.668347] Asymmetric key parser 'x509' registered client # [ 0.677443] NET: Registered PF_INET6 protocol family worker2 # [ 0.669256] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) lb # [ 0.679770] Linux agpgart interface v0.103 worker1 # [ 0.679999] usbcore: registered new interface driver usbserial_generic lb # [ 0.680544] ACPI: bus type drm_connector registered client # [ 0.680636] Segment Routing with IPv6 worker1 # [ 0.681090] usbserial: USB Serial support registered for generic lb # [ 0.683593] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.674550] io scheduler mq-deadline registered client # [ 0.683191] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.675329] io scheduler kyber registered lb # [ 0.684694] usbserial: USB Serial support registered for generic client # [ 0.684216] IPI shorthand broadcast: enabled worker1 # [ 0.684691] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker1 # [ 0.684691] Please enable it if your BIOS has the CPPC option. worker1 # [ 0.686948] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker2 # [ 0.678934] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.688173] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. lb # [ 0.688173] Please enable it if your BIOS has the CPPC option. worker2 # [ 0.680209] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A worker1 # [ 0.689853] drop_monitor: Initializing network drop monitor service lb # [ 0.690421] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.691003] NET: Registered PF_INET6 protocol family lb # [ 0.693297] drop_monitor: Initializing network drop monitor service worker2 # [ 0.684229] Linux agpgart interface v0.103 client # [ 0.692579] sched_clock: Marking stable (560014811, 132158099)->(767518152, -75345242) worker1 # [ 0.694199] Segment Routing with IPv6 worker2 # [ 0.685000] ACPI: bus type drm_connector registered lb # [ 0.694476] NET: Registered PF_INET6 protocol family lb # [ 0.697602] Segment Routing with IPv6 worker1 # [ 0.697722] In-situ OAM (IOAM) with IPv6 client # [ 0.697424] registered taskstats version 1 worker1 # [ 0.698777] IPI shorthand broadcast: enabled client # [ 0.698395] Loading compiled-in X.509 certificates worker2 # [ 0.689810] usbcore: registered new interface driver usbserial_generic lb # [ 0.700187] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.691062] usbserial: USB Serial support registered for generic lb # [ 0.701232] IPI shorthand broadcast: enabled worker2 # [ 0.693426] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. worker2 # [ 0.693426] Please enable it if your BIOS has the CPPC option. worker2 # [ 0.695743] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled worker1 # [ 0.707087] sched_clock: Marking stable (574014956, 132651397)->(779452883, -72786530) lb # [ 0.708457] sched_clock: Marking stable (575015610, 133132214)->(781409903, -73262079) worker2 # [ 0.699552] drop_monitor: Initializing network drop monitor service worker2 # [ 0.700734] NET: Registered PF_INET6 protocol family worker1 # [ 0.711779] registered taskstats version 1 worker1 # [ 0.712778] Loading compiled-in X.509 certificates worker2 # [ 0.703819] Segment Routing with IPv6 lb # [ 0.713386] registered taskstats version 1 lb # [ 0.714363] Loading compiled-in X.509 certificates worker2 # [ 0.706442] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.707564] IPI shorthand broadcast: enabled client # [ 0.716186] Demotion targets for Node 0: null client # [ 0.717044] Key type .fscrypt registered client # [ 0.720178] Key type fscrypt-provisioning registered client # [ 0.721102] ima: No TPM chip found, activating TPM-bypass! client # [ 0.722045] ima: Allocated hash algorithm: sha1 client # [ 0.724208] ima: No architecture policies found worker2 # [ 0.715782] sched_clock: Marking stable (581015032, 134380829)->(789086384, -73690523) client # [ 0.725124] PM: Magic number: 10:752:517 client # [ 0.728751] RAS: Correctable Errors collector initialized. worker2 # [ 0.720505] registered taskstats version 1 worker1 # [ 0.730678] Demotion targets for Node 0: null worker2 # [ 0.721472] Loading compiled-in X.509 certificates worker1 # [ 0.731543] Key type .fscrypt registered lb # [ 0.733166] Demotion targets for Node 0: null lb # [ 0.734101] Key type .fscrypt registered worker1 # [ 0.734671] Key type fscrypt-provisioning registered worker1 # [ 0.735636] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.736542] ima: Allocated hash algorithm: sha1 lb # [ 0.737152] Key type fscrypt-provisioning registered lb # [ 0.738117] ima: No TPM chip found, activating TPM-bypass! client # [ 0.737382] clk: Disabling unused clocks worker1 # [ 0.738698] ima: No architecture policies found lb # [ 0.739058] ima: Allocated hash algorithm: sha1 client # [ 0.738045] PM: genpd: Disabling unused power domains worker1 # [ 0.739614] PM: Magic number: 10:752:517 lb # [ 0.741192] ima: No architecture policies found worker1 # [ 0.743231] RAS: Correctable Errors collector initialized. lb # [ 0.744167] PM: Magic number: 10:752:517 lb # [ 0.745733] RAS: Correctable Errors collector initialized. worker2 # [ 0.739410] Demotion targets for Node 0: null worker2 # [ 0.740334] Key type .fscrypt registered worker1 # [ 0.751932] clk: Disabling unused clocks worker2 # [ 0.743400] Key type fscrypt-provisioning registered worker1 # [ 0.752627] PM: genpd: Disabling unused power domains lb # [ 0.753849] clk: Disabling unused clocks worker2 # [ 0.744372] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.745335] ima: Allocated hash algorithm: sha1 worker2 # [ 0.747427] ima: No architecture policies found worker2 # [ 0.748433] PM: Magic number: 10:752:517 lb # [ 0.758166] PM: genpd: Disabling unused power domains worker2 # [ 0.750014] RAS: Correctable Errors collector initialized. worker2 # [ 0.761109] clk: Disabling unused clocks worker2 # [ 0.763410] PM: genpd: Disabling unused power domains client # [ 0.898523] Freeing initrd memory: 29056K client # [ 0.901706] Freeing unused decrypted memory: 2028K client # [ 0.904287] Freeing unused kernel image (initmem) memory: 3652K client # [ 0.905405] Write protecting the kernel read-only data: 32768k client # [ 0.907275] Freeing unused kernel image (text/rodata gap) memory: 1184K client # [ 0.908824] Freeing unused kernel image (rodata/data gap) memory: 720K worker1 # [ 0.912289] Freeing initrd memory: 29056K worker1 # [ 0.915486] Freeing unused decrypted memory: 2028K lb # [ 0.916258] Freeing initrd memory: 29064K worker1 # [ 0.918080] Freeing unused kernel image (initmem) memory: 3652K lb # [ 0.919416] Freeing unused decrypted memory: 2028K worker1 # [ 0.919179] Write protecting the kernel read-only data: 32768k worker1 # [ 0.921014] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.922004] Freeing unused kernel image (initmem) memory: 3652K worker1 # [ 0.922531] Freeing unused kernel image (rodata/data gap) memory: 720K lb # [ 0.923153] Write protecting the kernel read-only data: 32768k lb # [ 0.924991] Freeing unused kernel image (text/rodata gap) memory: 1184K lb # [ 0.926518] Freeing unused kernel image (rodata/data gap) memory: 720K worker2 # [ 0.922317] Freeing initrd memory: 29060K worker2 # [ 0.925567] Freeing unused decrypted memory: 2028K worker2 # [ 0.928126] Freeing unused kernel image (initmem) memory: 3652K worker2 # [ 0.929258] Write protecting the kernel read-only data: 32768k worker2 # [ 0.931125] Freeing unused kernel image (text/rodata gap) memory: 1184K worker2 # [ 0.932711] Freeing unused kernel image (rodata/data gap) memory: 720K client # [ 0.959504] x86/mm: Checked W+X mappings: passed, no W+X pages found. client # [ 0.960601] Run /init as init process client # [ 0.971108] systemd[1]: Inserted module 'autofs4' worker1 # [ 0.973478] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker1 # [ 0.974585] Run /init as init process lb # [ 0.977284] x86/mm: Checked W+X mappings: passed, no W+X pages found. lb # [ 0.978410] Run /init as init process worker1 # [ 0.985271] systemd[1]: Inserted module 'autofs4' lb # [ 0.989046] systemd[1]: Inserted module 'autofs4' client # [ 0.990817] fuse: init (API version 7.45) worker2 # [ 0.983490] x86/mm: Checked W+X mappings: passed, no W+X pages found. worker2 # [ 0.984609] Run /init as init process client # [ 0.998152] ACPI: \_SB_.GSIG: Enabled at IRQ 22 client # [ 1.000750] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker2 # [ 0.995209] systemd[1]: Inserted module 'autofs4' worker1 # [ 1.005271] fuse: init (API version 7.45) client # [ 1.004794] ACPI: \_SB_.GSIE: Enabled at IRQ 20 client # [ 1.006936] ACPI: \_SB_.GSIF: Enabled at IRQ 21 lb # [ 1.008792] fuse: init (API version 7.45) worker1 # [ 1.012780] ACPI: \_SB_.GSIG: Enabled at IRQ 22 client # [ 1.012495] virtiofs virtio5: discovered new tag: nix-store client # [ 1.014134] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker1 # [ 1.015276] ACPI: \_SB_.GSIH: Enabled at IRQ 23 lb # [ 1.016253] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker1 # [ 1.018375] ACPI: \_SB_.GSIE: Enabled at IRQ 20 lb # [ 1.018887] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker1 # [ 1.020546] ACPI: \_SB_.GSIF: Enabled at IRQ 21 lb # [ 1.022839] ACPI: \_SB_.GSIE: Enabled at IRQ 20 worker2 # [ 1.015243] fuse: init (API version 7.45) client # [ 1.023762] virtiofs virtio6: discovered new tag: shared lb # [ 1.024988] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.026313] virtiofs virtio5: discovered new tag: nix-store client # [ 1.025397] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 1.027872] virtiofs virtio5: virtio_fs_setup_dax: No cache capability lb # [ 1.030408] virtiofs virtio5: discovered new tag: nix-store client # [ 1.029617] virtiofs virtio7: discovered new tag: xchg client # [ 1.031031] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.032176] virtiofs virtio5: virtio_fs_setup_dax: No cache capability worker2 # [ 1.023369] ACPI: \_SB_.GSIG: Enabled at IRQ 22 worker2 # [ 1.026546] ACPI: \_SB_.GSIH: Enabled at IRQ 23 worker1 # [ 1.036274] virtiofs virtio6: discovered new tag: shared worker1 # [ 1.037895] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.030060] ACPI: \_SB_.GSIE: Enabled at IRQ 20 lb # [ 1.039803] virtiofs virtio6: discovered new tag: shared lb # [ 1.041337] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 1.041268] virtiofs virtio7: discovered new tag: xchg worker2 # [ 1.032870] ACPI: \_SB_.GSIF: Enabled at IRQ 21 worker1 # [ 1.042858] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.044732] virtiofs virtio7: discovered new tag: xchg lb # [ 1.046280] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 1.040093] virtiofs virtio5: discovered new tag: nix-store worker2 # [ 1.042063] virtiofs virtio5: virtio_fs_setup_dax: No cache capability client # [ 1.054019] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.051984] virtiofs virtio6: discovered new tag: shared worker2 # [ 1.053518] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 1.057129] virtiofs virtio7: discovered new tag: xchg worker1 # [ 1.066421] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.058622] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 1.068932] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.082159] systemd[1]: Successfully made /usr/ read-only. client # [ 1.389585] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.394794] systemd[1]: Detected virtualization kvm. client # [ 1.395691] systemd[1]: Detected architecture x86-64. client # [ 1.396600] systemd[1]: Running in initrd. client # [ 1.397698] systemd[1]: Initializing machine ID from random generator. client # [ 1.399003] systemd[1]: Hostname set to . worker1 # [ 1.403066] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.415169] systemd[1]: Detected virtualization kvm. lb # [ 1.405596] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.417332] systemd[1]: Detected architecture x86-64. lb # [ 1.417627] systemd[1]: Detected virtualization kvm. worker1 # [ 1.419493] systemd[1]: Running in initrd. lb # [ 1.419773] systemd[1]: Detected architecture x86-64. lb # [ 1.421918] systemd[1]: Running in initrd. worker1 # [ 1.421948] systemd[1]: Initializing machine ID from random generator. lb # [ 1.424391] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.424838] systemd[1]: Hostname set to . lb # [ 1.427328] systemd[1]: Hostname set to . worker2 # [ 1.418812] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.430929] systemd[1]: Detected virtualization kvm. worker2 # [ 1.433129] systemd[1]: Detected architecture x86-64. worker2 # [ 1.435348] systemd[1]: Running in initrd. worker2 # [ 1.437846] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.440935] systemd[1]: Hostname set to . client # [ 1.584487] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.621701] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.625466] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.626717] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.628183] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.629313] systemd[1]: Reached target Path Units. client # [ 1.630235] systemd[1]: Reached target Slice Units. client # [ 1.631093] systemd[1]: Reached target Swaps. client # [ 1.631933] systemd[1]: Reached target Timer Units. client # [ 1.632936] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.634213] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.635374] systemd[1]: Listening on Journal Sockets. client # [ 1.636363] systemd[1]: Listening on udev Control Socket. client # [ 1.637413] systemd[1]: Listening on udev Kernel Socket. client # [ 1.638371] systemd[1]: Reached target Socket Units. client # [ 1.640094] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.643982] systemd[1]: Starting Load Kernel Module configfs... worker1 # [ 1.646756] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.648290] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.652353] systemd[1]: Starting Journal Service... worker2 # [ 1.664541] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.677239] systemd[1]: Starting Load Kernel Modules... client # [ 1.681277] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.690243] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.699128] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.700899] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.701507] systemd-journald[66]: Collecting audit messages is disabled. worker1 # [ 1.702911] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.702972] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.704109] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.705399] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.705534] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.706671] systemd[1]: Reached target Path Units. lb # [ 1.706598] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.707504] systemd[1]: Reached target Slice Units. client # [ 1.706824] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 1.708003] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.708373] systemd[1]: Reached target Swaps. lb # [ 1.709062] systemd[1]: Reached target Path Units. worker1 # [ 1.709185] systemd[1]: Reached target Timer Units. lb # [ 1.709934] systemd[1]: Reached target Slice Units. worker1 # [ 1.710168] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.710814] systemd[1]: Reached target Swaps. lb # [ 1.711646] systemd[1]: Reached target Timer Units. worker1 # [ 1.711411] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.712639] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.712605] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.713589] systemd[1]: Listening on udev Control Socket. lb # [ 1.713869] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.714611] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.715079] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.715559] systemd[1]: Reached target Socket Units. lb # [ 1.716119] systemd[1]: Listening on udev Control Socket. client # [ 1.715553] systemd[1]: Finished Load Kernel Module configfs. lb # [ 1.717195] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.717361] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.718169] systemd[1]: Reached target Socket Units. lb # [ 1.719954] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.721736] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.721610] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config lb # [ 1.723961] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 1.717337] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.721698] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.722872] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.724405] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.733256] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.725537] systemd[1]: Reached target Path Units. worker2 # [ 1.726444] systemd[1]: Reached target Slice Units. worker2 # [ 1.727354] systemd[1]: Reached target Swaps. worker2 # [ 1.728197] systemd[1]: Reached target Timer Units. worker2 # [ 1.729195] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.730480] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.731729] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.732750] systemd[1]: Listening on udev Control Socket. lb # [ 1.730767] systemd[1]: Starting Journal Service... worker2 # [ 1.733875] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.731770] systemd[1]: Starting Journal Service... client # [ 1.741787] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.734814] systemd[1]: Reached target Socket Units. worker2 # [ 1.736564] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.740469] systemd[1]: Starting Load Kernel Module configfs... client # [ 1.752347] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.755230] systemd[1]: Starting Load Kernel Modules... lb # [ 1.758325] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.759791] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.762267] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.766623] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.766257] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.768920] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.749831] systemd[1]: Starting Journal Service... client # [ 1.776432] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.778990] systemd-journald[66]: Collecting audit messages is disabled. lb # [ 1.780798] systemd-journald[66]: Collecting audit messages is disabled. worker1 # [ 1.782964] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.773997] systemd[1]: Starting Load Kernel Modules... lb # [ 1.783598] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.786512] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker2 # [ 1.778527] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.789744] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 1.793055] systemd[1]: Finished Load Kernel Module configfs. client # [ 1.793231] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.787579] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.797625] systemd[1]: Finished Load Kernel Module configfs. worker1 # [ 1.799087] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config lb # [ 1.801561] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config client # [ 1.802359] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.798836] systemd-journald[66]: Collecting audit messages is disabled. worker1 # [ 1.808759] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.800519] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.814223] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.814317] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.805839] systemd[1]: modprobe@configfs.service: Deactivated successfully. lb # [ 1.819896] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.820409] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.821293] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.813756] systemd[1]: Finished Load Kernel Module configfs. client # [ 1.825273] systemd[1]: Reached target Local File Systems. worker2 # [ 1.818025] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config worker1 # [ 1.829825] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.831333] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.696807] systemd-modules-load[67]: Inserted module 'dm_mod' client # [ 1.700271] systemd-modules-load[67]: Inserted module 'virtio_balloon' client # [ 1.701236] systemd-modules-load[67]: Inserted module 'virtio_gpu' client # [ 1.835143] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.828575] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.839413] systemd[1]: Started Journal Service. client # [ 1.716174] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.851200] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.841610] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.853694] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.849591] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.859851] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.727549] systemd[1]: Starting Create System Files and Directories... lb # [ 1.862328] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.871747] systemd[1]: Finished Load Kernel Modules. lb # [ 1.874685] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.869960] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.879916] systemd[1]: Starting Apply Kernel Variables... lb # [ 1.883226] systemd[1]: Starting Apply Kernel Variables... client # [ 1.753299] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.877599] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.759581] systemd-udevd[74]: Using default interface naming scheme 'v261'. worker1 # [ 1.899216] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.892466] systemd[1]: Finished Load Kernel Modules. lb # [ 1.902103] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.770520] systemd-modules-load[67]: Inserted module 'dm_mod' worker1 # [ 1.771849] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker1 # [ 1.772954] systemd-modules-load[67]: Inserted module 'virtio_gpu' lb # [ 1.770614] systemd-modules-load[68]: Inserted module 'dm_mod' worker1 # [ 1.906604] systemd[1]: Started Journal Service. worker2 # [ 1.897616] systemd[1]: Starting Apply Kernel Variables... lb # [ 1.774266] systemd-modules-load[68]: Inserted module 'virtio_balloon' lb # [ 1.775364] systemd-modules-load[68]: Inserted module 'virtio_gpu' lb # [ 1.909546] systemd[1]: Started Journal Service. client # [ 1.782567] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.785109] systemd[1]: Finished Apply Kernel Variables. lb # [ 1.787764] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 1.788074] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.788984] systemd[1]: Reached target Local File Systems. lb # [ 1.790462] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.792067] systemd[1]: Starting Create System Files and Directories... lb # [ 1.792070] systemd[1]: Reached target Local File Systems. worker2 # [ 1.917602] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.794961] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.784776] systemd-modules-load[67]: Inserted module 'dm_mod' worker1 # [ 1.799070] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.788300] systemd-modules-load[67]: Inserted module 'virtio_balloon' worker2 # [ 1.789359] systemd-modules-load[67]: Inserted module 'virtio_gpu' lb # [ 1.802072] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.926169] systemd[1]: Started Journal Service. worker2 # [ 1.802168] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.804750] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.806087] systemd[1]: Reached target Local File Systems. worker2 # [ 1.809121] systemd[1]: Starting Create System Files and Directories... worker1 # [ 1.821338] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.813786] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.825159] systemd[1]: Finished Create System Files and Directories. worker1 # [ 1.842400] systemd-udevd[82]: Using default interface naming scheme 'v261'. client # [ 1.842713] systemd[1]: Finished Coldplug All udev Devices. client # [ 1.843672] systemd[1]: Reached target System Initialization. client # [ 1.844639] systemd[1]: Reached target Basic System. lb # [ 1.846869] systemd-udevd[81]: Using default interface naming scheme 'v261'. worker2 # [ 1.840322] systemd[1]: Finished Create System Files and Directories. worker1 # [ 1.866115] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 1.870164] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 1.860859] systemd-udevd[81]: Using default interface naming scheme 'v261'. worker2 # [ 1.885226] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 1.918795] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 1.919702] systemd[1]: Reached target System Initialization. worker1 # [ 1.920528] systemd[1]: Reached target Basic System. lb # [ 1.924678] systemd[1]: Finished Coldplug All udev Devices. lb # [ 1.925664] systemd[1]: Reached target System Initialization. lb # [ 1.926489] systemd[1]: Reached target Basic System. worker2 # [ 1.935133] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 1.938058] systemd[1]: Reached target System Initialization. worker2 # [ 1.938884] systemd[1]: Reached target Basic System. client # [ 2.156819] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.165528] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.168611] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.178300] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.179214] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 client # [ 2.180971] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 client # [ 2.185507] serio: i8042 KBD port at 0x60,0x64 irq 1 client # [ 2.186197] serio: i8042 AUX port at 0x60,0x64 irq 12 client # [ 2.189193] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 client # [ 2.190144] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.191594] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.193049] usb usb1: Product: EHCI Host Controller client # [ 2.193850] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd client # [ 2.196175] usb usb1: SerialNumber: 0000:00:1d.7 client # [ 2.197520] hub 1-0:1.0: USB hub found client # [ 2.199458] hub 1-0:1.0: 6 ports detected client # [ 2.206869] uhci_hcd 0000:00:1d.0: UHCI Host Controller client # [ 2.207744] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.222629] uhci_hcd 0000:00:1d.0: detected 2 ports client # [ 2.231639] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 lb # [ 2.240561] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 2.241432] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 2.240410] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.242114] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 2.241581] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.249810] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 worker1 # [ 2.249789] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.256227] usb usb2: Product: UHCI Host Controller client # [ 2.256913] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.258506] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 2.252163] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 2.265491] serio: i8042 KBD port at 0x60,0x64 irq 1 worker1 # [ 2.265296] serio: i8042 KBD port at 0x60,0x64 irq 1 lb # [ 2.266261] serio: i8042 AUX port at 0x60,0x64 irq 12 worker2 # [ 2.259320] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 client # [ 2.271628] usb usb2: SerialNumber: 0000:00:1d.0 worker1 # [ 2.272716] ehci-pci 0000:00:1d.7: EHCI Host Controller worker1 # [ 2.273629] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker2 # [ 2.264259] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 2.275384] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.276591] ehci-pci 0000:00:1d.7: EHCI Host Controller client # [ 2.143688] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.277372] serio: i8042 AUX port at 0x60,0x64 irq 12 lb # [ 2.277480] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 lb # [ 2.279304] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 worker2 # [ 2.270269] serio: i8042 KBD port at 0x60,0x64 irq 1 client # [ 2.281401] hub 2-0:1.0: USB hub found worker1 # [ 2.282688] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker2 # [ 2.273961] serio: i8042 AUX port at 0x60,0x64 irq 12 worker1 # [ 2.284061] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 lb # [ 2.285314] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker1 # [ 2.285235] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.152091] (udev-worker)[87]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.287772] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 worker1 # [ 2.288665] usb usb1: Product: EHCI Host Controller lb # [ 2.288931] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.289329] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.292066] usb usb1: SerialNumber: 0000:00:1d.7 lb # [ 2.293153] usb usb1: Product: EHCI Host Controller client # [ 2.292506] SCSI subsystem initialized lb # [ 2.293811] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker2 # [ 2.284912] ehci-pci 0000:00:1d.7: EHCI Host Controller worker1 # [ 2.294830] hub 1-0:1.0: USB hub found client # [ 2.293932] hub 2-0:1.0: 2 ports detected lb # [ 2.295052] usb usb1: SerialNumber: 0000:00:1d.7 worker2 # [ 2.285650] ehci-pci 0000:00:1d.7: new USB bus registered, assigned bus number 1 worker1 # [ 2.295362] hub 1-0:1.0: 6 ports detected lb # [ 2.296316] hub 1-0:1.0: USB hub found worker2 # [ 2.287369] ehci-pci 0000:00:1d.7: irq 19, io mem 0xfebdb000 lb # [ 2.297935] hub 1-0:1.0: 6 ports detected worker1 # [ 2.298311] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker1 # [ 2.299069] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.299282] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 worker2 # [ 2.293492] ehci-pci 0000:00:1d.7: USB 2.0 started, EHCI 1.00 worker2 # [ 2.294367] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.18 client # [ 2.171936] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.306958] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker2 # [ 2.297365] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.307833] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 worker2 # [ 2.299711] usb usb1: Product: EHCI Host Controller worker2 # [ 2.301055] usb usb1: Manufacturer: Linux 6.18.52 ehci_hcd worker1 # [ 2.310929] uhci_hcd 0000:00:1d.0: detected 2 ports worker2 # [ 2.302992] usb usb1: SerialNumber: 0000:00:1d.7 client # [ 2.179646] (udev-worker)[87]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.304490] hub 1-0:1.0: USB hub found worker2 # [ 2.307198] hub 1-0:1.0: 6 ports detected worker1 # [ 2.316426] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 client # [ 2.186976] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.189307] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.324177] SCSI subsystem initialized lb # [ 2.325390] uhci_hcd 0000:00:1d.0: detected 2 ports worker1 # [ 2.325980] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker2 # [ 2.319990] uhci_hcd 0000:00:1d.0: UHCI Host Controller worker2 # [ 2.320738] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 client # [ 2.331366] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.332056] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.334799] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.336385] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 client # [ 2.341258] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.333733] uhci_hcd 0000:00:1d.0: detected 2 ports lb # [ 2.344800] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.345370] usb usb2: Product: UHCI Host Controller lb # [ 2.347214] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.216970] (udev-worker)[86]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.341829] uhci_hcd 0000:00:1d.0: irq 16, io port 0x0000c180 client # [ 2.350555] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker1 # [ 2.353030] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.221737] systemd[1]: Starting Virtual Console Setup... client # [ 2.354140] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.224067] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 2.225043] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.226714] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.360203] usb usb2: Product: UHCI Host Controller lb # [ 2.360928] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.229068] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.353744] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.231041] systemd[1]: Starting Virtual Console Setup... client # [ 2.363193] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.232748] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.222147] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.367772] usb usb3: Product: UHCI Host Controller client # [ 2.368721] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.369581] usb usb3: SerialNumber: 0000:00:1d.1 worker1 # [ 2.239377] (udev-worker)[88]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.229645] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.241439] (udev-worker)[88]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.241482] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.376671] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.376450] hub 3-0:1.0: USB hub found worker2 # [ 2.233586] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.369191] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.378878] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.378346] hub 3-0:1.0: 2 ports detected lb # [ 2.380528] SCSI subsystem initialized worker2 # [ 2.372814] usb usb2: Product: UHCI Host Controller worker2 # [ 2.239406] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.385759] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.385650] uhci_hcd 0000:00:1d.2: UHCI Host Controller client # [ 2.387363] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 worker2 # [ 2.379496] SCSI subsystem initialized lb # [ 2.391313] hub 2-0:1.0: USB hub found client # [ 2.391213] uhci_hcd 0000:00:1d.2: detected 2 ports worker1 # [ 2.259241] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.392030] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.261620] systemd[1]: Finished Virtual Console Setup. lb # [ 2.395220] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.395460] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.397024] hub 2-0:1.0: USB hub found lb # [ 2.399278] hub 2-0:1.0: 2 ports detected client # [ 2.399200] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.401732] hub 2-0:1.0: 2 ports detected client # [ 2.401482] usb usb4: Product: UHCI Host Controller client # [ 2.403298] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.394542] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 client # [ 2.404086] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.265277] systemd-vconsole-setup[98]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.277340] systemd-vconsole-setup[99]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.278705] systemd-fsck[107]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.268365] systemd[1]: Finished Virtual Console Setup. lb # [ 2.279801] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.404962] usb usb2: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.405813] usb usb2: SerialNumber: 0000:00:1d.0 client # [ 2.284830] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.420758] uhci_hcd 0000:00:1d.1: UHCI Host Controller client # [ 2.420748] hub 4-0:1.0: USB hub found worker1 # [ 2.421456] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.422522] uhci_hcd 0000:00:1d.1: detected 2 ports client # [ 2.423792] hub 4-0:1.0: 2 ports detected worker2 # [ 2.415455] hub 2-0:1.0: USB hub found worker1 # [ 2.428827] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker2 # [ 2.420398] hub 2-0:1.0: 2 ports detected lb # [ 2.296904] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 2.298341] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.300774] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 2.302057] systemd[1]: Reached target Initrd Root Device. lb # [ 2.435275] uhci_hcd 0000:00:1d.1: UHCI Host Controller worker2 # [ 2.426902] uhci_hcd 0000:00:1d.1: UHCI Host Controller lb # [ 2.436015] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.435817] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.437090] uhci_hcd 0000:00:1d.1: detected 2 ports worker1 # [ 2.436961] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.304680] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 2.306059] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 2.439190] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.432468] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3 worker1 # [ 2.444111] usb usb3: Product: UHCI Host Controller worker1 # [ 2.445104] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.303875] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 2.304791] systemd[1]: Reached target Initrd Root Device. worker1 # [ 2.450697] usb usb3: SerialNumber: 0000:00:1d.1 client # [ 2.449632] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.307062] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 2.442531] uhci_hcd 0000:00:1d.1: detected 2 ports worker2 # [ 2.443342] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 lb # [ 2.453359] uhci_hcd 0000:00:1d.1: irq 17, io port 0x0000c1a0 worker1 # [ 2.453731] hub 3-0:1.0: USB hub found worker2 # [ 2.447556] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.458937] hub 3-0:1.0: 2 ports detected lb # [ 2.459214] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 client # [ 2.459433] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) lb # [ 2.460357] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.460307] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.462196] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.463192] usb usb3: Product: UHCI Host Controller lb # [ 2.463981] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.454423] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker1 # [ 2.464795] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 lb # [ 2.466279] usb usb3: SerialNumber: 0000:00:1d.1 lb # [ 2.468600] hub 3-0:1.0: USB hub found worker2 # [ 2.460129] usb usb3: Product: UHCI Host Controller client # [ 2.469258] scsi host0: ahci lb # [ 2.470212] hub 3-0:1.0: 2 ports detected worker2 # [ 2.461094] usb usb3: Manufacturer: Linux 6.18.52 uhci_hcd client # [ 2.470531] scsi host1: ahci worker2 # [ 2.462085] usb usb3: SerialNumber: 0000:00:1d.1 worker2 # [ 2.463542] hub 3-0:1.0: USB hub found client # [ 2.472045] scsi host2: ahci worker1 # [ 2.473857] uhci_hcd 0000:00:1d.2: detected 2 ports client # [ 2.474291] scsi host3: ahci worker1 # [ 2.342694] systemd-fsck[107]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 2.466422] hub 3-0:1.0: 2 ports detected lb # [ 2.476417] uhci_hcd 0000:00:1d.2: UHCI Host Controller worker1 # [ 2.476529] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 client # [ 2.476290] scsi host4: ahci lb # [ 2.478287] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 client # [ 2.477976] scsi host5: ahci client # [ 2.479047] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker1 # [ 2.348410] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 2.480129] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker2 # [ 2.472577] uhci_hcd 0000:00:1d.2: UHCI Host Controller lb # [ 2.483275] uhci_hcd 0000:00:1d.2: detected 2 ports client # [ 2.481832] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.482785] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 lb # [ 2.484085] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 worker1 # [ 2.483952] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 client # [ 2.483060] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker2 # [ 2.475285] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4 client # [ 2.484234] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 lb # [ 2.353317] systemd-fsck[105]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 2.485370] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker2 # [ 2.478519] uhci_hcd 0000:00:1d.2: detected 2 ports worker1 # [ 2.488311] usb usb4: Product: UHCI Host Controller worker2 # [ 2.479360] uhci_hcd 0000:00:1d.2: irq 18, io port 0x0000c1c0 lb # [ 2.489293] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.489732] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.490507] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.346893] systemd-fsck[105]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 2.493110] hub 4-0:1.0: USB hub found lb # [ 2.360798] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.494583] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker2 # [ 2.351578] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 2.495756] hub 4-0:1.0: 2 ports detected lb # [ 2.496197] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 worker2 # [ 2.489573] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 worker1 # [ 2.499342] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker2 # [ 2.490742] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 lb # [ 2.501206] usb usb4: Product: UHCI Host Controller lb # [ 2.501927] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker2 # [ 2.494451] usb usb4: Product: UHCI Host Controller lb # [ 2.504193] usb usb4: SerialNumber: 0000:00:1d.2 worker2 # [ 2.495188] usb usb4: Manufacturer: Linux 6.18.52 uhci_hcd worker1 # [ 2.505226] ahci 0000:00:1f.2: flags: 64bit ncq only lb # [ 2.506349] hub 4-0:1.0: USB hub found worker2 # [ 2.497747] usb usb4: SerialNumber: 0000:00:1d.2 lb # [ 2.508346] hub 4-0:1.0: 2 ports detected worker2 # [ 2.499987] hub 4-0:1.0: USB hub found worker2 # [ 2.502442] hub 4-0:1.0: 2 ports detected worker1 # [ 2.516770] scsi host0: ahci worker1 # [ 2.519748] scsi host1: ahci worker1 # [ 2.523706] scsi host2: ahci worker1 # [ 2.527990] scsi host3: ahci worker2 # [ 2.519830] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker1 # [ 2.532618] scsi host4: ahci worker1 # [ 2.534041] scsi host5: ahci lb # [ 2.534703] ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode worker1 # [ 2.534580] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 lb # [ 2.535815] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker1 # [ 2.535716] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 2.536663] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 2.528844] ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) worker2 # [ 2.533328] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.543392] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 lb # [ 2.544234] ahci 0000:00:1f.2: flags: 64bit ncq only worker1 # [ 2.545076] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker1 # [ 2.546230] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker1 # [ 2.547418] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker2 # [ 2.539780] scsi host0: ahci worker1 # [ 2.548578] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 lb # [ 2.550229] scsi host0: ahci worker2 # [ 2.541844] scsi host1: ahci lb # [ 2.551862] scsi host1: ahci worker2 # [ 2.543284] scsi host2: ahci lb # [ 2.553482] scsi host2: ahci worker2 # [ 2.544416] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 2.555107] scsi host3: ahci worker2 # [ 2.546172] scsi host3: ahci lb # [ 2.556711] scsi host4: ahci worker2 # [ 2.547708] scsi host4: ahci lb # [ 2.558047] scsi host5: ahci worker2 # [ 2.549307] scsi host5: ahci lb # [ 2.559467] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 worker2 # [ 2.550313] ata1: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc100 irq 47 lpm-pol 1 lb # [ 2.560906] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 worker2 # [ 2.552018] ata2: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc180 irq 47 lpm-pol 1 lb # [ 2.562113] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 worker2 # [ 2.553327] ata3: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc200 irq 47 lpm-pol 1 lb # [ 2.563288] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 worker2 # [ 2.554571] ata4: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc280 irq 47 lpm-pol 1 lb # [ 2.564457] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 worker2 # [ 2.555733] ata5: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc300 irq 47 lpm-pol 1 lb # [ 2.565625] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 worker2 # [ 2.556889] ata6: SATA max UDMA/133 abar m4096@0xfebdc000 port 0xfebdc380 irq 47 lpm-pol 1 client # [ 2.569433] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 client # [ 2.572419] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 client # [ 2.575746] usb 1-1: Product: QEMU USB Tablet client # [ 2.577667] usb 1-1: Manufacturer: QEMU client # [ 2.579551] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 client # [ 2.610580] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 2.665953] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 lb # [ 2.667333] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker1 # [ 2.668871] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 lb # [ 2.670394] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker1 # [ 2.672112] usb 1-1: Product: QEMU USB Tablet lb # [ 2.673812] usb 1-1: Product: QEMU USB Tablet worker1 # [ 2.673892] usb 1-1: Manufacturer: QEMU lb # [ 2.675829] usb 1-1: Manufacturer: QEMU worker1 # [ 2.675583] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 lb # [ 2.677867] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 worker2 # [ 2.673748] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 worker2 # [ 2.676680] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 worker2 # [ 2.679948] usb 1-1: Product: QEMU USB Tablet worker2 # [ 2.681763] usb 1-1: Manufacturer: QEMU worker2 # [ 2.683483] usb 1-1: SerialNumber: 28754-0000:00:1d.7-1 client # [ 2.567540] systemd[1]: Mounting /sysroot... worker1 # [ 2.708759] hid: raw HID events driver (C) Jiri Kosina lb # [ 2.709233] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 2.715263] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 2.644631] systemd[1]: Mounting /sysroot... lb # [ 2.649993] systemd[1]: Mounting /sysroot... client # [ 2.799076] ata5: SATA link down (SStatus 0 SControl 300) client # [ 2.801777] ata2: SATA link down (SStatus 0 SControl 300) client # [ 2.804022] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) client # [ 2.806635] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 client # [ 2.808802] ata3.00: applying bridge limits worker2 # [ 2.666349] systemd[1]: Mounting /sysroot... client # [ 2.810688] ata6: SATA link down (SStatus 0 SControl 300) client # [ 2.813135] ata4: SATA link down (SStatus 0 SControl 300) client # [ 2.815501] ata1: SATA link down (SStatus 0 SControl 300) client # [ 2.817588] ata3.00: configured for UDMA/100 client # [ 2.820116] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 worker1 # [ 2.864038] ata6: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.866311] ata4: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.868896] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker1 # [ 2.871217] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker1 # [ 2.873586] ata3.00: applying bridge limits worker1 # [ 2.875591] ata2: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.866256] ata1: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.877948] ata5: SATA link down (SStatus 0 SControl 300) worker1 # [ 2.880089] ata1: SATA link down (SStatus 0 SControl 300) lb # [ 2.881020] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker1 # [ 2.882461] ata3.00: configured for UDMA/100 worker2 # [ 2.873736] ata2: SATA link down (SStatus 0 SControl 300) lb # [ 2.883466] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 lb # [ 2.885873] ata3.00: applying bridge limits worker2 # [ 2.875961] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) worker1 # [ 2.884871] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.887842] ata6: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.878628] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 worker2 # [ 2.880628] ata3.00: applying bridge limits lb # [ 2.890381] ata2: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.882683] ata5: SATA link down (SStatus 0 SControl 300) lb # [ 2.892626] ata1: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.885087] ata4: SATA link down (SStatus 0 SControl 300) client # [ 2.894852] usbcore: registered new interface driver usbhid lb # [ 2.895052] ata5: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.887239] ata6: SATA link down (SStatus 0 SControl 300) lb # [ 2.897459] ata4: SATA link down (SStatus 0 SControl 300) worker2 # [ 2.889125] ata3.00: configured for UDMA/100 lb # [ 2.899637] ata3.00: configured for UDMA/100 client # [ 2.899584] usbhid: USB HID core driver worker2 # [ 2.891130] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 lb # [ 2.902062] scsi 2:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 client # [ 2.916849] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 client # [ 2.920546] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 client # [ 2.926607] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray client # [ 2.941609] cdrom: Uniform CD-ROM driver Revision: 3.20 client # [ 2.955940] EXT4-fs (vda): mounted filesystem d8967c73-c804-4787-9148-a44ac641fad0 r/w with ordered data mode. Quota mode: none. worker1 # [ 2.957920] usbcore: registered new interface driver usbhid client # [ 2.827899] systemd[1]: Mounted /sysroot. worker1 # [ 2.961014] usbhid: USB HID core driver client # [ 2.829347] systemd[1]: Reached target Initrd Root File System. client # [ 2.831912] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 2.962499] usbcore: registered new interface driver usbhid worker2 # [ 2.963333] usbhid: USB HID core driver lb # [ 2.977802] usbcore: registered new interface driver usbhid client # [ 2.845644] systemd-sysroot-fstab-check[139]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 2.980514] usbhid: USB HID core driver client # [ 2.851171] systemd[1]: Reload requested from client PID 139 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 2.852597] systemd[1]: Reloading... worker1 # [ 2.986129] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 2.987935] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker1 # [ 2.994365] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker2 # [ 2.985527] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray lb # [ 3.000377] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray worker2 # [ 2.992594] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker2 # [ 2.994572] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker1 # [ 3.004981] cdrom: Uniform CD-ROM driver Revision: 3.20 worker2 # [ 2.999772] cdrom: Uniform CD-ROM driver Revision: 3.20 worker1 # [ 3.008355] EXT4-fs (vda): mounted filesystem 70146035-bf5b-448d-8907-857e8c6e25a3 r/w with ordered data mode. Quota mode: none. lb # [ 3.012342] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 worker1 # [ 2.881314] systemd[1]: Mounted /sysroot. lb # [ 3.014100] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:1d.7-1/input0 worker1 # [ 2.883061] systemd[1]: Reached target Initrd Root File System. lb # [ 3.016248] cdrom: Uniform CD-ROM driver Revision: 3.20 worker1 # [ 2.885037] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.015468] EXT4-fs (vda): mounted filesystem c78d41dc-5684-49b7-adc2-e6a090d67ada r/w with ordered data mode. Quota mode: none. worker2 # [ 2.887780] systemd[1]: Mounted /sysroot. worker1 # [ 2.898886] systemd-sysroot-fstab-check[138]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 2.888667] systemd[1]: Reached target Initrd Root File System. lb # [ 3.033925] EXT4-fs (vda): mounted filesystem f6a17f58-4013-4783-9a94-b59379a60662 r/w with ordered data mode. Quota mode: none. worker1 # [ 2.901647] systemd[1]: Reload requested from client PID 138 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 2.903993] systemd[1]: Reloading... worker2 # [ 2.893071] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 2.905628] systemd[1]: Mounted /sysroot. worker2 # [ 2.898132] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 2.909035] systemd[1]: Reached target Initrd Root File System. lb # [ 2.910635] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 2.903154] systemd[1]: Mounting /sysroot/run... worker2 # [ 2.906276] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 2.925767] systemd-sysroot-fstab-check[140]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 2.931062] systemd[1]: Reload requested from client PID 140 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 2.932516] systemd[1]: Reloading... worker2 # [ 2.921761] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 2.932913] systemd[1]: Reloading finished in 83 ms. client # [ 2.942555] systemd-sysroot-fstab-check[139]: Requesting initrd-fs.target/start/replace... worker2 # [ 2.933736] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 2.945145] systemd-sysroot-fstab-check[139]: Requesting swap.target/start/replace... worker2 # [ 2.937062] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 2.950865] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 2.952334] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 2.953547] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 2.964932] systemd-sysroot-fstab-check[144]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 2.968068] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 2.970668] systemd[1]: Mounted /sysroot/run. worker2 # [ 2.971364] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 2.986085] systemd[1]: Reloading finished in 82 ms. worker2 # [ 2.976832] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 2.982060] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 2.983331] systemd[1]: Reload requested from client PID 144 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 2.994773] systemd-sysroot-fstab-check[138]: Requesting initrd-fs.target/start/replace... worker2 # [ 2.985093] systemd[1]: Reloading... worker1 # [ 2.997139] systemd-sysroot-fstab-check[138]: Requesting swap.target/start/replace... worker1 # [ 3.003061] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.004376] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.005408] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.009683] systemd[1]: Reloading finished in 79 ms. lb # [ 3.019486] systemd-sysroot-fstab-check[140]: Requesting initrd-fs.target/start/replace... lb # [ 3.024167] systemd-sysroot-fstab-check[140]: Requesting swap.target/start/replace... lb # [ 3.026906] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.028324] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.030130] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.053594] systemd[1]: Reloading finished in 67 ms. worker2 # [ 3.063824] systemd-sysroot-fstab-check[144]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.066043] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.067467] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.069515] systemd-sysroot-fstab-check[144]: Requesting swap.target/start/replace... worker2 # [ 3.072666] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.073760] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.074776] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.078351] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.088323] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.089747] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.569339] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.576161] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.584162] systemd[1]: Mounting /sysroot/run... client # [ 3.595163] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.606693] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.646079] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.648341] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.650818] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.651866] systemd[1]: Mounted /sysroot/run. lb # [ 3.650959] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.652768] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.653147] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.655917] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.658168] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.659234] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.659298] systemd[1]: Mounting /sysroot/run... lb # [ 3.667176] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.668449] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.669557] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.671092] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.673357] systemd[1]: Mounting /sysroot/nix/store... lb # [ 3.676339] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.666106] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 3.683332] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.689831] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.694498] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.695648] systemd[1]: Reached target Initrd File Systems. client # [ 3.697277] systemd[1]: Starting Find NixOS closure... client # [ 3.700147] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 3.717988] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 3.718095] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 3.722805] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.724632] systemd[1]: Mounted /sysroot/run. lb # [ 3.724777] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.713958] systemd[1]: Mounted /sysroot/nix/store. client # [ 3.725654] systemd[1]: Finished Find NixOS closure. worker2 # [ 3.715840] systemd[1]: Reached target Initrd File Systems. lb # [ 3.726878] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.726917] systemd[1]: Reached target Initrd Default Target. worker1 # [ 3.727844] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.717648] systemd[1]: Starting Find NixOS closure... worker1 # [ 3.728805] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 3.728926] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 3.730735] systemd[1]: Mounted /sysroot/run. lb # [ 3.731616] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.721342] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 3.733089] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.737088] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.738742] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.743127] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.744331] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 3.744717] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.745167] systemd[1]: Stopped target Basic System. client # [ 3.745855] systemd[1]: Stopped target Initrd Root Device. client # [ 3.746628] systemd[1]: Stopped target Path Units. client # [ 3.747370] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.748440] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 3.749096] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.749579] systemd[1]: Stopped target Slice Units. client # [ 3.750325] systemd[1]: Stopped target Socket Units. lb # [ 3.750525] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.751125] systemd[1]: Stopped target System Initialization. client # [ 3.752134] systemd[1]: Stopped target Swaps. worker2 # [ 3.741966] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 3.753162] systemd[1]: Stopped target Timer Units. client # [ 3.754110] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 3.743097] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 3.754920] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 3.756436] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 3.757463] systemd[1]: Stopped Find NixOS closure. client # [ 3.758701] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 3.748910] systemd[1]: Finished Find NixOS closure. client # [ 3.760139] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.750168] systemd[1]: Reached target Initrd Default Target. client # [ 3.761242] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.751750] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 3.763143] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 3.764163] systemd[1]: Stopped Load Kernel Modules. client # [ 3.764897] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 3.765977] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 3.767045] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 3.768020] systemd[1]: Stopped Create System Files and Directories. client # [ 3.769934] systemd[1]: Stopped target Local File Systems. client # [ 3.772125] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 3.773084] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 3.774102] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 3.763978] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 3.765424] systemd[1]: Stopped target Basic System. worker2 # [ 3.766370] systemd[1]: Stopped target Initrd Root Device. client # [ 3.777135] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 3.768106] systemd[1]: Stopped target Path Units. client # [ 3.779183] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 3.768847] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 3.780166] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 3.770030] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 3.771095] systemd[1]: Stopped target Slice Units. worker2 # [ 3.771827] systemd[1]: Stopped target Socket Units. worker2 # [ 3.772639] systemd[1]: Stopped target System Initialization. worker2 # [ 3.773818] systemd[1]: Stopped target Swaps. worker2 # [ 3.774630] systemd[1]: Stopped target Timer Units. worker2 # [ 3.776155] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 3.777047] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 3.788498] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 3.777876] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 3.778864] systemd[1]: Stopped Find NixOS closure. client # [ 3.790513] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 3.781084] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.792088] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 3.782140] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 3.783385] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 3.784356] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 3.795120] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 3.786111] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 3.787760] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 3.788900] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 3.800687] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 3.789996] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 3.791275] systemd[1]: Stopped Create System Files and Directories. client # [ 3.802171] systemd[1]: Closed udev Control Socket. client # [ 3.804072] systemd[1]: Starting Cleanup udev Database... worker2 # [ 3.793887] systemd[1]: Stopped target Local File Systems. client # [ 3.804886] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 3.795076] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 3.806134] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 3.795995] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 3.807110] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 3.796962] systemd[1]: Stopped Coldplug All udev Devices. client # [ 3.808996] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 3.810054] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 3.811078] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 3.811969] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 3.802344] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 3.814122] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.803387] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 3.804361] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 3.812978] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 3.814486] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 3.816197] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 3.817273] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 3.818904] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 3.830113] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 3.819973] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.832159] systemd[1]: Finished Cleanup udev Database. client # [ 3.832921] systemd[1]: Reached target Switch Root. worker2 # [ 3.822245] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 3.834282] systemd[1]: Starting NixOS Activation... worker2 # [ 3.823902] systemd[1]: Closed udev Control Socket. worker2 # [ 3.824876] systemd[1]: Starting Cleanup udev Database... worker2 # [ 3.825729] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 3.826878] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 3.828227] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 3.829393] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 3.830659] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 3.832108] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 3.845324] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 3.846919] systemd[1]: Finished Cleanup udev Database. worker2 # [ 3.848974] systemd[1]: Reached target Switch Root. worker2 # [ 3.850073] systemd[1]: Starting NixOS Activation... client # [ 3.909311] initrd-nixos-activation-start[191]: booting system configuration /nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test worker2 # [ 3.919706] initrd-nixos-activation-start[192]: booting system configuration /nix/store/kknbsz59pyihvz42vvc4xnin6z1984g8-nixos-system-worker2-test client # [ 3.937990] initrd-nixos-activation-start[191]: running activation script... worker2 # [ 3.945725] initrd-nixos-activation-start[192]: running activation script... client # [ 4.137740] initrd-nixos-activation-start[214]: setting up /etc... worker2 # [ 4.131267] initrd-nixos-activation-start[215]: setting up /etc... worker2 # [ 4.233192] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 4.235070] systemd[1]: Finished NixOS Activation. worker2 # [ 4.236707] systemd[1]: Starting Switch Root... client # [ 4.248716] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 4.251095] systemd[1]: Finished NixOS Activation. client # [ 4.252379] systemd[1]: Starting Switch Root... worker2 # [ 4.249576] systemd[1]: Switching root. client # [ 4.266205] systemd[1]: Switching root. worker2 # [ 4.511732] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 4.523565] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). worker2 # [ 4.602112] NET: Registered PF_VSOCK protocol family client # [ 4.615612] NET: Registered PF_VSOCK protocol family worker1 # [ 4.646223] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.650687] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.693625] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.695232] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.697089] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.699540] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.699954] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.702987] systemd[1]: Reached target Initrd File Systems. lb # [ 4.704064] systemd[1]: Starting Find NixOS closure... lb # [ 4.706130] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.720782] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.724468] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.725201] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.726898] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.730248] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.731334] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.732983] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.734697] systemd[1]: Finished Find NixOS closure. lb # [ 4.735966] systemd[1]: Reached target Initrd Default Target. lb # [ 4.738092] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.745884] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.747198] systemd[1]: Stopped target Basic System. worker1 # [ 4.748122] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.749045] systemd[1]: Stopped target Path Units. worker1 # [ 4.751115] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.751378] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.752318] systemd[1]: Stopped target Basic System. worker1 # [ 4.752226] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.753253] systemd[1]: Stopped target Slice Units. lb # [ 4.753268] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.753916] systemd[1]: Stopped target Socket Units. worker1 # [ 4.754840] systemd[1]: Stopped target System Initialization. lb # [ 4.755189] systemd[1]: Stopped target Path Units. worker1 # [ 4.755670] systemd[1]: Stopped target Swaps. worker1 # [ 4.756343] systemd[1]: Stopped target Timer Units. lb # [ 4.755901] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.756992] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.756891] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.757797] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 4.757915] systemd[1]: Stopped target Slice Units. lb # [ 4.758613] systemd[1]: Stopped target Socket Units. worker1 # [ 4.759129] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.759323] systemd[1]: Stopped target System Initialization. worker1 # [ 4.760123] systemd[1]: Stopped Find NixOS closure. lb # [ 4.760110] systemd[1]: Stopped target Swaps. lb # [ 4.761076] systemd[1]: Stopped target Timer Units. worker1 # [ 4.761257] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.761933] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.762277] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.763163] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.764238] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.763965] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.765166] systemd[1]: Stopped Find NixOS closure. worker1 # [ 4.765182] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 4.766277] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.766580] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.767164] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.768168] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.768851] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.769284] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 4.770064] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.770313] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.771443] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 4.771505] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 4.772404] systemd[1]: Stopped target Local File Systems. lb # [ 4.772409] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 4.773442] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.774101] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 4.775127] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.775174] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 4.776319] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.776248] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.777151] systemd[1]: Stopped target Local File Systems. worker1 # [ 4.778097] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 4.779121] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.779113] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 4.780226] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.780036] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.781101] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.783127] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.784136] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.785184] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.788910] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 4.791515] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 4.792797] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.793985] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 4.797120] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.798924] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 4.799862] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 4.801343] systemd[1]: Closed udev Control Socket. worker1 # [ 4.802860] systemd[1]: Starting Cleanup udev Database... lb # [ 4.802762] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.803799] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 4.804139] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 4.805128] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.805317] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.805932] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.806637] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.808107] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.808740] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 4.809062] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.809858] systemd[1]: Closed udev Control Socket. worker1 # [ 4.809964] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 4.811148] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.811407] systemd[1]: Starting Cleanup udev Database... worker1 # [ 4.812450] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.812697] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 4.813727] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.814615] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 4.816130] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.817127] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.819108] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 4.826570] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 4.827784] systemd[1]: Finished Cleanup udev Database. worker1 # [ 4.829388] systemd[1]: Reached target Switch Root. worker1 # [ 4.830902] systemd[1]: Starting NixOS Activation... lb # [ 4.833400] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 4.835142] systemd[1]: Finished Cleanup udev Database. lb # [ 4.836985] systemd[1]: Reached target Switch Root. lb # [ 4.837950] systemd[1]: Starting NixOS Activation... worker2 # [ 4.960454] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 4.970454] systemd[1]: Detected virtualization kvm. worker2 # [ 4.972228] systemd[1]: Detected architecture x86-64. worker2 # [ 4.974176] systemd[1]: Detected first boot. client # [ 4.976717] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 4.986790] systemd[1]: Detected virtualization kvm. worker2 # [ 4.978505] systemd[1]: Initializing machine ID from random generator. client # [ 4.988812] systemd[1]: Detected architecture x86-64. client # [ 4.990824] systemd[1]: Detected first boot. client # [ 4.995365] systemd[1]: Initializing machine ID from random generator. worker1 # [ 4.901058] initrd-nixos-activation-start[190]: booting system configuration /nix/store/kgp1w89psmkzqxph18mrrd4l45zq8icp-nixos-system-worker1-test lb # [ 4.908959] initrd-nixos-activation-start[191]: booting system configuration /nix/store/n0idzb2a5795zcdcs33p9240n19lall0-nixos-system-lb-test worker1 # [ 4.927162] initrd-nixos-activation-start[190]: running activation script... lb # [ 4.934906] initrd-nixos-activation-start[191]: running activation script... worker2 # [ 5.101087] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 5.181197] systemd[1]: Applying preset policy. client # [ 5.240702] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 5.118917] initrd-nixos-activation-start[213]: setting up /etc... lb # [ 5.130795] initrd-nixos-activation-start[214]: setting up /etc... worker1 # [ 5.222470] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.224091] systemd[1]: Finished NixOS Activation. worker1 # [ 5.225626] systemd[1]: Starting Switch Root... worker2 # [ 5.354440] systemd[1]: Populated /etc with preset unit settings. lb # [ 5.232684] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.235072] systemd[1]: Finished NixOS Activation. client # [ 5.367117] systemd[1]: Applying preset policy. lb # [ 5.236159] systemd[1]: Starting Switch Root... worker1 # [ 5.238546] systemd[1]: Switching root. lb # [ 5.248847] systemd[1]: Switching root. worker1 # [ 5.502048] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). lb # [ 5.511650] systemd-journald[66]: Received SIGTERM from PID 1 (systemd). client # [ 5.538842] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 5.553236] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 5.554669] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 5.556767] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 5.558815] systemd[1]: Created slice Slice /system/getty. worker2 # [ 5.560132] systemd[1]: Created slice User and Session Slice. worker2 # [ 5.561084] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.562305] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 5.563440] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 5.564157] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 5.564901] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 5.565729] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 5.566465] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 5.567253] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 5.568128] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 5.569049] systemd[1]: Reached target Path Units. worker2 # [ 5.569741] systemd[1]: Reached target Remote File Systems. worker2 # [ 5.570503] systemd[1]: Reached target Slice Units. worker2 # [ 5.571222] systemd[1]: Reached target Swaps. worker2 # [ 5.573004] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 5.575497] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 5.593192] NET: Registered PF_VSOCK protocol family worker2 # [ 5.577277] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 5.598947] NET: Registered PF_VSOCK protocol family worker2 # [ 5.590918] systemd[1]: Listening on Factory Reset Management. worker2 # [ 5.591827] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 5.594838] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 5.595998] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 5.598335] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 5.599876] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 5.601004] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.602300] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 5.606355] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 5.607429] systemd[1]: Listening on udev Control Socket. worker2 # [ 5.608294] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 5.610596] systemd[1]: Mounting Huge Pages File System... worker2 # [ 5.613757] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 5.620540] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 5.625767] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 5.633307] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 5.644197] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 5.648105] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 5.652980] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 5.659565] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 5.677098] systemd[1]: Mounting FUSE Control File System... worker2 # [ 5.680721] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 5.693119] systemd[1]: Starting Journal Service... worker2 # [ 5.700596] systemd[1]: Starting Load Kernel Modules... worker2 # [ 5.707150] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 5.711900] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 5.714676] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 5.720721] systemd[1]: Starting Coldplug All udev Devices... client # [ 5.732386] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 5.725031] systemd[1]: Listening on Journal Log Access Socket. client # [ 5.733746] systemd[1]: Stopped initrd-switch-root.service. client # [ 5.735767] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 5.727964] systemd[1]: Mounted Huge Pages File System. client # [ 5.737828] systemd[1]: Created slice Slice /system/getty. worker2 # [ 5.730570] systemd[1]: Mounted POSIX Message Queue File System. client # [ 5.739194] systemd[1]: Created slice User and Session Slice. client # [ 5.740056] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.732510] systemd[1]: Mounted Kernel Debug File System. client # [ 5.741261] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 5.742346] systemd[1]: Expecting device /dev/hvc0... client # [ 5.743023] systemd[1]: Expecting device /dev/ttyS0... worker2 # [ 5.734753] systemd[1]: Mounted Kernel Trace File System. client # [ 5.743805] systemd[1]: Reached target Local Encrypted Volumes. client # [ 5.744650] systemd[1]: Stopped target initrd-fs.target. client # [ 5.745428] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 5.737610] systemd[1]: Finished Create List of Static Device Nodes. client # [ 5.746207] systemd[1]: Stopped target initrd-switch-root.target. client # [ 5.747015] systemd[1]: Reached target Virtual Machines and Containers. client # [ 5.747950] systemd[1]: Reached target Path Units. worker2 # [ 5.739749] systemd[1]: Mounted FUSE Control File System. client # [ 5.748699] systemd[1]: Reached target Remote File Systems. client # [ 5.749493] systemd[1]: Reached target Slice Units. worker2 # [ 5.744877] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 5.750276] systemd[1]: Reached target Swaps. client # [ 5.763987] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 5.766492] systemd[1]: Listening on Process Core Dump Socket. client # [ 5.768263] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 5.770114] systemd[1]: Listening on Factory Reset Management. client # [ 5.771055] systemd[1]: Listening on Hostname Service Socket. client # [ 5.773876] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 5.768043] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 5.776774] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 5.769857] systemd[1]: Finished Load Kernel Module configfs. client # [ 5.779758] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 5.780887] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 5.782021] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.783348] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 5.775698] systemd[1]: Mounting Kernel Configuration File System... client # [ 5.787436] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 5.788436] systemd[1]: Listening on udev Control Socket. client # [ 5.789281] systemd[1]: Listening on udev Varlink Socket. client # [ 5.791499] systemd[1]: Mounting Huge Pages File System... client # [ 5.793899] systemd[1]: Mounting POSIX Message Queue File System... client # [ 5.800711] systemd[1]: Mounting Kernel Debug File System... client # [ 5.808573] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 5.801876] systemd[1]: Mounted Kernel Configuration File System. client # [ 5.816653] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 5.811543] EXT4-fs (vda): re-mounted c78d41dc-5684-49b7-adc2-e6a090d67ada. client # [ 5.829380] systemd[1]: Starting Load Kernel Module configfs... worker2 # [ 5.822731] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 5.823738] systemd-journald[285]: Collecting audit messages is enabled. worker2 # [ 5.824900] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 5.826465] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 5.837347] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 5.835555] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 5.836416] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.848037] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 5.848576] loop: module loaded client # [ 5.856336] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 5.853006] systemd[1]: Finished Load Kernel Modules. client # [ 5.862561] systemd[1]: Mounting FUSE Control File System... client # [ 5.865267] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 5.858741] systemd[1]: Starting Firewall... worker2 # [ 5.863828] systemd[1]: Starting Apply Kernel Variables... client # [ 5.877721] systemd[1]: Starting Journal Service... client # [ 5.887502] systemd[1]: Starting Load Kernel Modules... worker2 # [ 5.880629] systemd[1]: Started Journal Service. worker2 # [ 5.748121] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 5.750246] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 5.753384] systemd-modules-load[286]: Inserted module 'loop' client # [ 5.897647] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 5.761071] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 5.911395] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 5.915308] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.921817] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 5.781609] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 5.790282] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 5.937014] systemd-journald[284]: Collecting audit messages is enabled. client # [ 5.943425] systemd[1]: Listening on Journal Log Access Socket. client # [ 5.946416] systemd[1]: Mounted Huge Pages File System. client # [ 5.950512] systemd[1]: Mounted POSIX Message Queue File System. client # [ 5.954557] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 5.951715] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 5.960402] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 5.961391] systemd[1]: Detected virtualization kvm. worker1 # [ 5.963179] systemd[1]: Detected architecture x86-64. worker2 # [ 5.820874] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 5.821845] systemd[1]: Reached target First Boot Complete. worker1 # [ 5.965108] systemd[1]: Detected first boot. client # [ 5.965875] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 5.958328] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.967986] systemd[1]: Detected virtualization kvm. lb # [ 5.969800] systemd[1]: Detected architecture x86-64. worker1 # [ 5.969327] systemd[1]: Initializing machine ID from random generator. lb # [ 5.971764] systemd[1]: Detected first boot. client # [ 5.839953] systemd[1]: Queued start job for default target Multi-User System. lb # [ 5.975777] systemd[1]: Initializing machine ID from random generator. client # [ 5.844149] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 5.977634] systemd[1]: Started Journal Service. client # [ 5.978452] loop: module loaded client # [ 5.848755] systemd-modules-load[285]: Inserted module 'loop' worker2 # [ 5.839466] systemd-oomd[287]: No swap; memory pressure usage will be degraded worker2 # [ 5.976798] systemd-journald[285]: Received client request to flush runtime journal. client # [ 5.855145] systemd[1]: modprobe@configfs.service: Deactivated successfully. client # [ 5.858262] systemd[1]: Finished Load Kernel Module configfs. client # [ 5.860353] systemd[1]: Finished Load Kernel Modules. client # [ 5.880664] systemd[1]: Mounting Kernel Configuration File System... client # [ 5.885930] systemd[1]: Starting Apply Kernel Variables... client # [ 6.021391] EXT4-fs (vda): re-mounted d8967c73-c804-4787-9148-a44ac641fad0. client # [ 5.890932] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 5.892086] systemd[1]: Mounted FUSE Control File System. client # [ 5.906579] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 5.913598] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 5.921750] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 5.922783] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 5.924492] systemd-oomd[287]: No swap; memory pressure usage will be degraded client # [ 5.934102] systemd[1]: Starting Load/Save OS Random Seed... client # [ 5.934930] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 5.936189] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 5.931656] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 5.934461] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 5.937456] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 5.958095] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 5.962394] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 5.964089] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 5.989075] systemd[1]: Mounted Kernel Configuration File System. client # [ 6.128171] systemd-journald[284]: Received client request to flush runtime journal. worker2 # [ 6.054940] systemd-udevd[319]: Using default interface naming scheme 'v261'. worker1 # [ 6.207304] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.078354] systemd[1]: Finished Load/Save OS Random Seed. client # [ 6.080027] systemd[1]: Reached target First Boot Complete. client # [ 6.082178] systemd[1]: Finished Apply Kernel Variables. lb # [ 6.216648] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.086313] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 6.113711] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 6.117391] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 6.127750] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 6.153537] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 6.145779] systemd[1]: Finished Coldplug All udev Devices. client # [ 6.169064] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 6.170151] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 6.305341] systemd[1]: Applying preset policy. client # [ 6.172611] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 6.331962] systemd[1]: Applying preset policy. client # [ 6.204210] systemd-udevd[316]: Using default interface naming scheme 'v261'. client # [ 6.237167] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 6.268482] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.321067] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 6.472174] systemd[1]: Populated /etc with preset unit settings. client # [ 6.362764] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 6.503422] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 6.362038] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 6.376629] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker2 # [ 6.377889] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. client # [ 6.406232] (udev-worker)[323]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 6.412067] (udev-worker)[323]: Network interface NamePolicy= disabled on kernel command line. client # [ 6.413771] (udev-worker)[325]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.415105] (udev-worker)[355]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.422135] systemd[1]: Mounting /run/wrappers... worker2 # [ 6.429261] (udev-worker)[351]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 6.431871] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 6.473881] systemd[1]: Mounted /run/wrappers. worker2 # [ 6.474700] systemd[1]: Reached target Local File Systems. worker2 # [ 6.483078] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.495951] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 6.497186] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 6.487302] systemd[1]: Starting register-nix-paths.service... client # [ 6.498646] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.501061] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 6.491352] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 6.502591] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.492230] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 6.504144] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.496173] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 6.501220] systemd[1]: Starting Create System Files and Directories... worker1 # [ 6.671431] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 6.672867] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 6.675022] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 6.677075] systemd[1]: Created slice Slice /system/getty. worker1 # [ 6.678431] systemd[1]: Created slice User and Session Slice. worker1 # [ 6.679368] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 6.680580] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 6.681675] systemd[1]: Expecting device /dev/hvc0... client # [ 6.680997] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 6.682388] systemd[1]: Expecting device /dev/ttyS0... worker1 # [ 6.683172] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 6.684034] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 6.684866] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 6.685622] systemd[1]: Stopped target initrd-switch-root.target. client # [ 6.685213] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker1 # [ 6.686443] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 6.687355] systemd[1]: Reached target Path Units. worker1 # [ 6.688092] systemd[1]: Reached target Remote File Systems. worker1 # [ 6.688911] systemd[1]: Reached target Slice Units. worker1 # [ 6.689587] systemd[1]: Reached target Swaps. worker1 # [ 6.691488] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 6.693904] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.704087] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.705569] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 6.695688] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.707625] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 6.709276] systemd[1]: Listening on Factory Reset Management. lb # [ 6.709637] systemd[1]: Created slice Slice /system/getty. worker1 # [ 6.710230] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.710996] systemd[1]: Created slice User and Session Slice. lb # [ 6.711916] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 6.712958] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.713182] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.714295] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 6.714301] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.715009] systemd[1]: Expecting device /dev/ttyS0... lb # [ 6.715762] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 6.716588] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 6.716823] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.717373] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.718205] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 6.718172] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 6.719056] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 6.719979] systemd[1]: Reached target Path Units. worker1 # [ 6.719339] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.720663] systemd[1]: Reached target Remote File Systems. lb # [ 6.721468] systemd[1]: Reached target Slice Units. worker1 # [ 6.720679] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 6.722211] systemd[1]: Reached target Swaps. worker2 # [ 6.578738] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 6.724062] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 6.724774] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 6.725783] systemd[1]: Listening on udev Control Socket. worker2 # [ 6.582306] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 6.726517] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 6.726595] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 6.728925] systemd[1]: Mounting Huge Pages File System... worker1 # [ 6.733018] systemd[1]: Mounting POSIX Message Queue File System... client # [ 6.602377] systemd[1]: Mounting /run/wrappers... worker1 # [ 6.738760] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.728426] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.742173] systemd[1]: Listening on Factory Reset Management. lb # [ 6.743081] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 6.745594] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.745850] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.747361] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.750324] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 6.751451] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 6.752045] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.752588] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.753850] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 6.757877] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.758884] systemd[1]: Listening on udev Control Socket. lb # [ 6.759742] systemd[1]: Listening on udev Varlink Socket. lb # [ 6.761840] systemd[1]: Mounting Huge Pages File System... worker1 # [ 6.762290] systemd[1]: Starting Load Kernel Module configfs... lb # [ 6.764080] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 6.766070] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 6.771283] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 6.771934] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.778543] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 6.779827] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 6.639165] systemd[1]: Finished Create System Files and Directories. lb # [ 6.786466] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 6.645120] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 6.647299] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 6.660120] systemd[1]: Mounted /run/wrappers. worker1 # [ 6.795530] systemd[1]: Mounting FUSE Control File System... client # [ 6.665125] systemd[1]: Reached target Local File Systems. worker1 # [ 6.798304] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.667177] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 6.668162] systemd[1]: Starting register-nix-paths.service... lb # [ 6.803020] systemd[1]: Starting Load Kernel Module configfs... client # [ 6.676042] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 6.676910] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 6.810842] systemd[1]: Starting Journal Service... lb # [ 6.811696] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 6.815744] systemd[1]: Starting Load Kernel Modules... client # [ 6.686079] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 6.819378] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 6.822371] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.823577] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.691971] systemd[1]: Starting Create System Files and Directories... worker1 # [ 6.829388] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 6.831792] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.837887] systemd[1]: Mounting FUSE Control File System... worker1 # [ 6.838402] systemd[1]: Starting Coldplug All udev Devices... lb # [ 6.839220] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 6.843270] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 6.845394] systemd[1]: Mounted Huge Pages File System. lb # [ 6.848089] systemd[1]: Starting Journal Service... worker1 # [ 6.847831] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 6.849478] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 6.851379] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 6.853468] systemd[1]: Mounted FUSE Control File System. lb # [ 6.854786] systemd[1]: Starting Load Kernel Modules... worker1 # [ 6.856724] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 6.861978] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.865477] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 6.724568] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 6.875432] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 6.882499] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.750158] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 6.701505] ACPI: button: Power Button [PWRF] worker1 # [ 6.887921] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 6.890783] systemd[1]: Finished Load Kernel Module configfs. client # [ 6.758736] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 6.892510] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 6.895224] systemd[1]: Mounting Kernel Configuration File System... client # [ 6.714607] mousedev: PS/2 mouse device common for all mice client # [ 6.743885] rtc_cmos PNP0B00:00: RTC can wake from S4 client # [ 6.754249] rtc_cmos PNP0B00:00: registered as rtc0 lb # [ 6.903454] systemd[1]: Listening on Journal Log Access Socket. lb # [ 6.907025] systemd[1]: Mounted Huge Pages File System. worker2 # [ 6.764493] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 6.765924] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 6.768432] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 6.912925] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 6.771231] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 6.916977] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 6.774526] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 6.919712] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 6.776681] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 6.921559] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.779116] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.923246] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.925882] systemd[1]: modprobe@configfs.service: Deactivated successfully. worker1 # [ 6.928955] EXT4-fs (vda): re-mounted 70146035-bf5b-448d-8907-857e8c6e25a3. worker2 # [ 6.786923] systemd[1]: Finished Rebuild Journal Catalog. client # [ 6.799175] systemd[1]: Finished Create System Files and Directories. worker2 # [ 6.789174] systemd[1]: Starting Update is Completed... worker1 # [ 6.934513] systemd-journald[283]: Collecting audit messages is enabled. lb # [ 6.936926] systemd[1]: Finished Load Kernel Module configfs. client # [ 6.806391] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 6.940379] systemd-journald[284]: Collecting audit messages is enabled. worker1 # [ 6.942225] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 6.810397] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 6.943556] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 6.947758] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 6.949183] EXT4-fs (vda): re-mounted f6a17f58-4013-4783-9a94-b59379a60662. lb # [ 6.951582] systemd[1]: Mounting Kernel Configuration File System... client # [ 6.754350] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T05:32:59 UTC (1790141579) client # [ 6.754423] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 6.953279] loop: module loaded worker1 # [ 6.953821] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 6.954588] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.962490] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 6.964129] loop: module loaded lb # [ 6.969354] systemd[1]: Finished Load Kernel Modules. lb # [ 6.972509] systemd[1]: Mounted FUSE Control File System. worker1 # [ 6.972590] systemd[1]: Finished Load Kernel Modules. lb # [ 6.842436] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 6.976108] systemd[1]: Starting Firewall... lb # [ 6.843758] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 6.977922] systemd[1]: Started Journal Service. lb # [ 6.845843] systemd-modules-load[285]: Inserted module 'loop' worker1 # [ 6.980731] systemd[1]: Starting Apply Kernel Variables... lb # [ 6.854759] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 6.994382] systemd[1]: Started Journal Service. client # [ 6.755383] systemd-journald[284]: Time jumped backwards, rotating. client # [ 6.773892] parport_pc 00:02: reported by Plug and Play ACPI client # [ 6.773975] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] lb # [ 6.868349] systemd-oomd[286]: No swap; memory pressure usage will be degraded worker1 # [ 6.865445] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 6.870055] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 6.875130] systemd-modules-load[284]: Inserted module 'loop' lb # [ 6.877070] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 6.879990] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 6.881471] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 6.881930] systemd[1]: Starting Firewall... lb # [ 6.885282] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 6.875212] systemd[1]: Finished Update is Completed. lb # [ 6.887983] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 6.894964] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 6.900075] systemd[1]: Starting Apply Kernel Variables... lb # [ 6.900879] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.902120] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 6.909272] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 6.910203] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 6.914610] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 6.914661] systemd[1]: Starting Update is Completed... client # [ 6.781724] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 client # [ 6.803441] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 6.887073] Console: switching to colour dummy device 80x25 worker1 # [ 6.937721] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 6.938684] systemd[1]: Reached target First Boot Complete. worker1 # [ 6.958382] systemd-oomd[285]: No swap; memory pressure usage will be degraded worker1 # [ 7.094084] systemd-journald[283]: Received client request to flush runtime journal. client # [ 6.961621] systemd[1]: Finished Update is Completed. client # [ 7.088597] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt client # [ 7.088711] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD lb # [ 7.106401] systemd-journald[284]: Received client request to flush runtime journal. worker2 # [ 7.101245] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker2 # [ 7.111321] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console client # [ 7.126970] [drm] Found bochs VGA, ID 0xb0c5. client # [ 7.126972] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. client # [ 7.130973] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client # [ 7.131726] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 client # [ 7.137749] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 client # [ 7.140915] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker2 # [ 7.030938] systemd[1]: Finished Firewall. worker1 # [ 7.057412] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.062964] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.066163] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 7.074649] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.076874] systemd[1]: Reached target First Boot Complete. worker1 # [ 7.078088] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 7.078926] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.080135] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.082390] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.083577] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 7.084643] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.085433] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.085774] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 7.094624] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.117566] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 7.120633] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 7.155641] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.158072] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 7.161952] systemd-udevd[318]: Using default interface naming scheme 'v261'. client # [ 7.164352] systemd[1]: Starting Virtual Console Setup... lb # [ 7.174311] systemd[1]: Finished Coldplug All udev Devices. lb # [ 7.181398] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.182805] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 7.185550] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 7.182444] ACPI: button: Power Button [PWRF] worker2 # [ 7.268582] rtc_cmos PNP0B00:00: RTC can wake from S4 client # [ 7.348004] Console: switching to colour frame buffer device 160x50 lb # [ 7.228433] systemd-udevd[322]: Using default interface naming scheme 'v261'. worker1 # [ 7.238776] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 7.282254] mousedev: PS/2 mouse device common for all mice worker2 # [ 7.283314] rtc_cmos PNP0B00:00: registered as rtc0 client # [ 7.376498] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker2 # [ 7.283798] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T05:32:59 UTC (1790141579) worker2 # [ 7.283888] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs lb # [ 7.277124] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 7.291411] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 7.285900] systemd-journald[285]: Time jumped backwards, rotating. worker2 # [ 7.320484] parport_pc 00:02: reported by Plug and Play ACPI client # [ 7.297681] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 7.300432] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 7.433513] ppdev: user-space parallel port driver client # [ 7.444066] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker2 # [ 7.320575] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] worker2 # [ 7.328552] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 client # [ 7.459983] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker2 # [ 7.339886] Console: switching to colour dummy device 80x25 worker2 # [ 7.354233] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized worker2 # [ 7.462146] [drm] Found bochs VGA, ID 0xb0c5. worker2 # [ 7.462148] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker2 # [ 7.471040] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker2 # [ 7.471723] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD client # [ 7.352383] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 7.355260] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 7.481119] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic client # [ 7.365962] systemd[1]: Starting Virtual Console Setup... worker2 # [ 7.490626] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker1 # [ 7.383350] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.395189] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.535829] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker2 # [ 7.536949] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 client # [ 7.579954] kvm_amd: TSC scaling supported client # [ 7.580601] kvm_amd: Nested Virtualization enabled client # [ 7.581054] kvm_amd: Nested Paging enabled client # [ 7.582328] kvm_amd: LBR virtualization supported client # [ 7.582784] kvm_amd: Virtual VMLOAD VMSAVE supported client # [ 7.583783] kvm_amd: Virtual GIF supported client # [ 7.584424] kvm_amd: Virtual NMI enabled worker2 # [ 7.456227] systemd[1]: Finished register-nix-paths.service. worker2 # [ 7.457699] systemd[1]: Reached target System Initialization. worker2 # [ 7.460084] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 7.461205] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 7.463935] systemd[1]: Reached target Timer Units. worker2 # [ 7.464712] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 7.465675] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 7.477944] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 7.478229] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 7.468227] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 7.469272] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 7.472885] systemd[1]: Reached target Socket Units. worker2 # [ 7.473650] systemd[1]: Reached target Basic System. worker2 # [ 7.474350] systemd[1]: Started backdoor.service. worker2 # [ 7.477461] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 7.479259] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 7.623393] EDAC MC: Ver: 3.0.0 worker2 # [ 7.486278] systemd[1]: Starting Post-Boot Actions... worker2 # [ 7.489626] systemd[1]: Started Reset console on configuration changes. worker2 # [ 7.492827] systemd[1]: Starting resolvconf update... worker1 # [ 7.503604] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. lb # [ 7.506112] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. worker2 # [ 7.503423] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 7.525550] (udev-worker)[358]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 7.525777] (udev-worker)[345]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 7.527935] (udev-worker)[345]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 7.530974] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. lb # [ 7.533259] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 7.541989] (udev-worker)[360]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 7.544263] systemd[1]: Mounting /run/wrappers... worker2 # [ 7.542631] systemd[1]: Starting Virtual Console Setup... lb # [ 7.576195] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.583187] systemd[1]: Mounted /run/wrappers. worker1 # [ 7.584451] systemd[1]: Reached target Local File Systems. worker1 # [ 7.588427] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 7.592437] systemd[1]: Starting register-nix-paths.service... worker2 # [ 7.583395] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 7.584565] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 7.600148] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 7.590101] systemd[1]: Starting Virtual Console Setup... worker1 # [ 7.603402] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.609095] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 7.600204] systemd[1]: Finished Post-Boot Actions. lb # [ 7.612234] systemd[1]: Mounted /run/wrappers. lb # [ 7.612968] systemd[1]: Reached target Local File Systems. worker1 # [ 7.615098] systemd[1]: Starting Create System Files and Directories... lb # [ 7.616437] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 7.620423] systemd[1]: Starting register-nix-paths.service... lb # [ 7.625179] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.626160] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 7.635152] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # connecting to host... worker2 # [ 7.635128] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 7.636215] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 7.649927] systemd[1]: Starting Create System Files and Directories... worker2 # [ 7.640817] systemd[1]: Reached target User and Group Name Lookups. client # [ 7.653193] systemd[1]: Finished register-nix-paths.service. worker2 # [ 7.644638] systemd[1]: Starting User Login Management... worker2 # [ 7.647504] nsncd[516]: Sep 23 05:32:59.993 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 7.663313] systemd-vconsole-setup[420]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 7.667074] systemd[1]: Finished Virtual Console Setup. client # [ 7.668369] systemd[1]: Reached target System Initialization. client # [ 7.669732] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 7.671119] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 7.673069] systemd[1]: Reached target Timer Units. client # [ 7.673859] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 7.674775] systemd[1]: Listening on Nix Daemon Socket. client # [ 7.675661] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 7.676815] systemd[1]: Reached target Socket Units. client # [ 7.678134] systemd[1]: Reached target Basic System. client # [ 7.679512] systemd[1]: Started backdoor.service. worker2 # [ 7.756248] Console: switching to colour frame buffer device 160x50 client # [ 7.683069] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 7.686110] systemd[1]: Starting nftables firewall... worker1 # [ 7.690520] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 7.693210] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 7.695231] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 7.699792] systemd[1]: Starting Post-Boot Actions... client # [ 7.707381] systemd[1]: Started Reset console on configuration changes. worker2 # [ 7.839606] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device worker2 # [ 7.672273] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 7.722351] systemd[1]: Starting resolvconf update... worker2 # [ 7.712700] dbus-broker-launch[520]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 7.727076] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 7.718190] dbus-broker-launch[520]: NSS returned no entry for 'systemd-timesync' lb # [ 7.728552] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.719187] dbus-broker-launch[520]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 7.735585] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 7.727172] systemd[1]: Started D-Bus System Message Bus. lb # [ 7.738933] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.743176] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.744992] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.747160] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.750199] systemd[1]: Finished Create System Files and Directories. client # [ 7.752088] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 7.754397] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 7.755746] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 7.756429] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # connecting to host... lb # [ 7.763681] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 7.763949] dbus-broker-launch[520]: Ready client # [ 7.778873] nsncd[428]: Sep 23 05:33:00.656 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 7.782288] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 7.792101] systemd[1]: Finished Post-Boot Actions. lb # [ 7.797384] systemd[1]: Finished Create System Files and Directories. client # [ 7.802786] systemd[1]: Reached target Host and Network Name Lookups. client: Guest shell says: b'Spawning backdoor root shell...\n' lb # [ 7.806440] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 7.809583] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 7.807100] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 7.833809] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 7.815584] systemd[1]: Stopped target Host and Network Name Lookups. client: connected to guest root shell worker2 # [ 7.816470] systemd[1]: Stopping Host and Network Name Lookups... client: (connecting took 8.49 seconds) worker2 # [ 7.819127] systemd[1]: Stopped target User and Group Name Lookups. client: (finished: waiting for the VM to finish booting, in 8.49 seconds) worker2 # [ 7.819948] systemd[1]: Stopping User and Group Name Lookups... client # [ 7.825868] systemd[1]: Starting User Login Management... worker2 # [ 7.824889] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 7.846238] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 7.828195] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 7.828992] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 7.832093] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 7.835705] systemd-logind[541]: Watching system buttons on /dev/input/event2 (Power Button) worker2 # [ 7.841633] systemd-logind[541]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker2 # [ 7.846620] systemd-logind[541]: New seat seat0. worker2 # [ 7.847352] systemd[1]: Started User Login Management. worker2 # [ 7.850568] systemd[1]: Starting linger-users.service... worker1 # [ 7.864731] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 7.866434] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.869430] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.872130] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.875280] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.877801] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.881084] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.013956] ppdev: user-space parallel port driver worker1 # [ 7.895889] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 7.898089] systemd[1]: Starting Update is Completed... lb # [ 7.904245] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 7.904863] dbus-broker-launch[437]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 7.922402] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 7.913239] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 7.915213] systemd[1]: Finished linger-users.service. lb # [ 7.928121] systemd[1]: Starting Update is Completed... worker2 # [ 7.921661] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 7.922742] nsncd[590]: Sep 23 05:33:00.271 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 7.925330] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 7.926248] systemd[1]: Reached target User and Group Name Lookups. client # [ 7.938116] dbus-broker-launch[437]: NSS returned no entry for 'systemd-timesync' client # [ 7.939165] dbus-broker-launch[437]: Invalid user-name in /nix/store/c4mprcfibsx637q0hk2g869f3wvd2k8k-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 7.943436] systemd-logind[458]: New seat seat0. worker2 # [ 7.938476] systemd[1]: Finished resolvconf update. worker2 # [ 7.941705] systemd[1]: Reached target Preparation for Network. client # [ 7.952256] systemd-logind[458]: Watching system buttons on /dev/input/event2 (Power Button) client # [ 7.953431] systemd-logind[458]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 7.943867] systemd[1]: Starting DHCP Client... client # [ 7.954588] systemd-logind[458]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) client # [ 7.955808] systemd[1]: Started User Login Management. worker2 # [ 7.946317] systemd[1]: Starting Address configuration of eth1... worker2 # [ 7.949134] systemd[1]: Starting Extra networking commands.... client # [ 7.965511] systemd[1]: Starting linger-users.service... lb # [ 7.984179] systemd[1]: Finished Update is Completed. worker1 # [ 7.988263] systemd[1]: Finished Update is Completed. client # [ 7.998730] systemd[1]: Started D-Bus System Message Bus. client # [ 7.999611] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 8.000493] systemd[1]: Stopping Host and Network Name Lookups... client # [ 8.001309] systemd[1]: Stopped target User and Group Name Lookups. client # [ 8.002089] systemd[1]: Stopping User and Group Name Lookups... lb # [ 8.138739] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console client # [ 8.009330] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 8.013385] systemd[1]: nscd.service: Deactivated successfully. client # [ 8.018879] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 8.010238] systemd-vconsole-setup[526]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 8.016754] systemd[1]: Finished Virtual Console Setup. client # [ 8.036538] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 8.049052] dbus-broker-launch[437]: Ready client # [ 8.066547] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 8.057102] systemd-logind[541]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 8.069812] systemd[1]: Finished linger-users.service. worker2 # [ 8.199231] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker2 # [ 8.066171] network-addresses-eth1-start[619]: adding address 192.168.1.4/24... done worker2 # [ 8.078600] network-addresses-eth1-start[619]: adding address 2001:db8:1::4/64... done client # [ 8.094330] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 8.096637] nsncd[520]: Sep 23 05:33:00.973 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 8.099594] systemd[1]: Reached target Host and Network Name Lookups. client # [ 8.100888] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 8.240344] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 client # [ 8.109682] systemd[1]: Finished resolvconf update. worker2 # [ 8.099181] systemd[1]: Finished Address configuration of eth1. client # [ 8.113943] systemd[1]: Starting DHCP Client... worker1 # [ 8.266880] bochs-drm 0000:00:01.0: vgaarb: deactivate vga console worker1 # [ 8.146464] systemd[1]: Finished Firewall. worker2 # [ 8.141434] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 8.143159] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 8.143923] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 8.288866] ACPI: button: Power Button [PWRF] worker2 # [ 8.146180] systemd[1]: Starting Virtual Console Setup... worker2 # [ 8.283069] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) client # [ 8.201499] dhcpcd[553]: dhcpcd-10.3.2 starting client # [ 8.207811] dhcpcd[559]: dev: loaded udev worker2 # [ 8.205316] systemd[1]: Finished Extra networking commands.. client # [ 8.348633] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 8.207140] systemd[1]: Reached target Network. worker2 # [ 8.207809] systemd[1]: Starting Permit User Sessions... lb # [ 8.146655] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 worker2 # [ 8.218606] dhcpcd[632]: dhcpcd-10.3.2 starting worker1 # [ 8.232421] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 8.224813] dhcpcd[690]: dev: loaded udev worker1 # [ 8.235590] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 8.376563] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 8.377028] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 8.258803] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 8.278201] systemd[1]: Finished Permit User Sessions. client # [ 8.421724] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.158821] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 8.161409] ACPI: button: Power Button [PWRF] lb # [ 8.205366] mousedev: PS/2 mouse device common for all mice client # [ 8.434408] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 8.435067] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 8.437099] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 8.437968] cfg80211: failed to load regulatory.db worker1 # [ 8.391013] mousedev: PS/2 mouse device common for all mice worker2 # [ 8.303840] systemd[1]: Started Getty on tty1. worker2 # [ 8.304602] systemd[1]: Reached target Login Prompts. lb # [ 8.315061] rtc_cmos PNP0B00:00: RTC can wake from S4 lb # [ 8.351852] Console: switching to colour dummy device 80x25 worker1 # [ 8.403420] rtc_cmos PNP0B00:00: RTC can wake from S4 worker1 # [ 8.425153] rtc_cmos PNP0B00:00: registered as rtc0 worker1 # [ 8.425224] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T05:33:00 UTC (1790141580) client # [ 8.477742] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 8.346853] dhcpcd[559]: eth0: waiting for carrier client # [ 8.348946] dhcpcd[559]: eth0: carrier acquired worker2 # [ 8.340369] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 8.355756] dhcpcd[559]: DUID 00:01:00:01:32:46:21:0d:52:54:00:12:34:56 client # [ 8.356917] dhcpcd[559]: eth0: IAID 00:12:34:56 client # [ 8.358590] dhcpcd[559]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 8.359317] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 8.363911] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 8.364630] systemd[1]: Finished nftables firewall. client # [ 8.366365] systemd[1]: Reached target Preparation for Network. lb # [ 8.371310] rtc_cmos PNP0B00:00: registered as rtc0 lb # [ 8.371376] rtc_cmos PNP0B00:00: setting system clock to 2026-09-23T05:33:00 UTC (1790141580) lb # [ 8.371455] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs client # [ 8.368909] systemd[1]: Starting Address configuration of eth1... client # [ 8.372416] systemd[1]: Starting Extra networking commands.... worker2 # [ 8.499713] kvm_amd: TSC scaling supported worker2 # [ 8.500163] kvm_amd: Nested Virtualization enabled worker2 # [ 8.503687] kvm_amd: Nested Paging enabled worker2 # [ 8.504113] kvm_amd: LBR virtualization supported worker2 # [ 8.504910] kvm_amd: Virtual VMLOAD VMSAVE supported worker2 # [ 8.507493] kvm_amd: Virtual GIF supported worker2 # [ 8.507909] kvm_amd: Virtual NMI enabled worker1 # [ 8.425295] rtc_cmos PNP0B00:00: alarms up to one day, y3k, 242 bytes nvram, hpet irqs worker1 # [ 8.426798] systemd-journald[283]: Time jumped backwards, rotating. worker1 # [ 8.431311] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:06.0/virtio4/input/input4 lb # [ 8.372420] systemd-journald[284]: Time jumped backwards, rotating. lb # [ 8.397234] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized lb # [ 8.397862] parport_pc 00:02: reported by Plug and Play ACPI client # [ 8.545460] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 8.541052] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 8.397946] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] client # [ 8.424552] network-addresses-eth1-start[585]: adding address 192.168.1.1/24... done lb # [ 8.426741] systemd[1]: Finished register-nix-paths.service. worker1 # [ 8.442567] Console: switching to colour dummy device 80x25 lb # [ 8.428337] systemd[1]: Reached target System Initialization. worker1 # [ 8.448828] parport_pc 00:02: reported by Plug and Play ACPI lb # [ 8.431388] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 8.432444] systemd[1]: Started niks3 garbage collection timer. client # [ 8.433354] network-addresses-eth1-start[585]: adding address 2001:db8:1::1/64... done worker2 # [ 8.558019] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 8.434175] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 8.558701] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 8.435536] systemd[1]: Reached target Timer Units. lb # [ 8.436560] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 8.560674] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 8.561547] cfg80211: failed to load regulatory.db lb # [ 8.442764] systemd[1]: Listening on niks3 server socket. lb # [ 8.443572] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 8.448925] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] lb # [ 8.444415] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 8.462225] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized client # [ 8.445372] systemd[1]: Finished Address configuration of eth1. lb # [ 8.445672] systemd[1]: Reached target Socket Units. lb # [ 8.446435] systemd[1]: Reached target Basic System. lb # [ 8.449277] systemd[1]: Started backdoor.service. lb # [ 8.452845] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 8.588238] [drm] Found bochs VGA, ID 0xb0c5. lb # [ 8.588653] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. worker2 # [ 8.582930] EDAC MC: Ver: 3.0.0 worker1 # [ 8.594855] [drm] Found bochs VGA, ID 0xb0c5. worker1 # [ 8.594858] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. lb # [ 8.462800] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 8.597785] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker1 # [ 8.598478] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 client # [ 8.467605] systemd[1]: Finished Extra networking commands.. lb # [ 8.469103] systemd[1]: Starting Post-Boot Actions... client # [ 8.469364] systemd[1]: Reached target Network. client # [ 8.471866] systemd[1]: Starting Permit User Sessions... lb # [ 8.605498] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt worker1 # [ 8.605933] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt lb # [ 8.606246] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker1 # [ 8.606673] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD worker2 # [ 8.603758] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 8.470695] dhcpcd[690]: eth0: waiting for carrier worker2 # [ 8.471668] dhcpcd[690]: libudev: received NULL device worker2 # [ 8.473288] dhcpcd[690]: libudev: received NULL device lb # [ 8.484533] systemd[1]: Started Reset console on configuration changes. worker2 # [ 8.474806] dhcpcd[690]: eth0: carrier acquired client # [ 8.487491] systemd[1]: Finished Permit User Sessions. worker2 # [ 8.479564] dhcpcd[690]: DUID 00:01:00:01:32:46:21:0c:52:54:00:12:34:56 client # [ 8.490539] systemd[1]: Started Getty on tty1. lb # [ 8.624377] bochs-drm 0000:00:01.0: [drm] Registered 1 planes with drm panic worker2 # [ 8.480844] dhcpcd[690]: eth0: IAID 00:12:34:56 client # [ 8.491782] systemd[1]: Reached target Login Prompts. lb # [ 8.625077] [drm] Initialized bochs-drm 1.0.0 for 0000:00:01.0 on minor 0 worker2 # [ 8.481637] dhcpcd[690]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 8.496227] systemd[1]: Starting resolvconf update... lb # [ 8.502850] systemd[1]: Started rustfs.service. lb # [ 8.528192] systemd[1]: Starting rustfs-bucket.service... worker2 # [ 8.517701] systemd-vconsole-setup[675]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 8.520156] systemd[1]: Finished Virtual Console Setup. lb # [ 8.539262] systemd[1]: Finished Firewall. lb # connecting to host... lb # [ 8.563586] nsncd[521]: Sep 23 05:33:00.824 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 8.702192] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker1 # [ 8.574125] systemd[1]: Starting Virtual Console Setup... lb # [ 8.574748] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 8.709735] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker1 # [ 8.577193] systemd[1]: Finished register-nix-paths.service. lb # [ 8.585281] systemd[1]: Finished Post-Boot Actions. lb # [ 8.610108] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 8.616156] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 8.673115] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 worker1 # [ 8.673358] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 worker1 # [ 8.750372] Console: switching to colour frame buffer device 160x50 lb # [ 8.634125] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 8.781787] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 8.650446] systemd[1]: Starting User Login Management... worker1 # [ 8.654941] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.656722] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.658082] systemd[1]: Starting Virtual Console Setup... lb # [ 8.688221] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 8.824682] ppdev: user-space parallel port driver lb # [ 8.722259] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.735724] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 8.737505] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.738814] systemd[1]: Starting Virtual Console Setup... worker1 # [ 8.878550] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) worker1 # [ 8.898877] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) lb # [ 8.829732] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 8.830808] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 8.984286] kvm_amd: TSC scaling supported worker1 # [ 8.984939] kvm_amd: Nested Virtualization enabled worker1 # [ 8.985381] kvm_amd: Nested Paging enabled worker1 # [ 8.985953] kvm_amd: LBR virtualization supported worker1 # [ 8.986385] kvm_amd: Virtual VMLOAD VMSAVE supported worker1 # [ 8.987265] kvm_amd: Virtual GIF supported worker1 # [ 8.987862] kvm_amd: Virtual NMI enabled lb # [ 8.858079] systemd[1]: Starting Virtual Console Setup... lb # [ 8.887423] dbus-broker-launch[550]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 9.026311] EDAC MC: Ver: 3.0.0 lb # [ 8.902729] dbus-broker-launch[550]: NSS returned no entry for 'systemd-timesync' lb # [ 8.905871] dbus-broker-launch[550]: Invalid user-name in /nix/store/xhp17752g31xx87ahxdgrix2v9kism59-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 8.920715] systemd-logind[553]: Watching system buttons on /dev/input/event2 (Power Button) lb # [ 9.058469] Console: switching to colour frame buffer device 160x50 worker1 # [ 8.938428] systemd-vconsole-setup[519]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 8.942978] systemd[1]: Finished Virtual Console Setup. worker1 # [ 8.945170] systemd[1]: Reached target System Initialization. worker1 # [ 8.945940] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 8.946996] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 8.947905] systemd[1]: Reached target Timer Units. worker1 # [ 8.948881] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 8.949817] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 8.952068] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 8.952929] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 8.954107] systemd[1]: Reached target Socket Units. worker1 # [ 8.954786] systemd[1]: Reached target Basic System. worker1 # [ 8.957063] systemd[1]: Started backdoor.service. worker1 # [ 8.959061] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 8.963064] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 8.970092] systemd[1]: Starting Post-Boot Actions... worker1 # [ 8.974063] systemd[1]: Started Reset console on configuration changes. worker1 # [ 8.978955] systemd[1]: Starting resolvconf update... worker1 # [ 8.984538] systemd[1]: Starting D-Bus System Message Bus... lb # [ 9.147650] bochs-drm 0000:00:01.0: [drm] fb0: bochs-drmdrmfb frame buffer device lb # [ 8.927740] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.016489] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.017688] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 9.019916] systemd[1]: Stopping User and Group Name Lookups... lb # [ 9.020771] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 9.009736] dhcpcd[690]: eth0: soliciting a DHCP lease lb # [ 9.021944] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.023611] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 9.024976] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.029620] systemd[1]: Started D-Bus System Message Bus. lb # [ 9.030560] dbus-broker-launch[550]: Ready lb # [ 9.031293] systemd-logind[553]: New seat seat0. lb # [ 9.036177] systemd-logind[553]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) lb # [ 9.037395] systemd[1]: Started User Login Management. lb # [ 9.038149] systemd-logind[553]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 9.162528] NET: Registered PF_PACKET protocol family lb # [ 9.043424] systemd[1]: Starting linger-users.service... worker1 # [ 9.045989] systemd[1]: Finished Post-Boot Actions. worker2 # [ 9.034992] dhcpcd[690]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 9.039357] dhcpcd[690]: eth0: probing address 10.0.2.15/24 lb # [ 9.058911] nsncd[613]: Sep 23 05:33:01.317 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.064778] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.068197] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.069109] systemd[1]: Reached target User and Group Name Lookups. worker1 # connecting to host... worker1 # [ 9.075152] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 9.075973] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 9.079616] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 9.080755] nsncd[526]: Sep 23 05:33:01.284 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 9.085812] systemd[1]: Starting User Login Management... lb # [ 9.093425] systemd[1]: Finished resolvconf update. lb # [ 9.098494] systemd[1]: Reached target Preparation for Network. lb # [ 9.106216] systemd[1]: Starting DHCP Client... worker1 # [ 9.108182] systemd[1]: Finished Import lastlog data into lastlog2 database. client: (finished: waiting for unit network-addresses-eth1.service, in 9.76 seconds) lb # [ 9.113946] systemd[1]: Starting Address configuration of eth1... client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 9.118211] systemd[1]: Starting Extra networking commands.... worker1 # [ 9.121720] dbus-broker-launch[530]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 9.131064] dbus-broker-launch[530]: NSS returned no entry for 'systemd-timesync' worker1 # [ 9.132225] dbus-broker-launch[530]: Invalid user-name in /nix/store/qip1v2bp3gxd2mgy76zm58p8c4nw43qv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 9.273258] iTCO_wdt iTCO_wdt.1.auto: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660) lb # [ 9.142210] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 9.145291] systemd[1]: Started D-Bus System Message Bus. lb # [ 9.148057] systemd[1]: Finished linger-users.service. client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.03 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 9.166404] dbus-broker-launch[530]: Ready lb # [ 9.323842] ppdev: user-space parallel port driver worker1 # [ 9.196954] systemd-logind[551]: New seat seat0. worker1 # [ 9.199224] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 9.201564] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 9.204205] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 9.208123] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 9.208926] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 9.210144] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 9.210932] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 9.214163] systemd-logind[551]: Watching system buttons on /dev/input/event2 (Power Button) worker1 # [ 9.215409] systemd-logind[551]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 9.219129] systemd-logind[551]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) worker1 # [ 9.221690] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 9.224307] systemd[1]: Started User Login Management. worker1 # [ 9.227262] systemd[1]: Starting linger-users.service... lb # [ 9.361791] iTCO_wdt iTCO_wdt.1.auto: initialized. heartbeat=30 sec (nowayout=0) worker1 # [ 9.274407] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 9.276955] systemd[1]: Finished linger-users.service. worker1 # [ 9.281318] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 9.282321] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 9.283858] nsncd[597]: Sep 23 05:33:01.489 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 9.286442] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 9.294668] systemd[1]: Finished resolvconf update. worker1 # [ 9.296077] systemd[1]: Reached target Preparation for Network. worker1 # [ 9.300117] systemd[1]: Starting DHCP Client... worker1 # [ 9.303174] systemd[1]: Starting Address configuration of eth1... worker1 # [ 9.306142] systemd[1]: Starting Extra networking commands.... lb # [ 9.313504] network-addresses-eth1-start[644]: adding address 192.168.1.2/24... done lb # [ 9.341700] network-addresses-eth1-start[644]: adding address 2001:db8:1::2/64... done lb # [ 9.355342] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 9.359492] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 9.377645] network-addresses-eth1-start[626]: adding address 192.168.1.3/24... done lb # [ 9.381145] systemd[1]: Starting Virtual Console Setup... lb # [ 9.390085] systemd[1]: Finished Address configuration of eth1. worker1 # [ 9.390817] network-addresses-eth1-start[626]: adding address 2001:db8:1::3/64... done worker1 # [ 9.407148] systemd[1]: Finished Address configuration of eth1. worker1 # [ 9.451782] dhcpcd[634]: dhcpcd-10.3.2 starting worker1 # [ 9.455658] systemd[1]: Finished Extra networking commands.. worker1 # [ 9.457199] systemd[1]: Reached target Network. worker1 # [ 9.460658] systemd[1]: Starting Permit User Sessions... worker1 # [ 9.462283] dhcpcd[690]: dev: loaded udev worker1 # [ 9.606605] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 9.607313] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 9.489257] systemd[1]: Finished Permit User Sessions. lb # [ 9.493976] dhcpcd[663]: dhcpcd-10.3.2 starting worker1 # [ 9.495057] systemd[1]: Started Getty on tty1. worker1 # [ 9.495710] systemd[1]: Reached target Login Prompts. lb # [ 9.519592] dhcpcd[722]: dev: loaded udev worker1 # [ 9.524733] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.531629] systemd[1]: Finished Extra networking commands.. lb # [ 9.536269] systemd[1]: Reached target Network. lb # [ 9.541147] systemd[1]: Starting PostgreSQL Server... lb # [ 9.544926] systemd[1]: Starting Permit User Sessions... worker1 # [ 9.690577] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 9.695932] 8021q: 802.1Q VLAN Support v1.8 lb # [ 9.696387] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 9.701883] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 9.702537] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 9.704210] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 9.705074] cfg80211: failed to load regulatory.db worker1 # [ 9.730693] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 9.731136] kvm_amd: TSC scaling supported lb # [ 9.731547] kvm_amd: Nested Virtualization enabled lb # [ 9.732014] kvm_amd: Nested Paging enabled worker1 # [ 9.599334] dhcpcd[690]: eth0: waiting for carrier worker1 # [ 9.600076] dhcpcd[690]: eth0: carrier acquired lb # [ 9.737317] kvm_amd: LBR virtualization supported lb # [ 9.737777] kvm_amd: Virtual VMLOAD VMSAVE supported worker1 # [ 9.604684] dhcpcd[690]: DUID 00:01:00:01:32:46:21:0d:52:54:00:12:34:56 worker1 # [ 9.605703] dhcpcd[690]: eth0: IAID 00:12:34:56 worker1 # [ 9.606468] dhcpcd[690]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 9.739974] kvm_amd: Virtual GIF supported lb # [ 9.740866] kvm_amd: Virtual NMI enabled lb # [ 9.654113] systemd[1]: Finished Permit User Sessions. lb # [ 9.654900] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 9.666484] systemd[1]: Started Getty on tty1. lb # [ 9.667218] systemd[1]: Reached target Login Prompts. lb # [ 9.886312] EDAC MC: Ver: 3.0.0 lb # [ 9.893980] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 9.765226] dhcpcd[559]: eth0: soliciting a DHCP lease client # [ 9.915214] NET: Registered PF_PACKET protocol family lb # [ 9.923477] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 9.790971] dhcpcd[559]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 9.926205] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 9.793364] dhcpcd[559]: eth0: probing address 10.0.2.15/24 lb # [ 9.928505] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 9.929368] cfg80211: failed to load regulatory.db lb # [ 10.004747] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 9.874503] dhcpcd[722]: eth0: waiting for carrier lb # [ 9.875944] dhcpcd[722]: eth0: carrier acquired lb # [ 9.884917] dhcpcd[722]: DUID 00:01:00:01:32:46:21:0e:52:54:00:12:34:56 lb # [ 9.885980] dhcpcd[722]: eth0: IAID 00:12:34:56 lb # [ 9.887344] dhcpcd[722]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 9.888764] postgresql-pre-start[739]: The files belonging to this database system will be owned by user "postgres". lb # [ 9.890313] postgresql-pre-start[739]: This user must also own the server process. lb # [ 9.893348] postgresql-pre-start[739]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 9.895329] postgresql-pre-start[739]: The default database encoding has accordingly been set to "UTF8". lb # [ 9.896507] postgresql-pre-start[739]: The default text search configuration will be set to "english". lb # [ 9.897687] postgresql-pre-start[739]: Data page checksums are enabled. lb # [ 9.898625] postgresql-pre-start[739]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 9.899892] postgresql-pre-start[739]: creating subdirectories ... ok lb # [ 9.900731] postgresql-pre-start[739]: selecting dynamic shared memory implementation ... posix lb # [ 9.965582] postgresql-pre-start[739]: selecting default "max_connections" ... 100 lb # [ 10.028229] systemd-vconsole-setup[691]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 10.031497] systemd[1]: Finished Virtual Console Setup. lb # [ 10.033247] postgresql-pre-start[739]: selecting default "shared_buffers" ... 128MB worker1 # [ 10.170083] dhcpcd[690]: eth0: soliciting a DHCP lease worker1 # [ 10.320430] NET: Registered PF_PACKET protocol family worker1 # [ 10.194148] dhcpcd[690]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 10.197481] dhcpcd[690]: eth0: probing address 10.0.2.15/24 lb: (finished: waiting for unit network-addresses-eth1.service, in 1.21 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.02 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.05 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.02 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.06 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service lb # [ 10.674994] postgresql-pre-start[739]: selecting default time zone ... UTC lb # [ 10.677695] postgresql-pre-start[739]: creating configuration files ... ok client # [ 10.770369] dhcpcd[559]: eth0: soliciting an IPv6 router client # [ 10.773569] dhcpcd[559]: eth0: Router Advertisement from fe80::2 client # [ 10.775676] dhcpcd[559]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 10.777854] dhcpcd[559]: eth0: adding route to fec0::/64 client # [ 10.779736] dhcpcd[559]: eth0: adding default route via fe80::2 lb # [ 10.839465] postgresql-pre-start[739]: running bootstrap script ... ok worker2 # [ 11.060230] dhcpcd[690]: eth0: soliciting an IPv6 router worker2 # [ 11.062410] dhcpcd[690]: eth0: Router Advertisement from fe80::2 worker2 # [ 11.064518] dhcpcd[690]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 11.066669] dhcpcd[690]: eth0: adding route to fec0::/64 worker2 # [ 11.068421] dhcpcd[690]: eth0: adding default route via fe80::2 lb # [ 11.212442] postgresql-pre-start[739]: performing post-bootstrap initialization ... ok lb # [ 11.358303] postgresql-pre-start[739]: syncing data to disk ... ok lb # [ 11.360060] postgresql-pre-start[739]: initdb: warning: enabling "trust" authentication for local connections lb # [ 11.361237] postgresql-pre-start[739]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 11.363112] postgresql-pre-start[739]: Success. You can now start the database server using: lb # [ 11.364203] postgresql-pre-start[739]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 11.432330] dhcpcd[722]: eth0: soliciting an IPv6 router lb # [ 11.433171] dhcpcd[722]: eth0: Router Advertisement from fe80::2 lb # [ 11.433956] dhcpcd[722]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 11.434909] dhcpcd[722]: eth0: adding route to fec0::/64 lb # [ 11.436198] dhcpcd[722]: eth0: adding default route via fe80::2 lb # [ 11.440588] postgres[782]: [782] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 11.443206] postgres[782]: [782] LOG: listening on IPv6 address "::1", port 5432 lb # [ 11.444352] postgres[782]: [782] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 11.447302] postgres[782]: [782] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 11.459222] postgres[791]: [791] LOG: database system was shut down at 2026-09-23 05:33:03 GMT lb # [ 11.466112] postgres[782]: [782] LOG: database system is ready to accept connections lb # [ 11.470739] systemd[1]: Started PostgreSQL Server. lb # [ 11.476759] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 11.609594] postgresql-setup-start[802]: CREATE DATABASE lb # [ 11.653609] postgresql-setup-start[815]: CREATE ROLE lb # [ 11.668358] postgresql-setup-start[818]: ALTER DATABASE lb # [ 11.672810] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 11.674090] systemd[1]: Reached target PostgreSQL. lb # [ 11.692755] dhcpcd[722]: eth0: soliciting a DHCP lease lb # [ 11.834381] NET: Registered PF_PACKET protocol family lb # [ 11.705325] dhcpcd[722]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.708299] dhcpcd[722]: eth0: probing address 10.0.2.15/24 worker1 # [ 12.409892] dhcpcd[690]: eth0: soliciting an IPv6 router worker1 # [ 12.413484] dhcpcd[690]: eth0: Router Advertisement from fe80::2 worker1 # [ 12.415619] dhcpcd[690]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 12.417889] dhcpcd[690]: eth0: adding route to fec0::/64 worker1 # [ 12.420109] dhcpcd[690]: eth0: adding default route via fe80::2 worker2 # [ 14.376750] dhcpcd[690]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 14.379783] dhcpcd[690]: eth0: adding route to 10.0.2.0/24 worker2 # [ 14.382502] dhcpcd[690]: eth0: adding default route via 10.0.2.2 worker2 # [ 14.464687] systemd[1]: Started DHCP Client. worker2 # [ 14.466162] systemd[1]: Reached target Multi-User System. worker2 # [ 14.466917] systemd[1]: Startup finished in 853ms (kernel) + 3.591s (initrd) + 10.020s (userspace) = 14.465s. worker1 # [ 14.930679] dhcpcd[690]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 14.933674] dhcpcd[690]: eth0: adding route to 10.0.2.0/24 worker1 # [ 14.936190] dhcpcd[690]: eth0: adding default route via 10.0.2.2 worker1 # [ 15.018474] systemd[1]: Started DHCP Client. worker1 # [ 15.020256] systemd[1]: Reached target Multi-User System. worker1 # [ 15.021282] systemd[1]: Startup finished in 845ms (kernel) + 4.592s (initrd) + 9.582s (userspace) = 15.021s. client # [ 15.639089] dhcpcd[559]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 15.641772] dhcpcd[559]: eth0: adding route to 10.0.2.0/24 client # [ 15.644357] dhcpcd[559]: eth0: adding default route via 10.0.2.2 client # [ 15.724580] systemd[1]: Started DHCP Client. client # [ 15.726796] systemd[1]: Reached target Multi-User System. client # [ 15.727915] systemd[1]: Startup finished in 831ms (kernel) + 3.628s (initrd) + 11.267s (userspace) = 15.727s. lb # [ 17.227613] dhcpcd[722]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.230720] dhcpcd[722]: eth0: adding route to 10.0.2.0/24 lb # [ 17.232848] dhcpcd[722]: eth0: adding default route via 10.0.2.2 lb # [ 17.318301] systemd[1]: Started DHCP Client. lb # [ 17.319947] systemd[1]: Reached target Network is Online. lb # [ 17.323297] systemd[1]: Started Envoy reverse proxy. lb # [ 17.327394] systemd[1]: Started mock-oidc.service. lb # [ 17.438679] mock-oidc-server[930]: Mock OIDC Server running lb # [ 17.439544] mock-oidc-server[930]: OIDC Address: 192.168.1.2:8080 lb # [ 17.440538] mock-oidc-server[930]: Issue Address: 0.0.0.0:8081 lb # [ 17.442127] mock-oidc-server[930]: Issuer: http://192.168.1.2:8080/oidc lb # [ 17.443025] mock-oidc-server[930]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 17.444091] mock-oidc-server[930]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 17.445354] mock-oidc-server[930]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 17.594653] envoy[929]: [2026-09-23 05:33:09.856][929][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104) lb # [ 17.597387] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 17.598937] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 17.601048] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 17.603056] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 17.614380] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 17.616242] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 17.618562] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 17.622574] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 17.626772] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 17.628450] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 17.631082] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 17.633123] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 17.634731] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 17.636559] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 17.639192] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 17.641428] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 17.644488] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 17.646626] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 17.649092] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 17.650884] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 17.652672] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 17.657032] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 17.660788] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 17.662922] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 17.666890] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 17.668625] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 17.671780] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 17.674083] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 17.676344] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 17.677876] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 17.680110] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 17.684879] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 17.687199] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 17.692224] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 17.693899] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 17.695708] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 17.699035] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 17.704571] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 17.706102] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 17.708828] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 17.711291] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 17.713581] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 17.739717] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 17.741352] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 17.743192] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 17.745579] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 17.756624] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 17.759205] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 17.762169] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 17.766150] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 17.767781] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 17.769894] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 17.771728] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 17.773439] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 17.776350] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 17.779781] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 17.782226] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 17.783627] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 17.786345] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 17.788168] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 17.790091] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 17.791677] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 17.793706] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 17.797153] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 17.798663] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 17.800802] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 17.802518] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 17.805394] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 17.807709] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 17.813640] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 17.815379] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 17.816790] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 17.820081] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 17.826215] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 17.828367] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 17.830149] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 17.832371] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 17.834077] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 17.839333] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 17.845485] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 17.847201] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 17.850098] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 17.852120] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 17.853926] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 17.856855] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 17.865890] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 17.868343] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 17.870283] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 17.871877] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 17.875082] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 17.876887] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 17.879201] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 17.883700] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 17.885232] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 17.886970] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 17.889672] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 17.896748] envoy[929]: [2026-09-23 05:33:09.858][929][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 17.899513] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 17.901082] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 17.903383] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 17.906307] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 17.908035] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 17.910397] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 17.912657] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 17.914412] envoy[929]: [2026-09-23 05:33:09.859][929][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 17.916259] envoy[929]: [2026-09-23 05:33:09.908][929][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 17.917509] envoy[929]: [2026-09-23 05:33:09.914][929][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 17.930547] envoy[929]: [2026-09-23 05:33:09.914][929][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 17.931970] envoy[929]: [2026-09-23 05:33:09.914][929][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 17.940288] envoy[929]: [2026-09-23 05:33:09.914][929][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 17.946304] envoy[929]: [2026-09-23 05:33:10.207][929][info][main] [source/server/server.cc:968] runtime: {} lb # [ 17.948764] envoy[929]: [2026-09-23 05:33:10.210][929][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 17.950486] envoy[929]: [2026-09-23 05:33:10.212][929][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 17.952344] envoy[929]: [2026-09-23 05:33:10.213][929][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 17.954103] envoy[929]: [2026-09-23 05:33:10.215][929][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 17.956224] envoy[929]: [2026-09-23 05:33:10.217][929][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 17.957727] envoy[929]: [2026-09-23 05:33:10.217][929][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 17.972452] envoy[929]: [2026-09-23 05:33:10.234][929][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 17.976617] envoy[929]: [2026-09-23 05:33:10.238][929][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 17.978636] envoy[929]: [2026-09-23 05:33:10.240][929][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. worker2 # [ 17.966725] systemd[1]: Reached target Network is Online. lb # [ 17.981566] envoy[929]: [2026-09-23 05:33:10.243][929][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 17.981561] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 17.974598] systemd[1]: Starting scheduler-token.service... worker2 # [ 18.049725] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 18.050859] systemd[1]: Finished scheduler-token.service. worker2 # [ 18.051919] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 74ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 18.057180] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 18.107205] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 18.117422] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.119581] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 18.132763] nix-grpc-daemon[780]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 18.134401] nix-grpc-daemon[780]: E0000 00:00:1790141590.339745 780 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.140305] nix-grpc-daemon[780]: E0000 00:00:1790141590.347481 780 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.143055] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 18.148206] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 18.161692] nix-grpc-daemon[780]: E0000 00:00:1790141590.368974 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.157356] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.170450] envoy[929]: [2026-09-23 05:33:10.431][929][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization worker1 # [ 18.177212] nix-grpc-daemon[780]: E0000 00:00:1790141590.384493 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.175543] envoy[929]: [2026-09-23 05:33:10.436][929][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker2 # [ 18.168224] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 18.170343] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 18.182056] nix-grpc-daemon[780]: E0000 00:00:1790141590.386850 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.192599] nix-grpc-daemon[780]: E0000 00:00:1790141590.399890 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.183556] nix-grpc-daemon[783]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 18.185112] nix-grpc-daemon[783]: E0000 00:00:1790141590.533710 783 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.198405] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.190867] nix-grpc-daemon[783]: E0000 00:00:1790141590.541376 783 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.194346] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 18.198094] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 18.209068] nix-grpc-daemon[783]: E0000 00:00:1790141590.559310 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.226768] nix-grpc-daemon[783]: E0000 00:00:1790141590.577190 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.231916] nix-grpc-daemon[783]: E0000 00:00:1790141590.579638 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.242358] nix-grpc-daemon[783]: E0000 00:00:1790141590.592800 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.247984] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 18.399482] nix-grpc-daemon[780]: ts=2026-09-23T05:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.448757] nix-grpc-daemon[783]: ts=2026-09-23T05:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 18.800147] nix-grpc-daemon[780]: ts=2026-09-23T05:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 18.849581] nix-grpc-daemon[783]: ts=2026-09-23T05:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 19.507665] rustfs-bucket-start[962]: mb s3://farm lb # [ 19.514376] systemd[1]: Finished rustfs-bucket.service. lb # [ 19.520945] systemd[1]: Starting niks3 server... worker1 # [ 19.601462] nix-grpc-daemon[780]: ts=2026-09-23T05:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 19.631705] postgres[974]: [974] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 19.634122] postgres[974]: [974] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 19.654267] niks3-server[969]: 2026/09/23 05:33:11 OK 20241026095416_initial_model.sql (12.3ms) lb # [ 19.657568] niks3-server[969]: 2026/09/23 05:33:11 OK 20251210153512_drop_unused_gin_index.sql (3.64ms) lb # [ 19.662917] niks3-server[969]: 2026/09/23 05:33:11 OK 20251218171726_add_pins.sql (4.08ms) worker2 # [ 19.651249] nix-grpc-daemon[783]: ts=2026-09-23T05:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 19.667370] niks3-server[969]: 2026/09/23 05:33:11 OK 20260628120000_add_object_size_and_stats.sql (3.16ms) lb # [ 19.671963] niks3-server[969]: 2026/09/23 05:33:11 OK 20260905000000_add_claims.sql (4.56ms) lb # [ 19.676161] niks3-server[969]: 2026/09/23 05:33:11 OK 20260920000000_drop_claims.sql (4.19ms) lb # [ 19.677466] niks3-server[969]: 2026/09/23 05:33:11 goose: successfully migrated database to version: 20260920000000 lb # [ 19.681554] niks3-server[969]: 2026/09/23 05:33:11 OK 1_commit_pending_closure.sql (4.11ms) lb # [ 19.684594] niks3-server[969]: 2026/09/23 05:33:11 OK 2_object_stats_trigger.sql (2.94ms) lb # [ 19.685676] niks3-server[969]: 2026/09/23 05:33:11 goose: up to current file version: 2 lb # [ 19.689082] niks3-server[969]: 2026/09/23 05:33:11 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 19.714539] niks3-server[969]: 2026/09/23 05:33:11 INFO Read proxy enabled — serving cache objects from S3 lb # [ 19.716291] niks3-server[969]: 2026/09/23 05:33:11 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 19.720881] systemd[1]: Started niks3 server. lb # [ 19.722472] niks3-server[969]: 2026/09/23 05:33:11 INFO systemd watchdog enabled interval=15s lb # [ 19.725077] niks3-server[969]: 2026/09/23 05:33:11 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 19.726242] systemd[1]: Reached target Multi-User System. lb # [ 19.727983] systemd[1]: Startup finished in 848ms (kernel) + 4.595s (initrd) + 14.283s (userspace) = 19.727s. lb: (finished: waiting for unit niks3.service, in 9.65 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.02 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 21.201938] nix-grpc-daemon[780]: ts=2026-09-23T05:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 21.251424] nix-grpc-daemon[783]: ts=2026-09-23T05:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" worker1 # [ 24.402774] nix-grpc-daemon[780]: ts=2026-09-23T05:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 24.452793] nix-grpc-daemon[783]: ts=2026-09-23T05:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 29.403491] nix-grpc-daemon[780]: ts=2026-09-23T05:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 29.453582] nix-grpc-daemon[783]: ts=2026-09-23T05:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 29.723954] nix-grpc-daemon[783]: ts=2026-09-23T05:33:22Z level=info event=scheduler_take_over lb # [ 29.734850] niks3-server[969]: 2026/09/23 05:33:21 INFO lead: acquired remote=192.168.1.4:32998 lb # [ 33.171920] envoy[929]: [2026-09-23 05:33:25.432][929][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 33.178909] envoy[929]: [2026-09-23 05:33:25.432][929][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 33.183377] envoy[929]: [2026-09-23 05:33:25.432][929][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 33.188762] envoy[929]: [2026-09-23 05:33:25.450][929][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 33.726214] envoy[929]: conn peer=[::1]:51222 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=5 lb: (finished: waiting for TCP port 50051 on localhost, in 13.49 seconds) worker2 # [ 33.763798] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:48026 worker2 # [ 33.817512] systemd[1]: Starting scheduler-token.service... worker2 # [ 33.855616] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 33.857229] systemd[1]: Finished scheduler-token.service. worker2 # [ 33.858654] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 40ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 34.404879] nix-grpc-daemon[780]: ts=2026-09-23T05:33:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.399091] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48030 worker2 # [ 34.403694] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 34.454200] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.465652] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 34.468550] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48030 worker2 # [ 34.473077] nix-grpc-daemon[783]: ts=2026-09-23T05:33:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.39 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 35.561327] nix-grpc-daemon[780]: ts=2026-09-23T05:33:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 35.571635] systemd[1]: Started Nix Daemon. worker1 # [ 35.684440] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 35.699070] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5) worker1 # [ 35.728401] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5) worker1 # [ 35.915199] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 357 ms (attempt 3/5) worker1 # [ 36.274082] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 318 ms (attempt 4/5) worker1 # [ 36.598095] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 36.605181] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 36.622661] nix-grpc-daemon[783]: ts=2026-09-23T05:33:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 36.633491] systemd[1]: Started Nix Daemon. worker2 # [ 36.748077] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 36.772288] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5) worker2 # [ 36.869070] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 2/5) worker2 # [ 36.945145] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 390 ms (attempt 3/5) worker2 # [ 37.336702] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 781 ms (attempt 4/5) worker2 # [ 38.121575] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 38.126218] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 38.138148] nix-grpc-daemon[783]: ts=2026-09-23T05:33:30Z level=debug event=path_info_miss path=qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test error="error: path '/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 38.169573] nix-grpc-daemon[780]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 38.181408] nix-grpc-daemon[780]: ts=2026-09-23T05:33:30Z level=debug event=path_info_miss path=qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test error="error: path '/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds) client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds) worker1 # [ 38.417270] nix-grpc-daemon[780]: ts=2026-09-23T05:33:30Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 38.420629] nix-grpc-daemon[780]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:56936 worker2 # [ 38.425267] nix-grpc-daemon[783]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:48046 worker2 # [ 38.435201] nix-grpc-daemon[783]: ts=2026-09-23T05:33:30Z level=debug event=path_info_miss path=qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test error="error: path '/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 38.444299] nix-grpc-daemon[783]: ts=2026-09-23T05:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:48046 worker2 # [ 38.453394] nix-grpc-daemon[783]: ts=2026-09-23T05:33:30Z level=debug event=path_info_miss path=qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test error="error: path '/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds) worker1 # [ 38.617180] nix-grpc-daemon[780]: ts=2026-09-23T05:33:30Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 38.742283] nix-grpc-daemon[783]: ts=2026-09-23T05:33:31Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.40 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 worker1 # [ 38.891489] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker1 # [ 38.900608] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 38.920179] nix-grpc-daemon[783]: ts=2026-09-23T05:33:31Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=0 worker2 # [ 38.925257] nix-grpc-daemon[783]: ts=2026-09-23T05:33:31Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 38.981167] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 38.994341] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying 1 paths to 'lb:50051'... worker1 # [ 39.006446] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'... worker1 # [ 39.027936] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 39.058317] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=info event=niks3_push_started pid=860 worker1 # [ 39.206837] nix-grpc-daemon[860]: time=2026-09-23T05:33:31.413Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 39.299551] nix-daemon[847]: accepted connection from pid 873, user nix-grpc-daemon (trusted) worker1 # [ 39.313314] nix-daemon[847]: reaped child process 876, status = succeeded lb # [ 39.322512] niks3-server[969]: 2026/09/23 05:33:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 39.339527] nix-grpc-daemon[860]: time=2026-09-23T05:33:31.546Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 39.341069] nix-grpc-daemon[860]: time=2026-09-23T05:33:31.548Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)" lb # [ 39.370366] niks3-server[969]: 2026/09/23 05:33:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 39.387340] niks3-server[969]: 2026/09/23 05:33:31 INFO Signed narinfos id=1 count=1 worker1 # [ 39.391247] nix-grpc-daemon[860]: time=2026-09-23T05:33:31.597Z level=INFO msg="Uploading 1 narinfos" lb # [ 39.410066] niks3-server[969]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst lb # [ 39.412659] niks3-server[969]: 2026/09/23 05:33:31 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 39.420587] niks3-server[969]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo lb # [ 39.430451] niks3-server[969]: 2026/09/23 05:33:31 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls lb # [ 39.437046] niks3-server[969]: 2026/09/23 05:33:31 INFO Completed upload id=1 worker1 # [ 39.438073] nix-grpc-daemon[860]: time=2026-09-23T05:33:31.645Z level=INFO msg="Upload complete. (226ms)" worker1 # [ 39.446444] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 39.452331] nix-grpc-daemon[780]: ts=2026-09-23T05:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 39.462382] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 39.473325] nix-daemon[847]: reaped child process 858, status = succeeded client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051... worker1 # [ 39.482371] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 40.511570] nix-daemon[847]: reaped child process 880, status = succeeded worker1 # [ 40.601845] nix-daemon[847]: accepted connection from pid 896, user nix-grpc-daemon (trusted) worker1 # [ 40.613716] nix-daemon[847]: reaped child process 899, status = succeeded lb # [ 40.622282] niks3-server[969]: 2026/09/23 05:33:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 40.634224] nix-grpc-daemon[860]: time=2026-09-23T05:33:32.841Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 40.637057] nix-grpc-daemon[860]: time=2026-09-23T05:33:32.843Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)" lb # [ 40.661898] niks3-server[969]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 40.676556] niks3-server[969]: 2026/09/23 05:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 40.681486] niks3-server[969]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls lb # [ 40.688720] niks3-server[969]: 2026/09/23 05:33:32 INFO Signed narinfos id=2 count=1 worker1 # [ 40.689151] nix-grpc-daemon[860]: time=2026-09-23T05:33:32.895Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.694495] niks3-server[969]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv lb # [ 40.704720] niks3-server[969]: 2026/09/23 05:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 40.711550] niks3-server[969]: 2026/09/23 05:33:32 INFO Completed upload id=2 lb # [ 40.713085] niks3-server[969]: 2026/09/23 05:33:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo worker1 # [ 40.712347] nix-grpc-daemon[860]: time=2026-09-23T05:33:32.919Z level=INFO msg="Upload complete. (217ms)" worker1 # [ 40.718965] nix-grpc-daemon[780]: ts=2026-09-23T05:33:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=1 worker2 # [ 40.718586] nix-grpc-daemon[783]: ts=2026-09-23T05:33:33Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=0 worker1 # [ 40.734236] nix-daemon[847]: reaped child process 878, status = succeeded worker2 # [ 40.722671] nix-grpc-daemon[783]: ts=2026-09-23T05:33:33Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 40.773859] nix-grpc-daemon[780]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 40.785782] nix-grpc-daemon[780]: ts=2026-09-23T05:33:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 40.799441] nix-grpc-daemon[860]: time=2026-09-23T05:33:33.006Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 40.807662] nix-grpc-daemon[780]: ts=2026-09-23T05:33:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'... worker1 # [ 40.824897] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 40.927806] nix-daemon[847]: accepted connection from pid 903, user nix-grpc-daemon (trusted) worker1 # [ 40.940777] nix-daemon[847]: reaped child process 906, status = succeeded lb # [ 40.949064] niks3-server[969]: 2026/09/23 05:33:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 40.958216] nix-grpc-daemon[860]: time=2026-09-23T05:33:33.165Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 40.959841] nix-grpc-daemon[860]: time=2026-09-23T05:33:33.167Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)" lb # [ 40.982813] niks3-server[969]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst lb # [ 40.989250] niks3-server[969]: 2026/09/23 05:33:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 40.997628] niks3-server[969]: 2026/09/23 05:33:33 INFO Signed narinfos id=3 count=1 worker1 # [ 41.001769] nix-grpc-daemon[860]: time=2026-09-23T05:33:33.208Z level=INFO msg="Uploading 1 narinfos" lb # [ 41.005516] niks3-server[969]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls lb # [ 41.014054] niks3-server[969]: 2026/09/23 05:33:33 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 41.022813] niks3-server[969]: 2026/09/23 05:33:33 INFO Completed upload id=3 worker1 # [ 41.023300] nix-grpc-daemon[860]: time=2026-09-23T05:33:33.230Z level=INFO msg="Upload complete. (184ms)" lb # [ 41.024981] niks3-server[969]: 2026/09/23 05:33:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo worker1 # [ 41.028142] nix-grpc-daemon[780]: ts=2026-09-23T05:33:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 41.034266] nix-grpc-daemon[780]: ts=2026-09-23T05:33:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 41.045829] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 41.052341] nix-daemon[847]: reaped child process 901, status = succeeded worker1 # [ 41.057354] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051... client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 42.471897] nix-daemon[847]: reaped child process 910, status = succeeded worker1 # [ 42.547786] nix-daemon[847]: accepted connection from pid 927, user nix-grpc-daemon (trusted) worker1 # [ 42.560240] nix-daemon[847]: reaped child process 930, status = succeeded lb # [ 42.567832] niks3-server[969]: 2026/09/23 05:33:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.579107] nix-grpc-daemon[860]: time=2026-09-23T05:33:34.786Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 42.581433] nix-grpc-daemon[860]: time=2026-09-23T05:33:34.787Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)" lb # [ 42.610418] niks3-server[969]: 2026/09/23 05:33:34 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv lb # [ 42.616463] niks3-server[969]: 2026/09/23 05:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker1 # [ 42.626708] nix-grpc-daemon[860]: time=2026-09-23T05:33:34.833Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.625996] niks3-server[969]: 2026/09/23 05:33:34 INFO Signed narinfos id=4 count=1 lb # [ 42.632350] niks3-server[969]: 2026/09/23 05:33:34 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst lb # [ 42.639325] niks3-server[969]: 2026/09/23 05:33:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls lb # [ 42.646178] niks3-server[969]: 2026/09/23 05:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 42.651041] niks3-server[969]: 2026/09/23 05:33:34 INFO Completed upload id=4 worker1 # [ 42.651651] nix-grpc-daemon[860]: time=2026-09-23T05:33:34.858Z level=INFO msg="Upload complete. (187ms)" lb # [ 42.653094] niks3-server[969]: 2026/09/23 05:33:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo worker1 # [ 42.657696] nix-grpc-daemon[780]: ts=2026-09-23T05:33:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=1 worker2 # [ 42.654305] nix-grpc-daemon[783]: ts=2026-09-23T05:33:35Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=0 worker2 # [ 42.658845] nix-grpc-daemon[783]: ts=2026-09-23T05:33:35Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 42.673221] nix-daemon[847]: reaped child process 908, status = succeeded worker1 # [ 42.715187] nix-grpc-daemon[780]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 42.727312] nix-grpc-daemon[780]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 42.740122] nix-grpc-daemon[860]: time=2026-09-23T05:33:34.947Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 42.748803] nix-grpc-daemon[780]: ts=2026-09-23T05:33:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'... worker1 # [ 42.768138] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 42.869319] nix-daemon[847]: accepted connection from pid 934, user nix-grpc-daemon (trusted) worker1 # [ 42.881411] nix-daemon[847]: reaped child process 937, status = succeeded lb # [ 42.889721] niks3-server[969]: 2026/09/23 05:33:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.898906] nix-grpc-daemon[860]: time=2026-09-23T05:33:35.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 42.900457] nix-grpc-daemon[860]: time=2026-09-23T05:33:35.107Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)" lb # [ 42.920242] niks3-server[969]: 2026/09/23 05:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 42.928793] niks3-server[969]: 2026/09/23 05:33:35 INFO Signed narinfos id=5 count=1 worker1 # [ 42.934910] nix-grpc-daemon[860]: time=2026-09-23T05:33:35.142Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.936974] niks3-server[969]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst lb # [ 42.943253] niks3-server[969]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls lb # [ 42.956449] niks3-server[969]: 2026/09/23 05:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 42.962486] niks3-server[969]: 2026/09/23 05:33:35 INFO Completed upload id=5 worker1 # [ 42.963041] nix-grpc-daemon[860]: time=2026-09-23T05:33:35.170Z level=INFO msg="Upload complete. (182ms)" worker1 # [ 42.966466] nix-grpc-daemon[780]: ts=2026-09-23T05:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 42.969862] niks3-server[969]: 2026/09/23 05:33:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo worker1 # [ 42.972401] nix-grpc-daemon[780]: ts=2026-09-23T05:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 42.982676] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 42.989707] nix-daemon[847]: reaped child process 932, status = succeeded worker1 # [ 42.994729] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051... client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 43.827884] nix-daemon[847]: reaped child process 941, status = succeeded worker1 # [ 43.914658] nix-daemon[847]: accepted connection from pid 957, user nix-grpc-daemon (trusted) worker1 # [ 43.925853] nix-daemon[847]: reaped child process 960, status = succeeded lb # [ 43.934445] niks3-server[969]: 2026/09/23 05:33:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.944213] nix-grpc-daemon[860]: time=2026-09-23T05:33:36.151Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 43.946241] nix-grpc-daemon[860]: time=2026-09-23T05:33:36.153Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)" lb # [ 43.972174] niks3-server[969]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls lb # [ 43.982676] niks3-server[969]: 2026/09/23 05:33:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 43.993944] niks3-server[969]: 2026/09/23 05:33:36 INFO Signed narinfos id=6 count=1 worker1 # [ 43.997215] nix-grpc-daemon[860]: time=2026-09-23T05:33:36.204Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.997255] niks3-server[969]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst lb # [ 44.000961] niks3-server[969]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv lb # [ 44.013328] niks3-server[969]: 2026/09/23 05:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 44.016336] niks3-server[969]: 2026/09/23 05:33:36 INFO Completed upload id=6 lb # [ 44.017737] niks3-server[969]: 2026/09/23 05:33:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo worker1 # [ 44.017138] nix-grpc-daemon[860]: time=2026-09-23T05:33:36.224Z level=INFO msg="Upload complete. (214ms)" worker1 # [ 44.024156] nix-grpc-daemon[780]: ts=2026-09-23T05:33:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=1 worker1 # [ 44.037550] nix-daemon[847]: reaped child process 939, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.29 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds) client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top client # copying 3 paths... client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.64 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds) worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker2 # [ 44.862398] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 44.867753] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 44.897075] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 0.79 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2 worker2 # [ 45.414878] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48046 client # don't know how to build these paths: client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker2 # [ 45.422870] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 45.435187] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=0 worker1 # [ 45.450623] nix-grpc-daemon[780]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 45.439677] nix-grpc-daemon[783]: ts=2026-09-23T05:33:37Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 45.463489] nix-grpc-daemon[780]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 45.476179] nix-grpc-daemon[860]: time=2026-09-23T05:33:37.682Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'... worker1 # [ 45.483657] nix-grpc-daemon[780]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 45.494997] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 45.593867] nix-daemon[847]: accepted connection from pid 977, user nix-grpc-daemon (trusted) worker1 # [ 45.606875] nix-daemon[847]: reaped child process 980, status = succeeded lb # [ 45.614768] niks3-server[969]: 2026/09/23 05:33:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.626169] nix-grpc-daemon[860]: time=2026-09-23T05:33:37.833Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.627515] nix-grpc-daemon[860]: time=2026-09-23T05:33:37.833Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)" lb # [ 45.651386] niks3-server[969]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls lb # [ 45.654260] niks3-server[969]: 2026/09/23 05:33:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker1 # [ 45.661090] nix-grpc-daemon[860]: time=2026-09-23T05:33:37.868Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.660936] niks3-server[969]: 2026/09/23 05:33:37 INFO Signed narinfos id=7 count=1 lb # [ 45.668458] niks3-server[969]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst lb # [ 45.676867] niks3-server[969]: 2026/09/23 05:33:37 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 45.679819] niks3-server[969]: 2026/09/23 05:33:37 INFO Completed upload id=7 worker1 # [ 45.681856] nix-grpc-daemon[860]: time=2026-09-23T05:33:37.889Z level=INFO msg="Upload complete. (175ms)" lb # [ 45.683906] niks3-server[969]: 2026/09/23 05:33:37 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo worker1 # [ 45.685911] nix-grpc-daemon[780]: ts=2026-09-23T05:33:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 45.690718] nix-grpc-daemon[780]: ts=2026-09-23T05:33:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 45.702168] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 45.707077] nix-daemon[847]: reaped child process 975, status = succeeded worker1 # [ 45.710870] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051... client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 46.547814] nix-daemon[847]: reaped child process 984, status = succeeded worker1 # [ 46.626153] nix-daemon[847]: accepted connection from pid 1000, user nix-grpc-daemon (trusted) worker1 # [ 46.639615] nix-daemon[847]: reaped child process 1003, status = succeeded lb # [ 46.646839] niks3-server[969]: 2026/09/23 05:33:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 46.655884] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.863Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 46.657878] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.864Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)" lb # [ 46.682057] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv lb # [ 46.694184] niks3-server[969]: 2026/09/23 05:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 46.698890] niks3-server[969]: 2026/09/23 05:33:38 INFO Signed narinfos id=8 count=1 worker1 # [ 46.704457] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.911Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.708076] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls lb # [ 46.712148] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst lb # [ 46.722238] niks3-server[969]: 2026/09/23 05:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 46.727178] niks3-server[969]: 2026/09/23 05:33:38 INFO Completed upload id=8 worker1 # [ 46.727771] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.935Z level=INFO msg="Upload complete. (210ms)" lb # [ 46.730079] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo worker1 # [ 46.733705] nix-grpc-daemon[780]: ts=2026-09-23T05:33:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=1 worker1 # [ 46.748120] nix-daemon[847]: reaped child process 982, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.48 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.80 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker2 # [ 47.103535] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 47.109305] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 47.126088] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=0 worker2 # [ 47.132566] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 47.137348] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 47.147714] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'... worker2 # [ 47.158827] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 47.169972] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 47.189942] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=info event=niks3_push_started pid=922 worker2 # [ 47.311889] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.661Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 47.403548] nix-daemon[870]: accepted connection from pid 935, user nix-grpc-daemon (trusted) worker2 # [ 47.415700] nix-daemon[870]: reaped child process 938, status = succeeded lb # [ 47.434543] niks3-server[969]: 2026/09/23 05:33:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.431911] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.782Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 47.433401] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.783Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)" lb # [ 47.471116] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls lb # [ 47.477389] niks3-server[969]: 2026/09/23 05:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 47.484123] niks3-server[969]: 2026/09/23 05:33:39 INFO Signed narinfos id=9 count=1 worker2 # [ 47.476944] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.826Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.498151] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst lb # [ 47.504943] niks3-server[969]: 2026/09/23 05:33:39 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 47.511143] niks3-server[969]: 2026/09/23 05:33:39 INFO Completed upload id=9 worker2 # [ 47.500878] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.851Z level=INFO msg="Upload complete. (184ms)" lb # [ 47.516463] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo worker2 # [ 47.504954] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 47.510208] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 47.518505] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 47.523865] nix-daemon[870]: reaped child process 920, status = succeeded worker2 # [ 47.528458] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051... client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 47.977509] envoy[929]: conn peer=[2001:db8:1::1]:39270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2650 lb # [ 47.981360] envoy[929]: conn peer=[2001:db8:1::1]:39286 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=64 lb # [ 47.984733] envoy[929]: conn peer=[2001:db8:1::1]:39288 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=7 lb # [ 47.988687] envoy[929]: conn peer=[2001:db8:1::1]:39292 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=8 lb # [ 47.992441] envoy[929]: conn peer=[2001:db8:1::1]:39302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4381 ms=5154 lb # [ 47.996063] envoy[929]: conn peer=[2001:db8:1::1]:59614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=49 lb # [ 47.999445] envoy[929]: conn peer=[2001:db8:1::1]:59624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1872 ms=1317 worker1 # [ 48.166756] nix-grpc-daemon[780]: E0000 00:00:1790141620.372780 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.196635] nix-grpc-daemon[780]: E0000 00:00:1790141620.403724 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 48.207134] nix-grpc-daemon[780]: E0000 00:00:1790141620.409309 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 48.213741] nix-grpc-daemon[783]: E0000 00:00:1790141620.562877 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 48.228063] nix-grpc-daemon[780]: E0000 00:00:1790141620.435355 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 48.245131] nix-grpc-daemon[783]: E0000 00:00:1790141620.594632 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 48.254131] nix-grpc-daemon[783]: E0000 00:00:1790141620.601276 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 48.276077] nix-grpc-daemon[783]: E0000 00:00:1790141620.626393 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 48.948910] nix-daemon[870]: reaped child process 942, status = succeeded worker2 # [ 49.029042] nix-daemon[870]: accepted connection from pid 958, user nix-grpc-daemon (trusted) worker2 # [ 49.043165] nix-daemon[870]: reaped child process 961, status = succeeded lb # [ 49.063251] niks3-server[969]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 49.061615] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 49.063128] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.413Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)" lb # [ 49.107770] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 49.113368] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv lb # [ 49.118533] niks3-server[969]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 49.123618] niks3-server[969]: 2026/09/23 05:33:41 INFO Signed narinfos id=10 count=1 worker2 # [ 49.113753] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.463Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.129118] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls lb # [ 49.139195] niks3-server[969]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 49.142955] niks3-server[969]: 2026/09/23 05:33:41 INFO Completed upload id=10 worker2 # [ 49.132385] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.482Z level=INFO msg="Upload complete. (195ms)" lb # [ 49.144576] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo worker2 # [ 49.138262] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 worker2 # [ 49.147874] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=0 worker2 # [ 49.151652] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 49.157312] nix-daemon[870]: reaped child process 940, status = succeeded worker2 # [ 49.191778] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 49.203952] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 49.216310] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.566Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'... worker2 # [ 49.225523] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 49.235136] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 49.337881] nix-daemon[870]: accepted connection from pid 967, user nix-grpc-daemon (trusted) worker2 # [ 49.354064] nix-daemon[870]: reaped child process 970, status = succeeded lb # [ 49.371642] niks3-server[969]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 49.369056] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 49.371162] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.720Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)" lb # [ 49.402128] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls lb # [ 49.410476] niks3-server[969]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 49.415495] niks3-server[969]: 2026/09/23 05:33:41 INFO Signed narinfos id=11 count=1 worker2 # [ 49.408349] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.758Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.431059] niks3-server[969]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 49.438220] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst lb # [ 49.447108] niks3-server[969]: 2026/09/23 05:33:41 INFO Completed upload id=11 worker2 # [ 49.436778] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.787Z level=INFO msg="Upload complete. (190ms)" lb # [ 49.449231] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo worker2 # [ 49.442144] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 49.448309] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 49.459640] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 49.467484] nix-daemon[870]: reaped child process 965, status = succeeded worker2 # [ 49.472674] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051... client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 50.713670] nix-daemon[870]: reaped child process 974, status = succeeded worker2 # [ 50.796905] nix-daemon[870]: accepted connection from pid 990, user nix-grpc-daemon (trusted) worker2 # [ 50.809817] nix-daemon[870]: reaped child process 993, status = succeeded lb # [ 50.829258] niks3-server[969]: 2026/09/23 05:33:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 50.827869] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.178Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 50.829305] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.179Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)" lb # [ 50.863076] niks3-server[969]: 2026/09/23 05:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 50.865661] niks3-server[969]: 2026/09/23 05:33:43 INFO Signed narinfos id=12 count=1 worker2 # [ 50.855163] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.205Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.870614] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst lb # [ 50.873479] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv lb # [ 50.882044] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls lb # [ 50.884439] niks3-server[969]: 2026/09/23 05:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 50.888556] niks3-server[969]: 2026/09/23 05:33:43 INFO Completed upload id=12 worker2 # [ 50.879319] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.229Z level=INFO msg="Upload complete. (198ms)" lb # [ 50.891514] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo worker2 # [ 50.883841] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 worker2 # [ 50.889783] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=0 worker2 # [ 50.891835] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 50.897399] nix-daemon[870]: reaped child process 972, status = succeeded worker2 # [ 50.933655] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 50.942742] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 50.951411] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.301Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'... worker2 # [ 50.960524] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 50.967467] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 51.066430] nix-daemon[870]: accepted connection from pid 999, user nix-grpc-daemon (trusted) worker2 # [ 51.081804] nix-daemon[870]: reaped child process 1002, status = succeeded lb # [ 51.098889] niks3-server[969]: 2026/09/23 05:33:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.097283] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.447Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 51.098797] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.449Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)" lb # [ 51.131859] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst lb # [ 51.137109] niks3-server[969]: 2026/09/23 05:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 51.141316] niks3-server[969]: 2026/09/23 05:33:43 INFO Signed narinfos id=13 count=1 worker2 # [ 51.133311] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.483Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.152500] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls lb # [ 51.156056] niks3-server[969]: 2026/09/23 05:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 51.159385] niks3-server[969]: 2026/09/23 05:33:43 INFO Completed upload id=13 worker2 # [ 51.150069] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.500Z level=INFO msg="Upload complete. (171ms)" lb # [ 51.164274] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo worker2 # [ 51.152645] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 51.157525] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 51.166060] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 51.170229] nix-daemon[870]: reaped child process 997, status = succeeded worker2 # [ 51.175683] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051... client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 52.620269] nix-daemon[870]: reaped child process 1006, status = succeeded worker2 # [ 52.704099] nix-daemon[870]: accepted connection from pid 1022, user nix-grpc-daemon (trusted) worker2 # [ 52.718169] nix-daemon[870]: reaped child process 1025, status = succeeded lb # [ 52.736296] niks3-server[969]: 2026/09/23 05:33:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.734847] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.085Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.737957] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.086Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)" lb # [ 52.777985] niks3-server[969]: 2026/09/23 05:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 52.779451] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.129Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.789829] niks3-server[969]: 2026/09/23 05:33:45 INFO Signed narinfos id=14 count=1 lb # [ 52.793311] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls lb # [ 52.799695] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv lb # [ 52.803820] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst lb # [ 52.815513] niks3-server[969]: 2026/09/23 05:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 52.820414] niks3-server[969]: 2026/09/23 05:33:45 INFO Completed upload id=14 worker2 # [ 52.810221] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.160Z level=INFO msg="Upload complete. (225ms)" lb # [ 52.822820] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo worker2 # [ 52.818812] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 worker2 # [ 52.833559] nix-daemon[870]: reaped child process 1004, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.87 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 53.080804] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:58908 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker1 # [ 53.124864] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:58908 worker1 # [ 53.131626] nix-grpc-daemon[780]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'... worker1 # [ 53.146464] nix-grpc-daemon[780]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'... worker1 # [ 53.158547] nix-grpc-daemon[780]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top worker2 # [ 53.299250] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47238 worker2 # [ 53.341903] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47238 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds) client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/804" client # removing stale temporary roots file "/nix/var/nix/temproots/841" client # removing stale temporary roots file "/nix/var/nix/temproots/897" client # removing stale temporary roots file "/nix/var/nix/temproots/820" client # removing stale temporary roots file "/nix/var/nix/temproots/861" client # removing stale temporary roots file "/nix/var/nix/temproots/882" client # removing stale temporary roots file "/nix/var/nix/temproots/874" client # removing stale temporary roots file "/nix/var/nix/temproots/853" client # deleting specified paths... client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 53.664758] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 53.671075] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 53.676277] nix-grpc-daemon[860]: time=2026-09-23T05:33:45.882Z level=INFO msg="All 1 paths already cached" worker1 # [ 53.682672] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 53.689333] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 53.695989] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying 3 paths... worker2 # [ 53.696725] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'... worker2 # [ 53.712793] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 53.718215] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 53.724721] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker1 # [ 53.739121] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 53.728473] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 53.743492] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.41 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 6.84 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 53.861634] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: dc4e3751560f485d9ab65d464c10fde6 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup client # [ 53.989710] nix[950]: don't know how to build these paths: client # [ 53.990669] nix[950]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker1 # [ 53.988384] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 53.987987] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker1 # [ 54.002186] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 53.992990] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 54.013754] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 client # [ 54.023691] nix[950]: copying 1 paths to 'lb:50051'... client # [ 54.024705] nix[950]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'... worker1 # [ 54.024934] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 54.036102] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 54.142756] nix-daemon[847]: accepted connection from pid 1011, user nix-grpc-daemon (trusted) worker1 # [ 54.156704] nix-daemon[847]: reaped child process 1014, status = succeeded lb # [ 54.163773] niks3-server[969]: 2026/09/23 05:33:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.173529] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 54.174966] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.382Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)" lb # [ 54.195617] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst lb # [ 54.202246] niks3-server[969]: 2026/09/23 05:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 54.211271] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.418Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.210881] niks3-server[969]: 2026/09/23 05:33:46 INFO Signed narinfos id=15 count=1 lb # [ 54.215703] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls lb # [ 54.228126] niks3-server[969]: 2026/09/23 05:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 54.234757] niks3-server[969]: 2026/09/23 05:33:46 INFO Completed upload id=15 worker1 # [ 54.235282] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.442Z level=INFO msg="Upload complete. (187ms)" lb # [ 54.237374] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo worker1 # [ 54.239674] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 54.245254] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 54.253873] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 54.262255] nix-daemon[847]: reaped child process 1009, status = succeeded worker1 # [ 54.267415] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # [ 54.266352] nix[950]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client # [ 54.916175] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: ed332e3a3f944750b206fd51a2aa0bd3 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds) client # [ 55.081331] nix[966]: don't know how to build these paths: worker2 # [ 55.068808] nix-grpc-daemon[783]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48046 client # [ 55.082154] nix[966]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker2 # [ 55.080667] nix-grpc-daemon[783]: ts=2026-09-23T05:33:47Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=0 worker1 # [ 55.094846] nix-grpc-daemon[780]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 client # [ 55.099719] nix[966]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 lb # [ 57.978616] envoy[929]: conn peer=[2001:db8:1::1]:59628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=4354 ms=5725 lb # [ 57.982493] envoy[929]: conn peer=[2001:db8:1::1]:59888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=153 lb # [ 57.986259] envoy[929]: conn peer=[2001:db8:1::1]:59902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=78 lb # [ 57.990294] envoy[929]: conn peer=[2001:db8:1::1]:59896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=377 ms=86 lb # [ 57.993762] envoy[929]: conn peer=[2001:db8:1::1]:59892 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=93 client: (finished: must succeed: sleep 2, in 2.04 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 58.285735] nix-daemon[847]: reaped child process 1018, status = succeeded worker1 # [ 58.362878] nix-daemon[847]: accepted connection from pid 1049, user nix-grpc-daemon (trusted) worker1 # [ 58.373963] nix-daemon[847]: reaped child process 1052, status = succeeded lb # [ 58.383446] niks3-server[969]: 2026/09/23 05:33:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 58.391342] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 58.392933] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.600Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)" lb # [ 58.436554] niks3-server[969]: 2026/09/23 05:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker1 # [ 58.441096] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.648Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.441888] niks3-server[969]: 2026/09/23 05:33:50 INFO Signed narinfos id=16 count=1 lb # [ 58.443935] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 58.452259] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls lb # [ 58.457687] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv lb # [ 58.463488] niks3-server[969]: 2026/09/23 05:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 58.467905] niks3-server[969]: 2026/09/23 05:33:50 INFO Completed upload id=16 worker1 # [ 58.468422] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.675Z level=INFO msg="Upload complete. (189ms)" lb # [ 58.473837] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo worker1 # [ 58.474818] nix-grpc-daemon[780]: ts=2026-09-23T05:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 58.482614] nix-grpc-daemon[780]: ts=2026-09-23T05:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 58.494179] nix-daemon[847]: reaped child process 1016, status = succeeded client # [ 58.505774] systemd[1]: dedup1.service: Deactivated successfully. client # [ 58.510279] systemd[1]: dedup1.service: Consumed 113ms CPU time over 4.646s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 58.546112] systemd[1]: dedup2.service: Deactivated successfully. client # [ 58.548926] systemd[1]: dedup2.service: Consumed 135ms CPU time over 3.632s wall clock time, 33.4M memory peak, 6.1K incoming IP traffic, 5.8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 59.711918] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 59.724231] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=0 worker2 # [ 59.728153] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.735755] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 59.748059] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 59.759803] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 59.770809] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.874434] nix-daemon[870]: accepted connection from pid 1056, user nix-grpc-daemon (trusted) worker2 # [ 59.889778] nix-daemon[870]: reaped child process 1059, status = succeeded lb # [ 59.908437] niks3-server[969]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.906614] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.256Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 59.908626] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.258Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)" lb # [ 59.937888] niks3-server[969]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 59.935545] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.285Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.947713] niks3-server[969]: 2026/09/23 05:33:52 INFO Signed narinfos id=17 count=1 lb # [ 59.953094] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst lb # [ 59.959147] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls lb # [ 59.973637] niks3-server[969]: 2026/09/23 05:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker2 # [ 59.969535] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.319Z level=INFO msg="Upload complete. (187ms)" lb # [ 59.979957] niks3-server[969]: 2026/09/23 05:33:52 INFO Completed upload id=17 lb # [ 59.983135] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo worker2 # [ 59.972731] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 59.977701] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 59.986521] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 59.993471] nix-daemon[870]: reaped child process 1054, status = succeeded worker2 # [ 59.998616] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 65.810172] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 65.821623] nix-daemon[870]: reaped child process 1061, status = succeeded worker2 # [ 65.822692] nix-daemon[870]: reaped child process 1063, status = succeeded worker1 # [ 66.025577] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 66.024222] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=0 worker2 # [ 66.028589] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 66.047059] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 66.059680] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 66.070922] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 66.081450] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 66.186748] nix-daemon[847]: accepted connection from pid 1065, user nix-grpc-daemon (trusted) worker1 # [ 66.200587] nix-daemon[847]: reaped child process 1068, status = succeeded lb # [ 66.208224] niks3-server[969]: 2026/09/23 05:33:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.217731] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.424Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.219854] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.426Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)" lb # [ 66.240303] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst lb # [ 66.253254] niks3-server[969]: 2026/09/23 05:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 66.261300] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.468Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.260968] niks3-server[969]: 2026/09/23 05:33:58 INFO Signed narinfos id=18 count=1 lb # [ 66.267242] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls lb # [ 66.278446] niks3-server[969]: 2026/09/23 05:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 66.285748] niks3-server[969]: 2026/09/23 05:33:58 INFO Completed upload id=18 lb # [ 66.288694] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo worker1 # [ 66.288385] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.495Z level=INFO msg="Upload complete. (195ms)" worker1 # [ 66.294514] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 66.300495] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 66.312109] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 66.320189] nix-daemon[847]: reaped child process 1063, status = succeeded worker1 # [ 66.327453] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) lb # [ 67.979166] envoy[929]: conn peer=[2001:db8:1::1]:59912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1543 ms=4499 lb # [ 67.982880] envoy[929]: conn peer=[2001:db8:1::1]:59924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=3461 lb # [ 67.986429] envoy[929]: conn peer=[2001:db8:1::1]:59932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=935 ms=6159 worker1 # [ 72.341637] nix-grpc-daemon[780]: ts=2026-09-23T05:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 72.366355] nix-daemon[847]: reaped child process 1070, status = succeeded worker1 # [ 72.368727] nix-daemon[847]: reaped child process 1072, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 12.93 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share client # [ 72.552436] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 35d75a25c14447419755cf8ccedbdda8 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share worker1 # [ 72.689267] nix-grpc-daemon[780]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 client # [ 72.695192] nix[1037]: don't know how to build these paths: client # [ 72.696067] nix[1037]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker2 # [ 72.693812] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 worker2 # [ 72.699810] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 72.704312] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 72.715632] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 client # [ 72.736033] nix[1037]: copying 1 paths to 'lb:50051'... client # [ 72.736872] nix[1037]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'... worker2 # [ 72.726163] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 72.737087] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 72.836082] nix-daemon[870]: accepted connection from pid 1090, user nix-grpc-daemon (trusted) worker2 # [ 72.848835] nix-daemon[870]: reaped child process 1093, status = succeeded lb # [ 72.868835] niks3-server[969]: 2026/09/23 05:34:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 72.864878] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 72.866772] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.216Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)" lb # [ 72.897128] niks3-server[969]: 2026/09/23 05:34:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 72.894993] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.245Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.906134] niks3-server[969]: 2026/09/23 05:34:05 INFO Signed narinfos id=19 count=1 lb # [ 72.919070] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls lb # [ 72.923547] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst lb # [ 72.934060] niks3-server[969]: 2026/09/23 05:34:05 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 72.939116] niks3-server[969]: 2026/09/23 05:34:05 INFO Completed upload id=19 worker2 # [ 72.928496] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.278Z level=INFO msg="Upload complete. (179ms)" lb # [ 72.942374] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo worker2 # [ 72.933143] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 72.940112] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 72.948805] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 72.959152] nix-daemon[870]: reaped child process 1088, status = succeeded worker2 # [ 72.963090] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # [ 72.972792] nix[1037]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... client # [ 73.618861] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 8b3e7b28e3224dbe8549663f3e4e2d9e client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds) client # [ 73.765774] nix[1053]: don't know how to build these paths: client # [ 73.766663] nix[1053]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker1 # [ 73.764873] nix-grpc-daemon[780]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 73.760910] nix-grpc-daemon[783]: ts=2026-09-23T05:34:06Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=0 client # [ 73.774619] nix[1053]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051... worker2 # [ 73.764267] nix-grpc-daemon[783]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 76.884480] systemd[1]: share1.service: Sent signal SIGINT to main process 1037 (nix) on client request. client # [ 76.892338] nix[1037]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 76.898071] nix[1037]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 76.941261] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 76.942713] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 76.944617] systemd[1]: share1.service: Consumed 114ms CPU time over 4.392s wall clock time, 41.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds) lb # [ 77.979954] envoy[929]: conn peer=[2001:db8:1::1]:41294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=957 ms=6369 lb # [ 77.983801] envoy[929]: conn peer=[2001:db8:1::1]:41308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=772 ms=4220 client: waiting for success: ! systemctl is-active share2 worker2 # [ 78.181606] nix-daemon[870]: reaped child process 1097, status = succeeded worker2 # [ 78.256556] nix-daemon[870]: accepted connection from pid 1133, user nix-grpc-daemon (trusted) worker2 # [ 78.269206] nix-daemon[870]: reaped child process 1136, status = succeeded lb # [ 78.288467] niks3-server[969]: 2026/09/23 05:34:10 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 78.287685] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 78.289422] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.639Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)" lb # [ 78.345741] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 78.350528] niks3-server[969]: 2026/09/23 05:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 78.355553] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls worker2 # [ 78.351515] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.701Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.361980] niks3-server[969]: 2026/09/23 05:34:10 INFO Signed narinfos id=20 count=1 lb # [ 78.368437] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv lb # [ 78.374828] niks3-server[969]: 2026/09/23 05:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 78.379277] niks3-server[969]: 2026/09/23 05:34:10 INFO Completed upload id=20 worker2 # [ 78.370104] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.720Z level=INFO msg="Upload complete. (197ms)" lb # [ 78.381057] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo worker2 # [ 78.373627] nix-grpc-daemon[783]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker2 # [ 78.377176] nix-grpc-daemon[783]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=1 worker2 # [ 78.382713] nix-daemon[870]: reaped child process 1095, status = succeeded client # [ 78.457858] systemd[1]: share2.service: Deactivated successfully. client # [ 78.461880] systemd[1]: share2.service: Consumed 120ms CPU time over 4.842s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds) (finished: subtest: the first client leaving does not fail the second, in 6.76 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 79.359828] systemd[1]: Started Nix Daemon. client # [ 79.449487] nix-daemon[1104]: accepted connection from pid 1101, user root (trusted) worker1 # [ 79.565278] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 79.570654] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 79.568500] nix-grpc-daemon[783]: ts=2026-09-23T05:34:11Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=0 worker2 # [ 79.570665] nix-grpc-daemon[783]: ts=2026-09-23T05:34:11Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 79.583943] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 79.595683] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 79.606741] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 79.617925] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 79.718186] nix-daemon[847]: accepted connection from pid 1111, user nix-grpc-daemon (trusted) worker1 # [ 79.730921] nix-daemon[847]: reaped child process 1114, status = succeeded lb # [ 79.738678] niks3-server[969]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 79.748067] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 79.749984] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.956Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)" lb # [ 79.768842] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst lb # [ 79.778058] niks3-server[969]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 79.783803] niks3-server[969]: 2026/09/23 05:34:12 INFO Signed narinfos id=21 count=1 worker1 # [ 79.787470] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.994Z level=INFO msg="Uploading 1 narinfos" lb # [ 79.794136] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls lb # [ 79.803751] niks3-server[969]: 2026/09/23 05:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 79.810532] niks3-server[969]: 2026/09/23 05:34:12 INFO Completed upload id=21 worker1 # [ 79.810833] nix-grpc-daemon[860]: time=2026-09-23T05:34:12.018Z level=INFO msg="Upload complete. (181ms)" lb # [ 79.813457] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo worker1 # [ 79.815188] nix-grpc-daemon[780]: ts=2026-09-23T05:34:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 79.821193] nix-grpc-daemon[780]: ts=2026-09-23T05:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 79.829706] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 79.837039] nix-daemon[847]: reaped child process 1109, status = succeeded worker1 # [ 79.842732] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 81.080554] nix-daemon[847]: reaped child process 1118, status = succeeded worker1 # [ 81.164955] nix-daemon[847]: accepted connection from pid 1134, user nix-grpc-daemon (trusted) worker1 # [ 81.177598] nix-daemon[847]: reaped child process 1137, status = succeeded lb # [ 81.183467] niks3-server[969]: 2026/09/23 05:34:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 81.193398] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.400Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 81.195614] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.402Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)" lb # [ 81.230234] niks3-server[969]: 2026/09/23 05:34:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker1 # [ 81.239488] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.446Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.240654] niks3-server[969]: 2026/09/23 05:34:13 INFO Signed narinfos id=22 count=1 lb # [ 81.248690] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv lb # [ 81.251761] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 81.256797] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls lb # [ 81.265054] niks3-server[969]: 2026/09/23 05:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 81.268474] niks3-server[969]: 2026/09/23 05:34:13 INFO Completed upload id=22 worker1 # [ 81.269229] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.476Z level=INFO msg="Upload complete. (219ms)" lb # [ 81.271393] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo worker1 # [ 81.275864] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=1 worker1 # [ 81.289814] nix-daemon[847]: reaped child process 1116, status = succeeded worker1 # [ 81.322809] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 81.339517] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 81.351771] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'... worker2 # [ 81.366482] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 81.571990] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 81.578903] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 81.589377] nix-grpc-daemon[922]: time=2026-09-23T05:34:13.939Z level=INFO msg="All 1 paths already cached" worker2 # [ 81.603329] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=0 worker2 # [ 81.607842] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 81.621361] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 81.633229] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 81.645946] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.853Z level=INFO msg="All 1 paths already cached" worker1 # [ 81.653376] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 81.661694] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 81.768448] nix-daemon[847]: accepted connection from pid 1141, user nix-grpc-daemon (trusted) worker1 # [ 81.782938] nix-daemon[847]: reaped child process 1144, status = succeeded lb # [ 81.788394] niks3-server[969]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 81.796756] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 81.798299] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.005Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)" lb # [ 81.822501] niks3-server[969]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker1 # [ 81.829837] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.037Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.830088] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls lb # [ 81.833677] niks3-server[969]: 2026/09/23 05:34:14 INFO Signed narinfos id=23 count=1 lb # [ 81.840245] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst lb # [ 81.852370] niks3-server[969]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 81.859415] niks3-server[969]: 2026/09/23 05:34:14 INFO Completed upload id=23 worker1 # [ 81.860105] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.067Z level=INFO msg="Upload complete. (184ms)" lb # [ 81.865042] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo worker1 # [ 81.864435] nix-grpc-daemon[780]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 81.871078] nix-grpc-daemon[780]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 81.881177] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 81.884970] nix-daemon[847]: reaped child process 1139, status = succeeded worker1 # [ 81.889946] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 82.915817] nix-daemon[847]: reaped child process 1148, status = succeeded worker1 # [ 82.992381] nix-daemon[847]: accepted connection from pid 1164, user nix-grpc-daemon (trusted) worker1 # [ 83.006048] nix-daemon[847]: reaped child process 1167, status = succeeded lb # [ 83.011693] niks3-server[969]: 2026/09/23 05:34:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 83.021148] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.228Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 83.024930] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.229Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)" lb # [ 83.052170] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst lb # [ 83.060260] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv lb # [ 83.066788] niks3-server[969]: 2026/09/23 05:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 83.073998] niks3-server[969]: 2026/09/23 05:34:15 INFO Signed narinfos id=24 count=1 worker1 # [ 83.074886] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.281Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.080068] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls lb # [ 83.090806] niks3-server[969]: 2026/09/23 05:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 83.094634] niks3-server[969]: 2026/09/23 05:34:15 INFO Completed upload id=24 lb # [ 83.096217] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo worker1 # [ 83.095295] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.302Z level=INFO msg="Upload complete. (198ms)" worker1 # [ 83.101757] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=1 worker2 # [ 83.095810] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 worker1 # [ 83.115878] nix-daemon[847]: reaped child process 1146, status = succeeded worker2 # [ 83.104387] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'... worker1 # [ 83.143709] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 83.148986] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 83.302096] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 83.310407] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 83.320250] nix-grpc-daemon[922]: time=2026-09-23T05:34:15.670Z level=INFO msg="All 2 paths already cached" worker2 # [ 83.334261] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=0 worker2 # [ 83.338485] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 83.357100] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 83.369861] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 83.382273] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.589Z level=INFO msg="All 2 paths already cached" worker1 # [ 83.390210] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 83.398537] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 83.500727] nix-daemon[847]: accepted connection from pid 1171, user nix-grpc-daemon (trusted) worker1 # [ 83.513531] nix-daemon[847]: reaped child process 1174, status = succeeded lb # [ 83.520756] niks3-server[969]: 2026/09/23 05:34:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 83.530385] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 83.531816] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.739Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)" lb # [ 83.554108] niks3-server[969]: 2026/09/23 05:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 83.561399] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.768Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.562208] niks3-server[969]: 2026/09/23 05:34:15 INFO Signed narinfos id=25 count=1 lb # [ 83.567848] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls lb # [ 83.573635] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst lb # [ 83.587279] niks3-server[969]: 2026/09/23 05:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 83.592372] niks3-server[969]: 2026/09/23 05:34:15 INFO Completed upload id=25 worker1 # [ 83.592599] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.799Z level=INFO msg="Upload complete. (179ms)" worker1 # [ 83.596497] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 83.599462] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo worker1 # [ 83.602388] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 83.612262] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 83.620567] nix-daemon[847]: reaped child process 1169, status = succeeded worker1 # [ 83.625385] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 84.439499] nix-daemon[847]: reaped child process 1178, status = succeeded worker1 # [ 84.515422] nix-daemon[847]: accepted connection from pid 1194, user nix-grpc-daemon (trusted) worker1 # [ 84.528079] nix-daemon[847]: reaped child process 1197, status = succeeded lb # [ 84.537149] niks3-server[969]: 2026/09/23 05:34:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 84.545592] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.752Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 84.547522] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.754Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)" lb # [ 84.579936] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst lb # [ 84.586627] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls lb # [ 84.595472] niks3-server[969]: 2026/09/23 05:34:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker1 # [ 84.601213] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.808Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.600955] niks3-server[969]: 2026/09/23 05:34:16 INFO Signed narinfos id=26 count=1 lb # [ 84.604552] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv lb # [ 84.614763] niks3-server[969]: 2026/09/23 05:34:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 84.618169] niks3-server[969]: 2026/09/23 05:34:16 INFO Completed upload id=26 worker1 # [ 84.618860] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.826Z level=INFO msg="Upload complete. (187ms)" lb # [ 84.620899] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo worker1 # [ 84.624870] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=1 worker1 # [ 84.638263] nix-daemon[847]: reaped child process 1176, status = succeeded worker2 # [ 84.660720] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 84.671877] nix-grpc-daemon[783]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'... worker1 # [ 84.711262] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 84.716535] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=232 client # [ 84.785416] nix-daemon[1104]: reaped child process 1106, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.53 seconds) client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.58 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds) client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds) worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 85.449275] nix-daemon[1104]: accepted connection from pid 1187, user root (trusted) client # this derivation will be built: client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker2 # [ 85.557076] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 85.562458] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying 1 paths... client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'... worker2 # [ 85.573785] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 85.585656] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 85.691568] nix-daemon[870]: accepted connection from pid 1152, user nix-grpc-daemon (trusted) worker2 # [ 85.705173] nix-daemon[870]: reaped child process 1155, status = succeeded lb # [ 85.724066] niks3-server[969]: 2026/09/23 05:34:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 85.723811] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.073Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 85.725305] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.073Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)" worker2 # [ 85.743549] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.093Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.754375] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 85.757976] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=27 count=1 lb # [ 85.766342] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls lb # [ 85.778161] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 85.784105] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 85.791253] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=27 worker2 # [ 85.780801] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.131Z level=INFO msg="Upload complete. (184ms)" lb # [ 85.798581] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo worker2 # [ 85.784964] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=120 worker2 # [ 85.798609] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=0 worker2 # [ 85.800638] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 85.802740] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 85.806244] nix-daemon[870]: reaped child process 1150, status = succeeded worker2 # [ 85.809871] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying 2 paths to 'lb:50051'... worker2 # [ 85.819090] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 85.825678] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 85.911111] nix-daemon[870]: accepted connection from pid 1161, user nix-grpc-daemon (trusted) worker2 # [ 85.924716] nix-daemon[870]: reaped child process 1164, status = succeeded lb # [ 85.942162] niks3-server[969]: 2026/09/23 05:34:18 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 85.950851] niks3-server[969]: 2026/09/23 05:34:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 85.949300] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.299Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 85.950950] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.301Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)" worker2 # [ 85.954234] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.303Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)" lb # [ 85.997408] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst lb # [ 86.000574] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst lb # [ 86.003064] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls lb # [ 86.006256] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 86.008321] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=28 count=1 lb # [ 86.010548] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign worker2 # [ 86.000711] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.350Z level=INFO msg="Uploading 3 narinfos" lb # [ 86.012663] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=29 count=2 lb # [ 86.015238] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls lb # [ 86.027335] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 86.030988] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=28 lb # [ 86.032293] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 86.035082] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo lb # [ 86.037421] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo lb # [ 86.040541] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo lb # [ 86.042388] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=29 worker2 # [ 86.032093] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.382Z level=INFO msg="Upload complete. (199ms)" worker2 # [ 86.037277] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=2 nar_bytes_in=976 worker2 # [ 86.043549] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 86.054284] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 86.064572] nix-daemon[870]: reaped child process 1159, status = succeeded worker2 # [ 86.069672] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 86.908353] nix-daemon[870]: reaped child process 1168, status = succeeded worker2 # [ 86.989870] nix-daemon[870]: accepted connection from pid 1184, user nix-grpc-daemon (trusted) worker2 # [ 87.001885] nix-daemon[870]: reaped child process 1187, status = succeeded lb # [ 87.021791] niks3-server[969]: 2026/09/23 05:34:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 87.019481] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.369Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 87.051611] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls lb # [ 87.058638] niks3-server[969]: 2026/09/23 05:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 87.054206] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.404Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.065348] niks3-server[969]: 2026/09/23 05:34:19 INFO Signed narinfos id=30 count=1 lb # [ 87.073866] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv lb # [ 87.084108] niks3-server[969]: 2026/09/23 05:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 87.091190] niks3-server[969]: 2026/09/23 05:34:19 INFO Completed upload id=30 worker2 # [ 87.080725] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.431Z level=INFO msg="Upload complete. (204ms)" lb # [ 87.094883] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo worker2 # [ 87.086213] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 87.100438] nix-daemon[870]: reaped child process 1166, status = succeeded client # copying 1 paths... worker1 # [ 87.147327] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 87.157784] nix-grpc-daemon[780]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 87.177839] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 87.183296] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=120 client # [ 87.262958] nix-daemon[1104]: reaped child process 1190, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.93 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 87.365923] nix-daemon[1104]: accepted connection from pid 1209, user root (trusted) client # this derivation will be built: client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker2 # [ 87.472954] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:48046 client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker2 # [ 87.478807] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 87.484242] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.833Z level=INFO msg="All 1 paths already cached" worker2 # [ 87.493668] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=0 worker1 # [ 87.507778] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker2 # [ 87.498540] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 87.520141] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 87.537529] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 87.548698] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 87.564780] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5) worker1 # [ 87.619432] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 132 ms (attempt 2/5) worker1 # [ 87.759434] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 3/5) worker1 # [ 87.798922] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 557 ms (attempt 4/5) lb # [ 87.980106] envoy[929]: conn peer=[2001:db8:1::1]:50254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=4671 lb # [ 87.983858] envoy[929]: conn peer=[2001:db8:1::1]:50258 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3408 tx=1921 ms=1864 lb # [ 87.987388] envoy[929]: conn peer=[2001:db8:1::1]:50266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=87 lb # [ 87.990858] envoy[929]: conn peer=[2001:db8:1::1]:40984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=16 lb # [ 87.994464] envoy[929]: conn peer=[2001:db8:1::1]:50276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2140 ms=1576 lb # [ 87.998540] envoy[929]: conn peer=[2001:db8:1::1]:40998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4297 tx=2371 ms=1414 lb # [ 88.002331] envoy[929]: conn peer=[2001:db8:1::1]:41010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=18 lb # [ 88.005785] envoy[929]: conn peer=[2001:db8:1::1]:41014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4218 tx=2055 ms=1637 lb # [ 88.009298] envoy[929]: conn peer=[2001:db8:1::1]:41022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=17 worker1 # [ 88.359800] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 88.363364] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 88.368295] nix-grpc-daemon[780]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 88.484764] nix-daemon[847]: accepted connection from pid 1212, user nix-grpc-daemon (trusted) worker1 # [ 88.499486] nix-daemon[847]: reaped child process 1215, status = succeeded lb # [ 88.507574] niks3-server[969]: 2026/09/23 05:34:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 88.516619] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.723Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 88.518244] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.725Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)" lb # [ 88.540643] niks3-server[969]: 2026/09/23 05:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 88.548720] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls lb # [ 88.554429] niks3-server[969]: 2026/09/23 05:34:20 INFO Signed narinfos id=31 count=1 worker1 # [ 88.557972] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.765Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.562365] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst lb # [ 88.572699] niks3-server[969]: 2026/09/23 05:34:20 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 88.577484] niks3-server[969]: 2026/09/23 05:34:20 INFO Completed upload id=31 worker1 # [ 88.578119] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.785Z level=INFO msg="Upload complete. (189ms)" worker1 # [ 88.581724] nix-grpc-daemon[780]: ts=2026-09-23T05:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 paths=1 nar_bytes_in=560 lb # [ 88.584236] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo worker1 # [ 88.587443] nix-grpc-daemon[780]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 88.597967] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) worker1 # [ 88.607106] nix-daemon[847]: reaped child process 1205, status = succeeded worker1 # [ 88.609880] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted) client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051... client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 89.852507] nix-daemon[847]: reaped child process 1219, status = succeeded worker1 # [ 89.943678] nix-daemon[847]: accepted connection from pid 1235, user nix-grpc-daemon (trusted) worker1 # [ 89.960060] nix-daemon[847]: reaped child process 1238, status = succeeded lb # [ 89.966662] niks3-server[969]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 89.975421] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.182Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 89.997116] niks3-server[969]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 90.003261] niks3-server[969]: 2026/09/23 05:34:22 INFO Signed narinfos id=32 count=1 worker1 # [ 90.006812] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.213Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.011909] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls lb # [ 90.023465] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv lb # [ 90.036435] niks3-server[969]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 90.039827] niks3-server[969]: 2026/09/23 05:34:22 INFO Completed upload id=32 worker1 # [ 90.040340] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.247Z level=INFO msg="Upload complete. (221ms)" lb # [ 90.043312] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo worker1 # [ 90.043817] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker1 # [ 90.057299] nix-daemon[847]: reaped child process 1217, status = succeeded client # copying 1 paths... worker2 # [ 90.079727] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.091814] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 90.122722] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.127757] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=120 client # [ 90.247464] nix-daemon[1104]: reaped child process 1212, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.99 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 90.353479] nix-daemon[1104]: accepted connection from pid 1231, user root (trusted) client # this derivation will be built: client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker1 # [ 90.472226] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:56936 client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'... worker1 # [ 90.478949] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:56936 worker1 # [ 90.484149] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.690Z level=INFO msg="All 1 paths already cached" worker2 # [ 90.483741] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=0 worker2 # [ 90.489523] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 90.495295] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.506829] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.519905] nix-grpc-daemon[922]: time=2026-09-23T05:34:22.869Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 90.527846] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.538449] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 90.635703] nix-daemon[870]: accepted connection from pid 1196, user nix-grpc-daemon (trusted) worker2 # [ 90.648528] nix-daemon[870]: reaped child process 1199, status = succeeded lb # [ 90.665974] niks3-server[969]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 90.665914] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.016Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 90.667901] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.017Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)" lb # [ 90.698300] niks3-server[969]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 90.705464] niks3-server[969]: 2026/09/23 05:34:22 INFO Signed narinfos id=33 count=1 worker2 # [ 90.697560] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.047Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.710941] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst lb # [ 90.714670] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls lb # [ 90.725064] niks3-server[969]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 90.728219] niks3-server[969]: 2026/09/23 05:34:22 INFO Completed upload id=33 worker2 # [ 90.717899] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.068Z level=INFO msg="Upload complete. (168ms)" lb # [ 90.730323] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo worker2 # [ 90.721688] nix-grpc-daemon[783]: ts=2026-09-23T05:34:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=560 worker2 # [ 90.726763] nix-grpc-daemon[783]: ts=2026-09-23T05:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 90.734277] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) worker2 # [ 90.740345] nix-daemon[870]: reaped child process 1194, status = succeeded worker2 # [ 90.745069] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted) client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 91.775635] nix-daemon[870]: reaped child process 1203, status = succeeded worker2 # [ 91.858955] nix-daemon[870]: accepted connection from pid 1219, user nix-grpc-daemon (trusted) worker2 # [ 91.871788] nix-daemon[870]: reaped child process 1222, status = succeeded lb # [ 91.889741] niks3-server[969]: 2026/09/23 05:34:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 91.888932] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.239Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 91.921352] niks3-server[969]: 2026/09/23 05:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 91.930547] niks3-server[969]: 2026/09/23 05:34:24 INFO Signed narinfos id=34 count=1 worker2 # [ 91.922885] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.273Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.940731] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv lb # [ 91.947535] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls lb # [ 91.956471] niks3-server[969]: 2026/09/23 05:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 91.964024] niks3-server[969]: 2026/09/23 05:34:24 INFO Completed upload id=34 worker2 # [ 91.953665] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.304Z level=INFO msg="Upload complete. (199ms)" lb # [ 91.966745] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo worker2 # [ 91.959601] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=1 client # copying 1 paths... worker2 # [ 91.969994] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:48046 client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 91.978657] nix-daemon[870]: reaped child process 1201, status = succeeded worker2 # [ 91.988815] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 worker2 # [ 91.992737] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=120 client # [ 92.112607] nix-daemon[1104]: reaped child process 1234, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.86 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 7.27 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds) client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds) client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds) worker2 # [ 92.867109] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 93.229068] nix-grpc-daemon[783]: ts=2026-09-23T05:34:25Z level=info event=signal_exit lb # [ 93.739882] niks3-server[969]: 2026/09/23 05:34:25 INFO lead: released remote=192.168.1.4:32998 lb # [ 94.731466] niks3-server[969]: 2026/09/23 05:34:26 INFO lead: acquired remote=192.168.1.3:53886 worker1 # [ 94.736521] nix-grpc-daemon[780]: ts=2026-09-23T05:34:26Z level=info event=scheduler_take_over worker2 # [ 94.730159] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=scheduler_restarting lb # [ 94.739692] envoy[929]: [2026-09-23 05:34:27.000][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 94.737307] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 94.753082] nix-grpc-daemon[780]: ts=2026-09-23T05:34:26Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 94.741565] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48030 worker2 # [ 94.747759] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 94.751184] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48030 worker2 # [ 94.754971] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 94.776806] nix-daemon[870]: reaped child process 872, status = succeeded worker1 # [ 94.954635] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 94.958696] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 94.952186] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 95.160286] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 95.164509] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 95.365197] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 95.376874] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53066 worker1 # [ 95.381451] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 95.381310] nix-grpc-daemon[783]: E0000 00:00:1790141667.731645 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker2 # [ 95.595368] nix-grpc-daemon[783]: E0000 00:00:1790141667.945693 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 95.633849] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 95.636495] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 95.639530] systemd[1]: nix-grpc-daemon.service: Consumed 2.068s CPU time over 1min 17.575s wall clock time, 64.9M memory peak, 81.4K incoming IP traffic, 173.1K outgoing IP traffic. worker2 # [ 95.652396] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 95.655737] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 95.658482] systemd[1]: nix-grpc-daemon.socket: Consumed 294us CPU time over 1min 28.194s wall clock time, 256K memory peak, 343K incoming IP traffic, 97.1K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2 worker1 # [ 95.872486] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:56936 client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker1 # [ 95.877901] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:56936 client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds) worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds) worker2 # [ 95.944676] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 96.005259] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 96.158722] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:53072 worker1 # [ 96.178181] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=info event=signal_exit worker2 # [ 96.628819] systemd[1]: Starting scheduler-token.service... worker2 # [ 96.689106] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 96.691505] systemd[1]: Finished scheduler-token.service. worker2 # [ 96.694566] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 59ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 96.702398] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 96.742126] niks3-server[969]: 2026/09/23 05:34:29 INFO lead: released remote=192.168.1.3:53886 worker2 # [ 96.782809] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 96.784749] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 96.787888] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 96.792051] nix-grpc-daemon[1246]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 96.794157] nix-grpc-daemon[1246]: E0000 00:00:1790141669.142501 1246 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 96.799168] nix-grpc-daemon[1246]: E0000 00:00:1790141669.147884 1246 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 96.803115] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 96.804730] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 96.810069] nix-grpc-daemon[1246]: E0000 00:00:1790141669.159632 1289 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 96.820751] niks3-server[969]: 2026/09/23 05:34:29 INFO lead: acquired remote=192.168.1.4:50460 worker2 # [ 96.814159] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_take_over worker2 # [ 96.824044] nix-grpc-daemon[1246]: E0000 00:00:1790141669.174530 1289 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 96.828052] nix-grpc-daemon[1246]: E0000 00:00:1790141669.176841 1289 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 96.838137] nix-grpc-daemon[1246]: E0000 00:00:1790141669.188605 1289 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 96.847111] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 96.848654] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.050317] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.054594] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.255933] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.260284] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.461256] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.465554] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.666912] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.671225] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 97.743601] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=scheduler_restarting lb # [ 97.741624] envoy[929]: [2026-09-23 05:34:30.002][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 97.749169] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 97.756099] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53066 worker1 # [ 97.760132] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 97.784185] nix-daemon[847]: reaped child process 849, status = succeeded worker2 # [ 97.873148] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.886149] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 97.889946] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:32998 worker2 # [ 97.894584] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 97.940927] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:33004 worker1 # [ 97.965703] nix-grpc-daemon[780]: ts=2026-09-23T05:34:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" lb # [ 97.980444] envoy[929]: conn peer=[2001:db8:1::1]:41030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2124 ms=2707 lb # [ 97.984228] envoy[929]: conn peer=[2001:db8:1::1]:41042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=59 lb # [ 97.987664] envoy[929]: conn peer=[2001:db8:1::1]:41058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=55 lb # [ 97.991241] envoy[929]: conn peer=[2001:db8:1::1]:41052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3727 tx=2084 ms=1584 lb # [ 97.994741] envoy[929]: conn peer=[2001:db8:1::4]:41766 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2633 tx=869 ms=61202 lb # [ 97.998468] envoy[929]: conn peer=[2001:db8:1::1]:45018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=17 lb # [ 98.002440] envoy[929]: conn peer=192.168.1.3:41782 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3191 tx=1318 ms=64583 worker1 # [ 98.354356] nix-grpc-daemon[780]: E0000 00:00:1790141670.561395 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 99.600781] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 99.604268] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 99.607227] systemd[1]: nix-grpc-daemon.service: Consumed 2.466s CPU time over 1min 21.619s wall clock time, 66M memory peak, 87.4K incoming IP traffic, 156.9K outgoing IP traffic. worker1 # [ 99.620767] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 99.624952] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 99.628909] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 30.679s wall clock time, 256K memory peak, 217.7K incoming IP traffic, 89.3K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2 worker2 # [ 99.826363] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 99.837375] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 99.854367] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5) worker2 # [ 99.868492] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 2/5) worker2 # [ 99.987606] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 325 ms (attempt 3/5) worker2 # [ 100.314731] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 4/5) worker2 # [ 100.447264] nix-grpc-daemon[1246]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 100.450946] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 100.472840] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 100.485139] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 100.505255] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=info event=niks3_push_started pid=1307 worker2 # [ 100.608981] nix-grpc-daemon[1307]: time=2026-09-23T05:34:32.959Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 100.698099] nix-daemon[870]: accepted connection from pid 1320, user nix-grpc-daemon (trusted) worker2 # [ 100.710282] nix-daemon[870]: reaped child process 1323, status = succeeded lb # [ 100.730313] niks3-server[969]: 2026/09/23 05:34:32 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 100.740638] niks3-server[969]: 2026/09/23 05:34:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.737480] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.087Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 100.739094] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.089Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)" worker2 # [ 100.740940] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.091Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)" lb # [ 100.799115] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 100.801317] niks3-server[969]: 2026/09/23 05:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 100.806926] niks3-server[969]: 2026/09/23 05:34:33 INFO Signed narinfos id=35 count=1 lb # [ 100.809218] niks3-server[969]: 2026/09/23 05:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 100.812648] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls lb # [ 100.815040] niks3-server[969]: 2026/09/23 05:34:33 INFO Signed narinfos id=36 count=2 worker2 # [ 100.805135] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.154Z level=INFO msg="Uploading 3 narinfos" lb # [ 100.824752] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst lb # [ 100.833681] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls lb # [ 100.841708] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo lb # [ 100.845216] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo lb # [ 100.848291] niks3-server[969]: 2026/09/23 05:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 100.854108] niks3-server[969]: 2026/09/23 05:34:33 INFO Completed upload id=35 lb # [ 100.855556] niks3-server[969]: 2026/09/23 05:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 100.858744] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo lb # [ 100.861307] niks3-server[969]: 2026/09/23 05:34:33 INFO Completed upload id=36 worker2 # [ 100.851120] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.201Z level=INFO msg="Upload complete. (238ms)" worker2 # [ 100.856777] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 100.871297] nix-daemon[870]: reaped child process 1305, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.21 seconds) worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds) worker1 # [ 100.970218] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 100.979604] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 101.078440] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 101.080672] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 101.084571] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 101.089056] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 101.090396] nix-grpc-daemon[1261]: E0000 00:00:1790141673.295850 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 101.097070] nix-grpc-daemon[1261]: E0000 00:00:1790141673.301076 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 101.100725] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 101.103511] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 101.106572] nix-grpc-daemon[1261]: E0000 00:00:1790141673.313924 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 101.120159] nix-grpc-daemon[1261]: E0000 00:00:1790141673.327500 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 101.124061] nix-grpc-daemon[1261]: E0000 00:00:1790141673.329848 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 101.134548] nix-grpc-daemon[1261]: E0000 00:00:1790141673.341842 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 101.146775] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.137918] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:32998 worker2 # [ 101.141717] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.05 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb # [ 101.238347] systemd[1]: Stopping niks3 server... lb # [ 101.240180] niks3-server[969]: 2026/09/23 05:34:33 INFO Shutdown signal received, draining in-flight requests timeout=10s worker1 # [ 101.239431] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=scheduler_lock_lost status=200 lb # [ 101.243873] niks3-server[969]: 2026/09/23 05:34:33 INFO lead: released remote=192.168.1.4:50460 worker2 # [ 101.233325] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=scheduler_lock_lost status=200 lb # [ 106.059155] systemd[1]: niks3.service: Deactivated successfully. lb # [ 106.064573] systemd[1]: Stopped niks3 server. lb # [ 106.066350] systemd[1]: niks3.service: Consumed 762ms CPU time over 1min 26.540s wall clock time, 28.6M memory peak, 94.9K incoming IP traffic, 137.9K outgoing IP traffic. lb # [ 106.083552] systemd[1]: Starting niks3 server... lb # [ 106.143713] niks3-server[1135]: 2026/09/23 05:34:38 goose: no migrations to run. current version: 20260920000000 lb # [ 106.152299] niks3-server[1135]: 2026/09/23 05:34:38 OK 1_commit_pending_closure.sql (6.66ms) lb # [ 106.156183] niks3-server[1135]: 2026/09/23 05:34:38 OK 2_object_stats_trigger.sql (2.22ms) lb # [ 106.157256] niks3-server[1135]: 2026/09/23 05:34:38 goose: up to current file version: 2 lb # [ 106.158522] niks3-server[1135]: 2026/09/23 05:34:38 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key lb # [ 106.169173] niks3-server[1135]: 2026/09/23 05:34:38 INFO Read proxy enabled — serving cache objects from S3 lb # [ 106.170555] niks3-server[1135]: 2026/09/23 05:34:38 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 106.172699] systemd[1]: Started niks3 server. lb # [ 106.174364] niks3-server[1135]: 2026/09/23 05:34:38 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 106.179723] niks3-server[1135]: 2026/09/23 05:34:38 INFO systemd watchdog enabled interval=15s lb # [ 106.182057] niks3-server[1135]: 2026/09/23 05:34:38 INFO lead: acquired remote=192.168.1.4:50544 lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 106.490245] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33034 client # don't know how to build these paths: client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker1 # [ 106.514897] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 106.527698] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 106.544882] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5) worker1 # [ 106.585330] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5) worker1 # [ 106.767760] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 351 ms (attempt 3/5) worker1 # [ 107.121186] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 520 ms (attempt 4/5) worker1 # [ 107.645390] nix-grpc-daemon[1261]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 107.648562] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 107.675969] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=0 worker2 # [ 107.682156] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 107.722219] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 107.736497] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33034 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'... worker2 # [ 107.749414] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 107.760346] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 107.864078] nix-daemon[870]: accepted connection from pid 1342, user nix-grpc-daemon (trusted) worker2 # [ 107.876873] nix-daemon[870]: reaped child process 1345, status = succeeded lb # [ 107.895692] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 107.897322] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 107.899742] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.249Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)" lb # [ 107.939822] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 107.952233] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls lb # [ 107.967845] niks3-server[1135]: 2026/09/23 05:34:40 INFO Signed narinfos id=37 count=1 worker2 # [ 107.959529] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.309Z level=INFO msg="Uploading 1 narinfos" lb # [ 107.976362] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst lb # [ 107.979514] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 107.982226] envoy[929]: conn peer=[2001:db8:1::1]:45022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=1047 lb # [ 107.988196] niks3-server[1135]: 2026/09/23 05:34:40 INFO Completed upload id=37 worker2 # [ 107.977832] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.328Z level=INFO msg="Upload complete. (205ms)" lb # [ 107.993060] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo worker2 # [ 107.984117] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=536 worker2 # [ 107.990449] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 108.001702] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 108.010151] nix-daemon[870]: reaped child process 1340, status = succeeded client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051... worker2 # [ 108.018560] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 109.451815] nix-daemon[870]: reaped child process 1349, status = succeeded worker2 # [ 109.537258] nix-daemon[870]: accepted connection from pid 1365, user nix-grpc-daemon (trusted) worker2 # [ 109.550046] nix-daemon[870]: reaped child process 1368, status = succeeded lb # [ 109.568494] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 109.569436] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 109.571241] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.921Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)" lb # [ 109.614752] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls lb # [ 109.619487] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 109.625674] niks3-server[1135]: 2026/09/23 05:34:41 INFO Signed narinfos id=38 count=1 worker2 # [ 109.617712] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.968Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.635743] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv lb # [ 109.640668] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 109.645786] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 109.650906] niks3-server[1135]: 2026/09/23 05:34:41 INFO Completed upload id=38 worker2 # [ 109.640616] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.991Z level=INFO msg="Upload complete. (214ms)" lb # [ 109.652683] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo worker2 # [ 109.648305] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 worker2 # [ 109.660683] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=0 worker2 # [ 109.665186] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 109.673622] nix-daemon[870]: reaped child process 1347, status = succeeded worker2 # [ 109.704231] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 109.716417] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 109.730157] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.080Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'... worker2 # [ 109.738923] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 109.749164] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 109.843983] nix-daemon[870]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) worker2 # [ 109.857115] nix-daemon[870]: reaped child process 1377, status = succeeded lb # [ 109.875454] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 109.875145] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.225Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 109.876569] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.227Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)" lb # [ 109.910615] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls lb # [ 109.915960] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 109.913158] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.263Z level=INFO msg="Uploading 1 narinfos" lb # [ 109.923794] niks3-server[1135]: 2026/09/23 05:34:42 INFO Signed narinfos id=39 count=1 lb # [ 109.930666] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst lb # [ 109.942401] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 109.950510] niks3-server[1135]: 2026/09/23 05:34:42 INFO Completed upload id=39 lb # [ 109.953567] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo worker2 # [ 109.940233] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.290Z level=INFO msg="Upload complete. (180ms)" worker2 # [ 109.945244] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 109.951309] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 109.961769] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 109.970437] nix-daemon[870]: reaped child process 1372, status = succeeded worker2 # [ 109.976121] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 111.412644] nix-daemon[870]: reaped child process 1381, status = succeeded worker2 # [ 111.494839] nix-daemon[870]: accepted connection from pid 1397, user nix-grpc-daemon (trusted) worker2 # [ 111.506309] nix-daemon[870]: reaped child process 1400, status = succeeded lb # [ 111.524896] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.523254] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.873Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 111.524903] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.875Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)" lb # [ 111.579299] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 111.583940] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst lb # [ 111.591655] niks3-server[1135]: 2026/09/23 05:34:43 INFO Signed narinfos id=40 count=1 worker2 # [ 111.585454] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.935Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.597286] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv lb # [ 111.600859] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls lb # [ 111.609121] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 111.613524] niks3-server[1135]: 2026/09/23 05:34:43 INFO Completed upload id=40 worker2 # [ 111.602717] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.953Z level=INFO msg="Upload complete. (219ms)" lb # [ 111.615328] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo worker2 # [ 111.608777] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 worker2 # [ 111.622568] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=0 worker2 # [ 111.627096] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 111.634943] nix-daemon[870]: reaped child process 1379, status = succeeded worker2 # [ 111.665939] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 111.676689] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 111.688464] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.038Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'... worker2 # [ 111.697359] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 111.705584] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 111.799539] nix-daemon[870]: accepted connection from pid 1406, user nix-grpc-daemon (trusted) worker2 # [ 111.812791] nix-daemon[870]: reaped child process 1409, status = succeeded lb # [ 111.830672] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.828603] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.179Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 111.830120] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.180Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)" lb # [ 111.866286] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker2 # [ 111.864886] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.215Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.875525] niks3-server[1135]: 2026/09/23 05:34:44 INFO Signed narinfos id=41 count=1 lb # [ 111.879935] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst lb # [ 111.886125] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls lb # [ 111.897545] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 111.903564] niks3-server[1135]: 2026/09/23 05:34:44 INFO Completed upload id=41 worker2 # [ 111.892472] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.242Z level=INFO msg="Upload complete. (175ms)" lb # [ 111.906345] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo worker2 # [ 111.896995] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 111.902865] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 worker2 # [ 111.913117] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker2 # [ 111.921150] nix-daemon[870]: reaped child process 1404, status = succeeded worker2 # [ 111.924715] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051... client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 113.161367] nix-daemon[870]: reaped child process 1413, status = succeeded worker2 # [ 113.244233] nix-daemon[870]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker2 # [ 113.256193] nix-daemon[870]: reaped child process 1432, status = succeeded lb # [ 113.274431] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 113.272746] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.623Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 113.274573] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.624Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)" lb # [ 113.316169] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv lb # [ 113.322319] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls lb # [ 113.325904] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 113.329826] niks3-server[1135]: 2026/09/23 05:34:45 INFO Signed narinfos id=42 count=1 worker2 # [ 113.321891] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.672Z level=INFO msg="Uploading 1 narinfos" lb # [ 113.336212] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst lb # [ 113.346944] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 113.350444] niks3-server[1135]: 2026/09/23 05:34:45 INFO Completed upload id=42 worker2 # [ 113.341227] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.691Z level=INFO msg="Upload complete. (207ms)" lb # [ 113.354439] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo worker2 # [ 113.349253] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 worker2 # [ 113.362980] nix-daemon[870]: reaped child process 1411, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 7.03 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 12.34 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2 # [ 113.651235] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 113.805632] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:46Z level=info event=signal_exit lb # [ 114.185165] niks3-server[1135]: 2026/09/23 05:34:46 INFO lead: released remote=192.168.1.4:50544 worker2 # [ 115.175878] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting lb # [ 115.185834] envoy[929]: [2026-09-23 05:34:47.446][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 115.183950] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 115.195295] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 115.189308] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:32998 worker2 # [ 115.193687] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 115.197574] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:32998 worker2 # [ 115.200944] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 115.222745] nix-daemon[870]: reaped child process 1299, status = succeeded worker1 # [ 115.395764] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.400242] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.396885] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 115.601199] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.605438] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 115.806619] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 115.810859] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 115.857496] nix-grpc-daemon[1246]: E0000 00:00:1790141688.207830 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 115.995102] nix-grpc-daemon[1246]: E0000 00:00:1790141688.345293 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 116.011687] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.015879] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.182925] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_take_over lb # [ 116.182637] niks3-server[1135]: 2026/09/23 05:34:48 INFO lead: acquired remote=192.168.1.3:41008 worker1 # [ 116.218254] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.221380] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 116.283253] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 116.285903] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 116.289658] systemd[1]: nix-grpc-daemon.service: Consumed 959ms CPU time over 19.580s wall clock time, 38.3M memory peak, 46.8K incoming IP traffic, 85.2K outgoing IP traffic. worker2 # [ 116.301220] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 116.305523] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 116.309164] systemd[1]: nix-grpc-daemon.socket: Consumed 288us CPU time over 20.363s wall clock time, 256K memory peak, 218.6K incoming IP traffic, 34.9K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker1 # [ 116.418625] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.420471] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.621605] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.625760] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 116.826795] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.834739] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:53172 worker1 # [ 116.838907] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker1 # [ 117.159919] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:53188 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2 worker1 # [ 117.598901] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37216 client # don't know how to build these paths: client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker1 # [ 117.607691] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 117.631953] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=0 worker1 # [ 117.637306] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 117.675899] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 117.689276] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'... worker1 # [ 117.702289] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 117.714347] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 117.734168] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=niks3_push_started pid=1338 worker1 # [ 117.843362] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.050Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 117.938729] nix-daemon[847]: accepted connection from pid 1351, user nix-grpc-daemon (trusted) worker1 # [ 117.953832] nix-daemon[847]: reaped child process 1354, status = succeeded lb # [ 117.962809] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 117.973328] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 117.974873] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.182Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)" lb # [ 117.984472] envoy[929]: conn peer=[2001:db8:1::1]:38738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9735 tx=4424 ms=6873 lb # [ 117.989235] envoy[929]: conn peer=[2001:db8:1::4]:55290 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5923 tx=665 ms=18544 lb # [ 118.015348] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls lb # [ 118.022537] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 118.027547] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.234Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.027184] niks3-server[1135]: 2026/09/23 05:34:50 INFO Signed narinfos id=43 count=1 lb # [ 118.037855] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst lb # [ 118.041807] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 118.048410] niks3-server[1135]: 2026/09/23 05:34:50 INFO Completed upload id=43 worker1 # [ 118.048967] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.256Z level=INFO msg="Upload complete. (202ms)" lb # [ 118.050420] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo worker1 # [ 118.053663] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 118.058753] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 118.070225] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 118.073810] nix-daemon[847]: reaped child process 1336, status = succeeded worker1 # [ 118.078926] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051... client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 119.120651] nix-daemon[847]: reaped child process 1358, status = succeeded worker1 # [ 119.203784] nix-daemon[847]: accepted connection from pid 1374, user nix-grpc-daemon (trusted) worker1 # [ 119.219383] nix-daemon[847]: reaped child process 1377, status = succeeded lb # [ 119.227649] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.236494] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 119.239242] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.445Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)" lb # [ 119.262705] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv lb # [ 119.271689] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 119.280695] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 119.286532] niks3-server[1135]: 2026/09/23 05:34:51 INFO Signed narinfos id=44 count=1 worker1 # [ 119.289481] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.496Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.293469] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls lb # [ 119.302249] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 119.306866] niks3-server[1135]: 2026/09/23 05:34:51 INFO Completed upload id=44 worker1 # [ 119.307373] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.514Z level=INFO msg="Upload complete. (219ms)" lb # [ 119.309560] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo worker1 # [ 119.318100] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 worker1 # [ 119.325803] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=0 worker1 # [ 119.330500] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 119.338502] nix-daemon[847]: reaped child process 1356, status = succeeded worker1 # [ 119.370715] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 119.381913] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 119.396196] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.603Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'... worker1 # [ 119.404888] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 119.415163] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 119.517989] nix-daemon[847]: accepted connection from pid 1383, user nix-grpc-daemon (trusted) worker1 # [ 119.533305] nix-daemon[847]: reaped child process 1386, status = succeeded lb # [ 119.540996] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.550035] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.757Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 119.551580] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.758Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)" lb # [ 119.575931] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 119.583237] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls lb # [ 119.590332] niks3-server[1135]: 2026/09/23 05:34:51 INFO Signed narinfos id=45 count=1 worker1 # [ 119.593386] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.800Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.613540] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 119.617622] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst lb # [ 119.621405] niks3-server[1135]: 2026/09/23 05:34:51 INFO Completed upload id=45 worker1 # [ 119.622199] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.829Z level=INFO msg="Upload complete. (196ms)" lb # [ 119.623904] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo worker1 # [ 119.626932] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 119.633163] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 119.641894] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 119.652253] nix-daemon[847]: reaped child process 1381, status = succeeded worker1 # [ 119.657215] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051... client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 120.485750] nix-daemon[847]: reaped child process 1390, status = succeeded worker1 # [ 120.563025] nix-daemon[847]: accepted connection from pid 1406, user nix-grpc-daemon (trusted) worker1 # [ 120.577889] nix-daemon[847]: reaped child process 1409, status = succeeded lb # [ 120.586558] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.595699] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 120.597211] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.804Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)" lb # [ 120.630630] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv lb # [ 120.638147] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker1 # [ 120.645105] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.852Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.645197] niks3-server[1135]: 2026/09/23 05:34:52 INFO Signed narinfos id=46 count=1 lb # [ 120.650556] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls lb # [ 120.655460] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst lb # [ 120.664428] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 120.667635] niks3-server[1135]: 2026/09/23 05:34:52 INFO Completed upload id=46 worker1 # [ 120.668166] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.875Z level=INFO msg="Upload complete. (204ms)" lb # [ 120.671521] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo worker1 # [ 120.678984] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 worker1 # [ 120.685636] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=0 worker1 # [ 120.690197] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 120.699361] nix-daemon[847]: reaped child process 1388, status = succeeded worker1 # [ 120.729130] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 120.742412] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 120.756108] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.963Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'... worker1 # [ 120.765117] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 120.773732] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 120.877832] nix-daemon[847]: accepted connection from pid 1415, user nix-grpc-daemon (trusted) worker1 # [ 120.892432] nix-daemon[847]: reaped child process 1418, status = succeeded lb # [ 120.901467] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 120.910507] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 120.912034] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.119Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)" lb # [ 120.933209] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls lb # [ 120.939761] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign worker1 # [ 120.945815] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.153Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.945989] niks3-server[1135]: 2026/09/23 05:34:53 INFO Signed narinfos id=47 count=1 lb # [ 120.959528] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst lb # [ 120.966910] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 120.972982] niks3-server[1135]: 2026/09/23 05:34:53 INFO Completed upload id=47 worker1 # [ 120.973680] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.180Z level=INFO msg="Upload complete. (187ms)" lb # [ 120.976331] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo worker1 # [ 120.978165] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 120.983267] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 120.994216] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 121.000163] nix-daemon[847]: reaped child process 1413, status = succeeded worker1 # [ 121.005335] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051... client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 122.043567] nix-daemon[847]: reaped child process 1422, status = succeeded worker1 # [ 122.136741] nix-daemon[847]: accepted connection from pid 1438, user nix-grpc-daemon (trusted) worker1 # [ 122.150796] nix-daemon[847]: reaped child process 1441, status = succeeded lb # [ 122.158902] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 122.169105] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 122.170600] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.377Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)" lb # [ 122.202641] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv lb # [ 122.211151] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls lb # [ 122.218981] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 122.226281] niks3-server[1135]: 2026/09/23 05:34:54 INFO Signed narinfos id=48 count=1 worker1 # [ 122.227281] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.434Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.234844] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst lb # [ 122.243130] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 122.248347] niks3-server[1135]: 2026/09/23 05:34:54 INFO Completed upload id=48 lb # [ 122.250820] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo worker1 # [ 122.250393] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.457Z level=INFO msg="Upload complete. (236ms)" worker1 # [ 122.258570] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 worker1 # [ 122.273935] nix-daemon[847]: reaped child process 1420, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.84 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds) worker2 # [ 122.441686] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 122.449829] systemd[1]: Starting scheduler-token.service... worker2 # [ 122.496729] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 122.498467] systemd[1]: Finished scheduler-token.service. worker2 # [ 122.500198] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 122.503759] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 122.580969] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 122.582921] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 122.586115] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 122.590488] nix-grpc-daemon[1453]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 122.593183] nix-grpc-daemon[1453]: E0000 00:00:1790141694.940907 1453 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 122.599565] nix-grpc-daemon[1453]: E0000 00:00:1790141694.946481 1453 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 122.603467] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 122.608122] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 122.610096] nix-grpc-daemon[1453]: E0000 00:00:1790141694.957067 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 122.620965] nix-grpc-daemon[1453]: E0000 00:00:1790141694.971467 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 122.625152] nix-grpc-daemon[1453]: E0000 00:00:1790141694.973707 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 122.635625] nix-grpc-daemon[1453]: E0000 00:00:1790141694.986137 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 122.649178] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 122.671190] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 122.675846] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:53172 worker1 # [ 122.683098] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 9.34 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs client # [ 122.897730] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 604ee7ef781c4a29bdd8a4f2016574ec client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds) worker1 # [ 123.034274] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37216 client # [ 123.040918] nix[1358]: don't know how to build these paths: client # [ 123.041881] nix[1358]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker1 # [ 123.048752] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=0 worker1 # [ 123.054092] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 123.093368] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 123.107743] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37216 client # [ 123.118155] nix[1358]: copying 1 paths to 'lb:50051'... worker1 # [ 123.119290] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 client # [ 123.120058] nix[1358]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'... worker1 # [ 123.132593] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 123.242351] nix-daemon[847]: accepted connection from pid 1454, user nix-grpc-daemon (trusted) worker1 # [ 123.257135] nix-daemon[847]: reaped child process 1457, status = succeeded lb # [ 123.265279] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 123.276431] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.483Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 123.277877] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.483Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)" lb # [ 123.296615] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker1 # [ 123.302210] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.509Z level=INFO msg="Uploading 1 narinfos" lb # [ 123.303096] niks3-server[1135]: 2026/09/23 05:34:55 INFO Signed narinfos id=49 count=1 lb # [ 123.314660] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls lb # [ 123.324538] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst lb # [ 123.328644] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 123.336367] niks3-server[1135]: 2026/09/23 05:34:55 INFO Completed upload id=49 worker1 # [ 123.337047] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.544Z level=INFO msg="Upload complete. (192ms)" lb # [ 123.339923] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo worker1 # [ 123.341596] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 123.348311] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 worker1 # [ 123.359111] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 123.364707] nix-daemon[847]: reaped child process 1452, status = succeeded worker1 # [ 123.369897] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # [ 123.369422] nix[1358]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 125.568077] systemd-journald[285]: Received client request to rotate journal, rotating. worker2 # [ 125.588401] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/4fa082a50d854427bd7f5936381953f0. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 125.511707] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 worker1 # [ 126.107515] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:58Z level=info event=draining lb # [ 127.984464] envoy[929]: conn peer=[2001:db8:1::1]:45890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4347 ms=4670 client: (finished: must succeed: sleep 5.0, in 5.03 seconds) worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 130.810718] nix-daemon[847]: reaped child process 1461, status = succeeded worker1 # [ 130.896957] nix-daemon[847]: accepted connection from pid 1502, user nix-grpc-daemon (trusted) worker1 # [ 130.911083] nix-daemon[847]: reaped child process 1505, status = succeeded lb # [ 130.921503] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 130.932363] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 130.933868] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.141Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)" lb # [ 130.970226] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 130.974989] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.182Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.975362] niks3-server[1135]: 2026/09/23 05:35:03 INFO Signed narinfos id=50 count=1 lb # [ 130.993664] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls lb # [ 130.996451] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv lb # [ 131.000654] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 131.005077] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 131.007987] niks3-server[1135]: 2026/09/23 05:35:03 INFO Completed upload id=50 worker1 # [ 131.008646] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.215Z level=INFO msg="Upload complete. (223ms)" lb # [ 131.012173] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo worker1 # [ 131.017725] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 131.032241] nix-daemon[847]: reaped child process 1459, status = succeeded client # [ 131.045452] systemd[1]: rs.service: Deactivated successfully. client # [ 131.049140] systemd[1]: rs.service: Consumed 125ms CPU time over 8.150s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic. worker1 # [ 131.109339] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:03Z level=info event=signal_exit lb # [ 131.181434] niks3-server[1135]: 2026/09/23 05:35:03 INFO lead: released remote=192.168.1.3:41008 worker1 # [ 132.185882] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting lb # [ 132.184698] envoy[929]: [2026-09-23 05:35:04.445][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 132.192816] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 132.182746] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 132.199394] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53172 worker1 # [ 132.203261] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 132.206885] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:53172 worker1 # [ 132.210969] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 132.231561] nix-daemon[847]: reaped child process 1316, status = succeeded worker1 # [ 132.235611] nix-grpc-daemon[1261]: E0000 00:00:1790141704.442785 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker2 # [ 132.383404] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 132.387689] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 132.407817] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 132.588880] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 132.593958] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 132.616845] niks3-server[1135]: 2026/09/23 05:35:04 INFO lead: acquired remote=192.168.1.4:33120 worker2 # [ 132.609486] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_take_over worker1 # [ 132.678375] nix-grpc-daemon[1261]: E0000 00:00:1790141704.885360 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 132.779565] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 132.783081] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 132.785620] systemd[1]: nix-grpc-daemon.service: Consumed 1.206s CPU time over 31.801s wall clock time, 38.2M memory peak, 47.5K incoming IP traffic, 98.3K outgoing IP traffic. worker1 # [ 132.801558] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 132.794746] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 132.799459] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.12 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } worker1 # [ 132.899263] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 132.903951] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 132.908238] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 132.912515] nix-grpc-daemon[1512]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 132.916481] nix-grpc-daemon[1512]: E0000 00:00:1790141705.114317 1512 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 132.919557] nix-grpc-daemon[1512]: E0000 00:00:1790141705.118271 1512 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.922493] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 132.924321] systemd[1]: Started Nix worker-protocol over gRPC. client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds) worker1 # [ 132.926421] nix-grpc-daemon[1512]: E0000 00:00:1790141705.132761 1554 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client: must fail: journalctl -u rs | grep -q 'reconnecting' worker1 # [ 132.937572] nix-grpc-daemon[1512]: E0000 00:00:1790141705.144862 1554 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.942056] nix-grpc-daemon[1512]: E0000 00:00:1790141705.147184 1554 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 132.951809] nix-grpc-daemon[1512]: E0000 00:00:1790141705.159069 1554 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 132.964248] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 132.966414] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 133.000123] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.004080] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 133.167966] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 133.172254] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 133.206183] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.210141] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 133.373562] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.368279] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38468 worker2 # [ 133.372220] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 worker2 # [ 133.411136] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 133.421143] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 133.424920] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38468 worker2 # [ 133.428847] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.24 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid client # [ 134.135852] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 7e078066fc7843f0bf57dce8d9225dfd client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds) worker2 # [ 134.271319] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38480 worker2 # [ 134.282142] nix-daemon[870]: accepted connection from pid 1453, user nix-grpc-daemon (trusted) client # [ 134.305653] nix[1394]: don't know how to build these paths: client # [ 134.306642] nix[1394]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker2 # [ 134.304101] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=0 worker2 # [ 134.308757] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 134.374901] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40440 worker1 # [ 134.386515] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) worker1 # [ 134.403801] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5) worker1 # [ 134.493950] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5) worker1 # [ 134.515636] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 271 ms (attempt 3/5) worker1 # [ 134.789838] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 509 ms (attempt 4/5) worker1 # [ 135.306513] nix-grpc-daemon[1512]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 135.309624] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 135.326885] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40440 client # [ 135.337799] nix[1394]: copying 1 paths to 'lb:50051'... client # [ 135.342189] nix[1394]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'... worker1 # [ 135.341815] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40440 worker1 # [ 135.350690] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) worker1 # [ 135.372121] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=info event=niks3_push_started pid=1581 worker1 # [ 135.477848] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.684Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 135.574096] nix-daemon[847]: accepted connection from pid 1594, user nix-grpc-daemon (trusted) worker1 # [ 135.588213] nix-daemon[847]: reaped child process 1597, status = succeeded lb # [ 135.596583] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 135.586982] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:07Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:38486 worker1 # [ 135.613140] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 135.614679] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.822Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)" lb # [ 135.642087] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 135.647537] niks3-server[1135]: 2026/09/23 05:35:07 INFO Signed narinfos id=51 count=1 worker1 # [ 135.651967] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.859Z level=INFO msg="Uploading 1 narinfos" lb # [ 135.652955] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst lb # [ 135.658705] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls lb # [ 135.669634] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 135.672938] niks3-server[1135]: 2026/09/23 05:35:07 INFO Completed upload id=51 worker1 # [ 135.674614] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.881Z level=INFO msg="Upload complete. (193ms)" lb # [ 135.674912] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo worker1 # [ 135.678648] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40440 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 135.684262] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40440 worker1 # [ 135.693741] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) worker1 # [ 135.698195] nix-daemon[847]: reaped child process 1579, status = succeeded worker1 # [ 135.702513] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) client # [ 135.701970] nix[1394]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 136.636491] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1453 (nix-grpc-daemon) on client request. worker2 # [ 136.646633] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 136.664327] niks3-server[1135]: 2026/09/23 05:35:08 INFO lead: released remote=192.168.1.4:33120 worker1 # [ 136.671854] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:08Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" client # [ 136.672482] nix[1394]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting lb # [ 136.671677] envoy[929]: [2026-09-23 05:35:08.928][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} worker2 # [ 136.666670] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 136.668724] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 136.669965] systemd[1]: nix-grpc-daemon.service: Consumed 229ms CPU time over 14.158s wall clock time, 14.1M memory peak, 10.6K incoming IP traffic, 21.3K outgoing IP traffic. worker2 # [ 136.680060] systemd[1]: Starting scheduler-token.service... worker2 # [ 136.684214] nix-daemon[870]: reaped child process 1563, status = succeeded worker2 # [ 136.716441] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 136.718072] systemd[1]: Finished scheduler-token.service. worker2 # [ 136.719836] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 38ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 136.721984] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 136.725412] systemd[1]: Starting scheduler-token.service... worker2 # [ 136.759378] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 136.760967] systemd[1]: Finished scheduler-token.service. worker2 # [ 136.762140] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 35ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 136.767132] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 136.843717] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 136.848049] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 136.851401] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 136.858287] nix-grpc-daemon[1593]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 136.873073] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.861693] nix-grpc-daemon[1593]: E0000 00:00:1790141709.201359 1593 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 136.865102] nix-grpc-daemon[1593]: E0000 00:00:1790141709.205335 1593 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 136.870088] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 136.872350] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 136.882381] niks3-server[1135]: 2026/09/23 05:35:09 INFO lead: acquired remote=192.168.1.4:53770 worker2 # [ 136.874161] nix-grpc-daemon[1593]: E0000 00:00:1790141709.224679 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 136.878122] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=scheduler_take_over worker2 # [ 136.882662] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:38512 worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds) client: must succeed: sleep 5.0 worker2 # [ 136.888630] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38528 worker2 # [ 136.893101] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=1 worker2 # [ 136.903062] nix-grpc-daemon[1593]: E0000 00:00:1790141709.252718 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 136.905189] nix-grpc-daemon[1593]: E0000 00:00:1790141709.252810 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 136.916271] nix-grpc-daemon[1593]: E0000 00:00:1790141709.266737 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 136.925230] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.930066] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 136.931675] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:38528 worker2 # [ 136.935612] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb # [ 137.984757] envoy[929]: conn peer=[2001:db8:1::1]:45730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2943 tx=1521 ms=7992 lb # [ 137.988833] envoy[929]: conn peer=[2001:db8:1::3]:46462 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3049 tx=870 ms=31257 lb # [ 137.992416] envoy[929]: conn peer=[2001:db8:1::4]:55140 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6171 tx=512 ms=14020 client: (finished: must succeed: sleep 5.0, in 5.03 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 142.122745] nix-daemon[847]: reaped child process 1601, status = succeeded worker1 # [ 142.198899] nix-daemon[847]: accepted connection from pid 1634, user nix-grpc-daemon (trusted) worker1 # [ 142.210046] nix-daemon[847]: reaped child process 1637, status = succeeded lb # [ 142.218421] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 142.228383] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.435Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 142.231061] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.437Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)" lb # [ 142.266707] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 142.273908] niks3-server[1135]: 2026/09/23 05:35:14 INFO Signed narinfos id=52 count=1 worker1 # [ 142.278494] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.485Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.288664] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls lb # [ 142.303888] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 142.308573] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 142.318032] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv lb # [ 142.322370] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo lb # [ 142.326855] niks3-server[1135]: 2026/09/23 05:35:14 INFO Completed upload id=52 worker1 # [ 142.327482] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.534Z level=INFO msg="Upload complete. (213ms)" worker1 # [ 142.334678] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40440 duration_s=6 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 142.349582] nix-daemon[847]: reaped child process 1599, status = succeeded client # [ 142.365363] systemd[1]: mid.service: Deactivated successfully. client # [ 142.369101] systemd[1]: mid.service: Consumed 132ms CPU time over 8.232s wall clock time, 43.4M memory peak, 7.6K incoming IP traffic, 8.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.14 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr client # [ 143.247500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 03b278f3d08d4d8890724ec39aa85e82 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds) worker2 # [ 143.357923] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 143.367870] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # [ 143.389987] nix[1430]: don't know how to build these paths: client # [ 143.391023] nix[1430]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker2 # [ 143.390141] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=0 worker2 # [ 143.394121] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 143.434353] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 143.446625] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5) worker2 # [ 143.477299] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5) worker2 # [ 143.530190] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 290 ms (attempt 3/5) worker2 # [ 143.827366] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5) worker2 # [ 144.090363] nix-grpc-daemon[1593]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 144.093351] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 144.108452] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 client # [ 144.127971] nix[1430]: copying 1 paths to 'lb:50051'... client # [ 144.132148] nix[1430]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'... worker2 # [ 144.121411] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 144.131164] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 144.152311] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=info event=niks3_push_started pid=1668 worker2 # [ 144.254450] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.604Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca="" worker2 # [ 144.339087] nix-daemon[870]: accepted connection from pid 1681, user nix-grpc-daemon (trusted) worker2 # [ 144.350281] nix-daemon[870]: reaped child process 1684, status = succeeded lb # [ 144.369345] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.364529] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.715Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 144.366207] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.716Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)" lb # [ 144.393533] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 144.396691] niks3-server[1135]: 2026/09/23 05:35:16 INFO Signed narinfos id=53 count=1 worker2 # [ 144.386317] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.736Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.398154] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls lb # [ 144.403072] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst lb # [ 144.406985] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 144.411079] niks3-server[1135]: 2026/09/23 05:35:16 INFO Completed upload id=53 worker2 # [ 144.401137] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.751Z level=INFO msg="Upload complete. (142ms)" lb # [ 144.413246] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo worker2 # [ 144.403361] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 144.407549] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 144.416767] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 144.420209] nix-daemon[870]: reaped child process 1666, status = succeeded worker2 # [ 144.427669] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # [ 144.436975] nix[1430]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 146.657167] systemd[1]: intr.service: Sent signal SIGINT to main process 1430 (nix) on client request. worker2 # [ 146.648434] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:18Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv client # [ 146.663176] nix[1430]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 146.667952] nix[1430]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds) client # [ 146.697677] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 146.700574] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 146.703946] systemd[1]: intr.service: Consumed 118ms CPU time over 3.453s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 146.848356] nix-daemon[870]: reaped child process 1690, status = succeeded worker2 # [ 146.851267] nix-daemon[1692]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 146.855399] nix-daemon[870]: reaped child process 1692, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.60 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw client # [ 147.869555] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: bbf402ed07a34245875e21c1895d9ae1 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds) lb # [ 147.984358] envoy[929]: conn peer=[2001:db8:1::1]:49868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1590 ms=8072 lb # [ 147.988151] envoy[929]: conn peer=[2001:db8:1::1]:53634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=828 ms=3318 worker2 # [ 147.994805] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55984 client # [ 148.011628] nix[1450]: don't know how to build these paths: client # [ 148.012582] nix[1450]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker2 # [ 148.008142] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=0 worker2 # [ 148.012558] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 148.072492] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41848 worker1 # [ 148.081033] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41848 client # [ 148.087162] nix[1450]: copying 1 paths to 'lb:50051'... worker1 # [ 148.090801] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41848 client # [ 148.089875] nix[1450]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'... worker1 # [ 148.096988] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) worker1 # [ 148.180129] nix-daemon[847]: accepted connection from pid 1669, user nix-grpc-daemon (trusted) worker1 # [ 148.192629] nix-daemon[847]: reaped child process 1672, status = succeeded lb # [ 148.199456] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 148.207780] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 148.209853] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.416Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)" lb # [ 148.229270] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls lb # [ 148.235275] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 148.241070] niks3-server[1135]: 2026/09/23 05:35:20 INFO Signed narinfos id=54 count=1 worker1 # [ 148.244506] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.451Z level=INFO msg="Uploading 1 narinfos" lb # [ 148.250525] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst lb # [ 148.259488] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 148.267688] niks3-server[1135]: 2026/09/23 05:35:20 INFO Completed upload id=54 worker1 # [ 148.270408] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.477Z level=INFO msg="Upload complete. (167ms)" lb # [ 148.271043] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo worker1 # [ 148.272793] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41848 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 148.279202] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41848 worker1 # [ 148.282986] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) worker1 # [ 148.285231] nix-daemon[847]: reaped child process 1667, status = succeeded worker1 # [ 148.290414] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted) client # [ 148.289931] nix[1450]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051... worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 149.161446] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 149.188584] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2 worker2 # [ 149.358328] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55984 client # don't know how to build these paths: client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker1 # [ 149.378492] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41848 worker2 # [ 149.388529] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=0 worker2 # [ 149.393692] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 149.445114] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 149.458483] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 client # copying 1 paths to 'lb:50051'... worker2 # [ 149.471692] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'... worker2 # [ 149.492150] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 149.591263] nix-daemon[870]: accepted connection from pid 1743, user nix-grpc-daemon (trusted) worker2 # [ 149.604315] nix-daemon[870]: reaped child process 1746, status = succeeded lb # [ 149.621284] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 149.619977] nix-grpc-daemon[1668]: time=2026-09-23T05:35:21.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 149.622250] nix-grpc-daemon[1668]: time=2026-09-23T05:35:21.972Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)" lb # [ 149.651706] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 149.657394] niks3-server[1135]: 2026/09/23 05:35:21 INFO Signed narinfos id=55 count=1 worker2 # [ 149.652771] nix-grpc-daemon[1668]: time=2026-09-23T05:35:22.002Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.668312] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls lb # [ 149.672820] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst lb # [ 149.683862] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 149.686922] niks3-server[1135]: 2026/09/23 05:35:21 INFO Completed upload id=55 worker2 # [ 149.676427] nix-grpc-daemon[1668]: time=2026-09-23T05:35:22.026Z level=INFO msg="Upload complete. (172ms)" lb # [ 149.689932] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo worker2 # [ 149.680679] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 149.685616] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 149.694458] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 149.700482] nix-daemon[870]: reaped child process 1741, status = succeeded worker2 # [ 149.704846] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051... client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker1 # [ 149.919576] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:22Z level=info event=draining client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 150.741347] nix-daemon[870]: reaped child process 1750, status = succeeded worker2 # [ 150.808594] nix-daemon[870]: accepted connection from pid 1766, user nix-grpc-daemon (trusted) worker2 # [ 150.820584] nix-daemon[870]: reaped child process 1769, status = succeeded lb # [ 150.839863] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 150.837911] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 150.839630] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.189Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)" lb # [ 150.880856] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv lb # [ 150.887347] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 150.900410] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 150.904472] niks3-server[1135]: 2026/09/23 05:35:23 INFO Signed narinfos id=56 count=1 worker2 # [ 150.896583] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.246Z level=INFO msg="Uploading 1 narinfos" lb # [ 150.911263] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls lb # [ 150.921416] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 150.925239] niks3-server[1135]: 2026/09/23 05:35:23 INFO Completed upload id=56 lb # [ 150.927249] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo worker2 # [ 150.916221] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.266Z level=INFO msg="Upload complete. (203ms)" worker2 # [ 150.924065] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=1 worker2 # [ 150.934816] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=0 worker2 # [ 150.939309] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 150.948718] nix-daemon[870]: reaped child process 1748, status = succeeded worker2 # [ 150.979620] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 150.994714] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 151.007839] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.358Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 151.016570] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'... worker2 # [ 151.036140] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 151.132877] nix-daemon[870]: accepted connection from pid 1775, user nix-grpc-daemon (trusted) worker2 # [ 151.145784] nix-daemon[870]: reaped child process 1778, status = succeeded lb # [ 151.163458] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 151.160377] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.510Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 151.161925] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.512Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)" lb # [ 151.195064] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker2 # [ 151.191976] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.542Z level=INFO msg="Uploading 1 narinfos" lb # [ 151.203725] niks3-server[1135]: 2026/09/23 05:35:23 INFO Signed narinfos id=57 count=1 lb # [ 151.209772] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst lb # [ 151.216103] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls lb # [ 151.228817] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 151.235245] niks3-server[1135]: 2026/09/23 05:35:23 INFO Completed upload id=57 worker2 # [ 151.224782] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.575Z level=INFO msg="Upload complete. (176ms)" lb # [ 151.241186] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo worker2 # [ 151.228871] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 151.236239] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 151.244912] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 151.254823] nix-daemon[870]: reaped child process 1773, status = succeeded worker2 # [ 151.259083] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051... client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 152.494829] nix-daemon[870]: reaped child process 1782, status = succeeded worker2 # [ 152.579087] nix-daemon[870]: accepted connection from pid 1798, user nix-grpc-daemon (trusted) worker2 # [ 152.592775] nix-daemon[870]: reaped child process 1801, status = succeeded lb # [ 152.608769] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 152.609277] nix-grpc-daemon[1668]: time=2026-09-23T05:35:24.959Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 152.610894] nix-grpc-daemon[1668]: time=2026-09-23T05:35:24.961Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)" lb # [ 152.652165] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv lb # [ 152.664827] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 152.670055] niks3-server[1135]: 2026/09/23 05:35:24 INFO Signed narinfos id=58 count=1 worker2 # [ 152.662331] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.012Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.676843] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls lb # [ 152.681447] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst lb # [ 152.690243] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 152.694188] niks3-server[1135]: 2026/09/23 05:35:24 INFO Completed upload id=58 worker2 # [ 152.683875] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.034Z level=INFO msg="Upload complete. (217ms)" lb # [ 152.696995] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo worker2 # [ 152.691892] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=1 worker2 # [ 152.703989] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=0 worker2 # [ 152.708178] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 152.717331] nix-daemon[870]: reaped child process 1780, status = succeeded worker2 # [ 152.748237] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 152.760849] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 152.774376] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.124Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'... worker2 # [ 152.783553] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 152.794377] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 152.889648] nix-daemon[870]: accepted connection from pid 1807, user nix-grpc-daemon (trusted) worker2 # [ 152.902091] nix-daemon[870]: reaped child process 1810, status = succeeded lb # [ 152.919936] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 152.920448] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.270Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 152.922087] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.272Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)" lb # [ 152.955960] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 152.961487] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls worker2 # [ 152.958197] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.308Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.969314] niks3-server[1135]: 2026/09/23 05:35:25 INFO Signed narinfos id=59 count=1 lb # [ 152.978640] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst lb # [ 152.990709] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 152.998530] niks3-server[1135]: 2026/09/23 05:35:25 INFO Completed upload id=59 worker2 # [ 152.988155] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.338Z level=INFO msg="Upload complete. (180ms)" lb # [ 153.000611] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo worker2 # [ 152.992394] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 152.998423] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 153.008873] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 153.020808] nix-daemon[870]: reaped child process 1805, status = succeeded worker2 # [ 153.023962] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051... client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 154.256672] nix-daemon[870]: reaped child process 1814, status = succeeded worker2 # [ 154.341384] nix-daemon[870]: accepted connection from pid 1830, user nix-grpc-daemon (trusted) worker2 # [ 154.354336] nix-daemon[870]: reaped child process 1833, status = succeeded lb # [ 154.371853] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 154.372938] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.723Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 154.375417] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.725Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)" lb # [ 154.413101] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls lb # [ 154.425462] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 154.430324] niks3-server[1135]: 2026/09/23 05:35:26 INFO Signed narinfos id=60 count=1 worker2 # [ 154.422536] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.772Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.442378] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 154.449955] niks3-server[1135]: 2026/09/23 05:35:26 INFO Completed upload id=60 worker2 # [ 154.439595] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.790Z level=INFO msg="Upload complete. (206ms)" lb # [ 154.452337] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst lb # [ 154.455436] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv lb # [ 154.458381] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo worker2 # [ 154.445564] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=1 worker2 # [ 154.460740] nix-daemon[870]: reaped child process 1812, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.31 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 154.710128] nix-daemon[847]: reaped child process 1676, status = succeeded worker1 # [ 154.781282] nix-daemon[847]: accepted connection from pid 1714, user nix-grpc-daemon (trusted) worker1 # [ 154.794397] nix-daemon[847]: reaped child process 1717, status = succeeded lb # [ 154.799853] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 154.809058] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.016Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 154.811235] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.018Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)" lb # [ 154.840283] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls lb # [ 154.849980] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 154.856660] niks3-server[1135]: 2026/09/23 05:35:27 INFO Signed narinfos id=61 count=1 worker1 # [ 154.861103] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.068Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.860737] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv lb # [ 154.868299] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 154.878696] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 154.884749] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo lb # [ 154.887905] niks3-server[1135]: 2026/09/23 05:35:27 INFO Completed upload id=61 worker1 # [ 154.888589] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.095Z level=INFO msg="Upload complete. (187ms)" worker1 # [ 154.895848] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41848 duration_s=6 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=1 worker1 # [ 154.910121] nix-daemon[847]: reaped child process 1674, status = succeeded worker1 # [ 154.920310] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:27Z level=info event=signal_exit client # [ 154.924668] systemd[1]: sw.service: Deactivated successfully. client # [ 154.929168] systemd[1]: sw.service: Consumed 120ms CPU time over 7.058s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker2 # [ 155.924308] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:38528 worker2 # [ 155.930879] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:28Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 155.934932] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:28Z level=info event=scheduler_restarting worker1 # [ 155.950185] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 155.972176] nix-daemon[847]: reaped child process 1569, status = succeeded worker1 # [ 156.825414] nix-grpc-daemon[1512]: E0000 00:00:1790141729.032545 1558 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true worker1 # [ 156.952477] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 156.957368] systemd[1]: nix-grpc-daemon.service: Consumed 644ms CPU time over 24.153s wall clock time, 35.8M memory peak, 25.1K incoming IP traffic, 35K outgoing IP traffic. worker1 # [ 156.972659] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 157.056684] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 157.058695] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 157.061942] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 157.066327] nix-grpc-daemon[1731]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 157.067660] nix-grpc-daemon[1731]: E0000 00:00:1790141729.273547 1731 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 157.073492] nix-grpc-daemon[1731]: E0000 00:00:1790141729.278836 1731 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 157.077989] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 157.081149] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 157.084090] nix-grpc-daemon[1731]: E0000 00:00:1790141729.290382 1773 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 157.096832] nix-grpc-daemon[1731]: E0000 00:00:1790141729.304169 1773 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.32 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 157.100295] nix-grpc-daemon[1731]: E0000 00:00:1790141729.306451 1773 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 157.111206] nix-grpc-daemon[1731]: E0000 00:00:1790141729.318517 1773 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 157.124211] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.116454] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:38528 worker2 # [ 157.120977] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 9.40 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 157.366832] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 96dd86909bdd4f4584775c7a7c62fc14 client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 157.504149] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 157.515814] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) client # [ 157.528281] nix[1506]: don't know how to build these paths: client # [ 157.529213] nix[1506]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv client # [ 157.530120] nix[1506]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv client # [ 157.531166] nix[1506]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv client # [ 157.532407] nix[1506]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker2 # [ 157.524409] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=0 worker2 # [ 157.530575] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 157.534985] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=0 worker2 # [ 157.538939] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 157.543306] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=0 worker2 # [ 157.547251] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=0 worker1 # [ 157.585100] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker2 # [ 157.574967] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker1 # [ 157.596476] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5) worker2 # [ 157.587550] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 client # [ 157.608189] nix[1506]: copying 1 paths to 'lb:50051'... worker2 # [ 157.598688] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 client # [ 157.611142] nix[1506]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'... worker1 # [ 157.612558] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5) worker2 # [ 157.611549] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 157.705929] nix-daemon[870]: accepted connection from pid 1848, user nix-grpc-daemon (trusted) worker2 # [ 157.718615] nix-daemon[870]: reaped child process 1851, status = succeeded lb # [ 157.736911] niks3-server[1135]: 2026/09/23 05:35:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 157.735364] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 157.737508] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.087Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)" lb # [ 157.769111] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 157.774250] niks3-server[1135]: 2026/09/23 05:35:30 INFO Signed narinfos id=62 count=1 worker2 # [ 157.766577] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.116Z level=INFO msg="Uploading 1 narinfos" worker1 # [ 157.775518] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 3/5) lb # [ 157.783194] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst worker1 # [ 157.786599] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 387 ms (attempt 4/5) lb # [ 157.795412] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 157.803147] niks3-server[1135]: 2026/09/23 05:35:30 INFO Completed upload id=62 worker2 # [ 157.792344] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.142Z level=INFO msg="Upload complete. (170ms)" lb # [ 157.806392] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls worker2 # [ 157.796718] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 157.813780] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo worker2 # [ 157.801923] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 157.813602] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 157.815928] nix-daemon[870]: reaped child process 1846, status = succeeded worker2 # [ 157.820675] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # [ 157.831196] nix[1506]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051... lb # [ 157.985348] envoy[929]: conn peer=[2001:db8:1::1]:53650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9624 tx=4405 ms=5150 lb # [ 157.989859] envoy[929]: conn peer=[2001:db8:1::1]:53648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1519 ms=6900 lb # [ 157.993930] envoy[929]: conn peer=[2001:db8:1::3]:44178 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1597 tx=532 ms=22984 worker1 # [ 158.182898] nix-grpc-daemon[1731]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 158.186779] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 158.202423] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 client # [ 158.212573] nix[1506]: copying 1 paths to 'lb:50051'... client # [ 158.215760] nix[1506]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'... worker1 # [ 158.217070] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 158.225549] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 158.248946] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=info event=niks3_push_started pid=1792 worker1 # [ 158.348891] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.555Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca="" worker1 # [ 158.432717] nix-daemon[847]: accepted connection from pid 1805, user nix-grpc-daemon (trusted) worker1 # [ 158.444400] nix-daemon[847]: reaped child process 1808, status = succeeded lb # [ 158.451985] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 158.460472] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.667Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 158.461971] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.669Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)" lb: must succeed: curl -sf localhost:9901/clusters lb # [ 158.492911] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 158.498776] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls worker1 # [ 158.509787] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.716Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.508808] niks3-server[1135]: 2026/09/23 05:35:30 INFO Signed narinfos id=63 count=1 lb # [ 158.513229] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst lb # [ 158.532124] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 158.536565] niks3-server[1135]: 2026/09/23 05:35:30 INFO Completed upload id=63 worker1 # [ 158.537109] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.744Z level=INFO msg="Upload complete. (184ms)" lb # [ 158.539336] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo worker1 # [ 158.540106] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 158.546129] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 158.555242] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 158.560600] nix-daemon[847]: reaped child process 1790, status = succeeded worker1 # [ 158.565455] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) client # [ 158.565657] nix[1506]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051... lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 159.835614] nix-daemon[870]: reaped child process 1855, status = succeeded worker2 # [ 159.908616] nix-daemon[870]: accepted connection from pid 1883, user nix-grpc-daemon (trusted) worker2 # [ 159.921488] nix-daemon[870]: reaped child process 1886, status = succeeded lb # [ 159.938809] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 159.938373] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 159.940069] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.290Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)" lb # [ 159.981635] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst lb # [ 159.990359] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls lb # [ 159.995766] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 160.001255] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=64 count=1 worker2 # [ 159.993197] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.343Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.005534] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker1 # [ 160.006250] nix-daemon[847]: reaped child process 1812, status = succeeded lb # [ 160.010955] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 160.015342] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=64 worker2 # [ 160.006261] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.356Z level=INFO msg="Upload complete. (178ms)" lb # [ 160.018725] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo worker2 # [ 160.011532] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=6 outputs=1 worker2 # [ 160.017949] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 160.023395] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 160.031913] nix-daemon[870]: reaped child process 1853, status = succeeded worker2 # [ 160.033377] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 160.038843] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 client # [ 160.049866] nix[1506]: copying 1 paths to 'lb:50051'... client # [ 160.052515] nix[1506]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'... worker2 # [ 160.047570] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker1 # [ 160.086687] nix-daemon[847]: accepted connection from pid 1837, user nix-grpc-daemon (trusted) worker1 # [ 160.100218] nix-daemon[847]: reaped child process 1840, status = succeeded lb # [ 160.104691] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.114439] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.116238] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.323Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)" lb # [ 160.137767] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst worker2 # [ 160.129947] nix-daemon[870]: accepted connection from pid 1890, user nix-grpc-daemon (trusted) lb # [ 160.143159] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv lb # [ 160.145953] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 160.149530] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=65 count=1 worker1 # [ 160.150316] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.357Z level=INFO msg="Uploading 1 narinfos" worker2 # [ 160.142404] nix-daemon[870]: reaped child process 1893, status = succeeded lb # [ 160.153267] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls lb # [ 160.157060] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 160.159849] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.162907] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.370Z level=INFO msg="Upload complete. (189ms)" lb # [ 160.162929] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=65 worker1 # [ 160.165421] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=7 outputs=1 worker2 # [ 160.155226] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=1 lb # [ 160.166434] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo worker2 # [ 160.158923] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.170989] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker2 # [ 160.160504] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.509Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)" worker1 # [ 160.175513] nix-daemon[847]: reaped child process 1810, status = succeeded lb # [ 160.178986] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls worker1 # [ 160.181544] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 lb # [ 160.182075] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 160.183846] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=66 count=1 worker2 # [ 160.174508] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.524Z level=INFO msg="Uploading 1 narinfos" worker1 # [ 160.192426] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 client # [ 160.192171] nix[1506]: copying 1 paths to 'lb:50051'... lb # [ 160.192621] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 160.194707] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst client # [ 160.194348] nix[1506]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'... lb # [ 160.198707] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=66 worker1 # [ 160.200462] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker2 # [ 160.189226] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.539Z level=INFO msg="Upload complete. (136ms)" lb # [ 160.200709] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo worker2 # [ 160.191538] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 160.194235] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 160.202710] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 160.204079] nix-daemon[870]: reaped child process 1888, status = succeeded worker2 # [ 160.209250] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # [ 160.219183] nix[1506]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051... worker1 # [ 160.282443] nix-daemon[847]: accepted connection from pid 1844, user nix-grpc-daemon (trusted) worker1 # [ 160.295145] nix-daemon[847]: reaped child process 1847, status = succeeded lb # [ 160.301083] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.309320] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.516Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.311213] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.518Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)" lb # [ 160.329352] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst lb # [ 160.336159] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 160.339489] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=67 count=1 worker1 # [ 160.342967] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.550Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.345887] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls lb # [ 160.349959] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 160.353833] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=67 worker1 # [ 160.354140] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.561Z level=INFO msg="Upload complete. (148ms)" lb # [ 160.355422] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo worker1 # [ 160.356344] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 160.360215] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 160.366165] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 160.368400] nix-daemon[847]: reaped child process 1842, status = succeeded worker1 # [ 160.373471] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) client # [ 160.372870] nix[1506]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051... worker1 # [ 161.006412] nix-daemon[847]: reaped child process 1851, status = succeeded worker1 # [ 161.071651] nix-daemon[847]: accepted connection from pid 1872, user nix-grpc-daemon (trusted) worker1 # [ 161.085219] nix-daemon[847]: reaped child process 1875, status = succeeded lb # [ 161.090779] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 161.100276] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 161.101750] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.309Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)" lb # [ 161.135494] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 161.140574] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.347Z level=INFO msg="Uploading 1 narinfos" lb # [ 161.139840] niks3-server[1135]: 2026/09/23 05:35:33 INFO Signed narinfos id=68 count=1 lb # [ 161.151135] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv lb # [ 161.154666] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls lb # [ 161.163466] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst lb # [ 161.167062] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 161.170431] niks3-server[1135]: 2026/09/23 05:35:33 INFO Completed upload id=68 worker1 # [ 161.171079] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.378Z level=INFO msg="Upload complete. (191ms)" lb # [ 161.173256] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo worker1 # [ 161.176896] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=9 outputs=1 worker1 # [ 161.188840] nix-daemon[847]: reaped child process 1849, status = succeeded worker2 # [ 162.225692] nix-daemon[870]: reaped child process 1897, status = succeeded worker2 # [ 162.298875] nix-daemon[870]: accepted connection from pid 1922, user nix-grpc-daemon (trusted) worker2 # [ 162.312178] nix-daemon[870]: reaped child process 1925, status = succeeded lb # [ 162.329523] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 162.327982] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.678Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 162.329689] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.679Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)" lb # [ 162.373738] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker2 # [ 162.370172] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.720Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.380413] niks3-server[1135]: 2026/09/23 05:35:34 INFO Signed narinfos id=69 count=1 lb # [ 162.387969] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv lb # [ 162.393462] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst lb # [ 162.401301] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls lb # [ 162.411507] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 162.414547] niks3-server[1135]: 2026/09/23 05:35:34 INFO Completed upload id=69 worker2 # [ 162.405430] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.755Z level=INFO msg="Upload complete. (189ms)" lb # [ 162.418513] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo worker2 # [ 162.408075] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=8 outputs=1 worker2 # [ 162.415837] nix-daemon[870]: reaped child process 1895, status = succeeded client # [ 162.453692] systemd[1]: burst.service: Deactivated successfully. client # [ 162.457956] systemd[1]: burst.service: Consumed 156ms CPU time over 5.091s wall clock time, 53.7M memory peak, 13.8K incoming IP traffic, 18.5K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.11 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 163.474585] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 163.480662] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker2 # [ 163.506171] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=0 worker2 # [ 163.508271] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 163.510110] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=0 worker2 # [ 163.511665] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=0 worker1 # [ 163.561714] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 163.574549] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 163.586401] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 163.594994] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 163.696607] nix-daemon[847]: accepted connection from pid 1887, user nix-grpc-daemon (trusted) worker1 # [ 163.708632] nix-daemon[847]: reaped child process 1890, status = succeeded lb # [ 163.716424] niks3-server[1135]: 2026/09/23 05:35:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 163.725682] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.932Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 163.727792] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.934Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)" lb # [ 163.748813] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst lb # [ 163.756483] niks3-server[1135]: 2026/09/23 05:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 163.764650] niks3-server[1135]: 2026/09/23 05:35:36 INFO Signed narinfos id=70 count=1 worker1 # [ 163.768407] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.975Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.771409] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls lb # [ 163.783336] niks3-server[1135]: 2026/09/23 05:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 163.789575] niks3-server[1135]: 2026/09/23 05:35:36 INFO Completed upload id=70 worker1 # [ 163.789636] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.996Z level=INFO msg="Upload complete. (181ms)" lb # [ 163.792973] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo worker1 # [ 163.794463] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 163.800505] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 163.809081] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 163.819645] nix-daemon[847]: reaped child process 1885, status = succeeded worker1 # [ 163.824587] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 165.060731] nix-daemon[847]: reaped child process 1894, status = succeeded worker1 # [ 165.135484] nix-daemon[847]: accepted connection from pid 1910, user nix-grpc-daemon (trusted) worker1 # [ 165.147109] nix-daemon[847]: reaped child process 1913, status = succeeded lb # [ 165.153772] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.163551] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 165.165131] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.372Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)" lb # [ 165.199369] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls lb # [ 165.206270] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 165.216847] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.423Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.217413] niks3-server[1135]: 2026/09/23 05:35:37 INFO Signed narinfos id=71 count=1 lb # [ 165.220681] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 165.226427] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv lb # [ 165.236864] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 165.240835] niks3-server[1135]: 2026/09/23 05:35:37 INFO Completed upload id=71 worker1 # [ 165.241498] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.448Z level=INFO msg="Upload complete. (209ms)" lb # [ 165.244284] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo worker2 # [ 165.237472] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:37Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 165.242357] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:37Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=0 worker1 # [ 165.247984] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=10 outputs=1 worker1 # [ 165.257748] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 165.268571] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 165.276809] nix-daemon[847]: reaped child process 1892, status = succeeded worker1 # [ 165.279598] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 165.285515] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 165.367751] nix-daemon[847]: accepted connection from pid 1917, user nix-grpc-daemon (trusted) worker1 # [ 165.380918] nix-daemon[847]: reaped child process 1920, status = succeeded lb # [ 165.386152] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.394303] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.601Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 165.396299] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.603Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)" lb # [ 165.417954] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 165.424182] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.631Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.423980] niks3-server[1135]: 2026/09/23 05:35:37 INFO Signed narinfos id=72 count=1 lb # [ 165.431381] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls lb # [ 165.437723] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst lb # [ 165.450587] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 165.454430] niks3-server[1135]: 2026/09/23 05:35:37 INFO Completed upload id=72 worker1 # [ 165.456343] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.663Z level=INFO msg="Upload complete. (165ms)" lb # [ 165.457850] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo worker1 # [ 165.458887] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 165.464124] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 165.472691] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 165.480785] nix-daemon[847]: reaped child process 1915, status = succeeded worker1 # [ 165.485386] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 166.322893] nix-daemon[847]: reaped child process 1924, status = succeeded worker1 # [ 166.399930] nix-daemon[847]: accepted connection from pid 1940, user nix-grpc-daemon (trusted) worker1 # [ 166.411759] nix-daemon[847]: reaped child process 1943, status = succeeded lb # [ 166.417834] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.426641] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 166.428137] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.635Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)" lb # [ 166.458462] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv lb # [ 166.470927] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 166.473921] niks3-server[1135]: 2026/09/23 05:35:38 INFO Signed narinfos id=73 count=1 worker1 # [ 166.476995] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.684Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.483234] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls lb # [ 166.488312] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 166.498777] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 166.501897] niks3-server[1135]: 2026/09/23 05:35:38 INFO Completed upload id=73 worker1 # [ 166.502320] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.709Z level=INFO msg="Upload complete. (209ms)" lb # [ 166.504528] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo worker2 # [ 166.497115] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:38Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 166.508089] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=11 outputs=1 worker2 # [ 166.502349] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:38Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=0 worker1 # [ 166.518186] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 166.529525] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 166.536530] nix-daemon[847]: reaped child process 1922, status = succeeded worker1 # [ 166.539702] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 166.545250] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 166.628460] nix-daemon[847]: accepted connection from pid 1947, user nix-grpc-daemon (trusted) worker1 # [ 166.640453] nix-daemon[847]: reaped child process 1950, status = succeeded lb # [ 166.645421] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.654102] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.861Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 166.655599] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.862Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)" lb # [ 166.678912] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst lb # [ 166.685293] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 166.690845] niks3-server[1135]: 2026/09/23 05:35:38 INFO Signed narinfos id=74 count=1 worker1 # [ 166.694039] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.901Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.699277] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls lb # [ 166.712319] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 166.718770] niks3-server[1135]: 2026/09/23 05:35:38 INFO Completed upload id=74 worker1 # [ 166.719231] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.926Z level=INFO msg="Upload complete. (168ms)" lb # [ 166.720869] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo worker1 # [ 166.721960] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 166.726875] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 166.736106] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 166.743787] nix-daemon[847]: reaped child process 1945, status = succeeded worker1 # [ 166.748698] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 167.980550] nix-daemon[847]: reaped child process 1954, status = succeeded lb # [ 167.985266] envoy[929]: conn peer=[2001:db8:1::1]:56168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=5167 ms=4935 worker1 # [ 168.056341] nix-daemon[847]: accepted connection from pid 1970, user nix-grpc-daemon (trusted) worker1 # [ 168.067888] nix-daemon[847]: reaped child process 1973, status = succeeded lb # [ 168.077083] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.086078] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.293Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 168.087632] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.294Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)" lb # [ 168.120382] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker1 # [ 168.129679] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.336Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.129671] niks3-server[1135]: 2026/09/23 05:35:40 INFO Signed narinfos id=75 count=1 lb # [ 168.135634] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 168.142714] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv lb # [ 168.147445] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls lb # [ 168.158885] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 168.163117] niks3-server[1135]: 2026/09/23 05:35:40 INFO Completed upload id=75 worker1 # [ 168.165395] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.372Z level=INFO msg="Upload complete. (209ms)" lb # [ 168.167696] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo worker2 # [ 168.160619] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:40Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 168.165706] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:40Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=0 worker1 # [ 168.174171] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=12 outputs=1 worker1 # [ 168.181689] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 168.192519] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 168.200488] nix-daemon[847]: reaped child process 1952, status = succeeded worker1 # [ 168.203416] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.410Z level=INFO msg="All 1 paths already cached" worker1 # [ 168.206220] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 168.210840] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 168.292476] nix-daemon[847]: accepted connection from pid 1977, user nix-grpc-daemon (trusted) worker1 # [ 168.305441] nix-daemon[847]: reaped child process 1980, status = succeeded lb # [ 168.311531] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 168.319953] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 168.321975] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.528Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)" lb # [ 168.341918] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 168.350631] niks3-server[1135]: 2026/09/23 05:35:40 INFO Signed narinfos id=76 count=1 worker1 # [ 168.355086] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.562Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.354637] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls lb # [ 168.363989] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst lb # [ 168.375245] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 168.380931] niks3-server[1135]: 2026/09/23 05:35:40 INFO Completed upload id=76 worker1 # [ 168.380958] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.588Z level=INFO msg="Upload complete. (165ms)" lb # [ 168.384712] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo worker1 # [ 168.384246] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 168.389662] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 168.400108] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 168.404623] nix-daemon[847]: reaped child process 1975, status = succeeded worker1 # [ 168.409581] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 170.035277] nix-daemon[847]: reaped child process 1984, status = succeeded worker1 # [ 170.107285] nix-daemon[847]: accepted connection from pid 2000, user nix-grpc-daemon (trusted) worker1 # [ 170.120159] nix-daemon[847]: reaped child process 2003, status = succeeded lb # [ 170.126921] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.136532] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.343Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 170.138218] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.345Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)" lb # [ 170.167725] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 170.176546] niks3-server[1135]: 2026/09/23 05:35:42 INFO Signed narinfos id=77 count=1 worker1 # [ 170.180387] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.387Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.186747] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls lb # [ 170.191096] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv lb # [ 170.195677] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst lb # [ 170.205418] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 170.209879] niks3-server[1135]: 2026/09/23 05:35:42 INFO Completed upload id=77 lb # [ 170.211569] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo worker1 # [ 170.210629] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.417Z level=INFO msg="Upload complete. (191ms)" worker2 # [ 170.206614] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:42Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 170.211745] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:42Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=0 worker1 # [ 170.219254] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=13 outputs=1 worker1 # [ 170.227463] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 170.240519] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 170.247579] nix-daemon[847]: reaped child process 1982, status = succeeded worker1 # [ 170.250153] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.457Z level=INFO msg="All 1 paths already cached" worker1 # [ 170.255887] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 170.262845] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 170.345361] nix-daemon[847]: accepted connection from pid 2007, user nix-grpc-daemon (trusted) worker1 # [ 170.358300] nix-daemon[847]: reaped child process 2010, status = succeeded lb # [ 170.364333] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 170.373463] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 170.375455] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.582Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)" lb # [ 170.395929] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 170.404635] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.611Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.405122] niks3-server[1135]: 2026/09/23 05:35:42 INFO Signed narinfos id=78 count=1 lb # [ 170.407625] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls lb # [ 170.416209] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst lb # [ 170.427475] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 170.434751] niks3-server[1135]: 2026/09/23 05:35:42 INFO Completed upload id=78 worker1 # [ 170.435289] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.642Z level=INFO msg="Upload complete. (166ms)" lb # [ 170.437922] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo worker1 # [ 170.438976] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 170.444720] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 170.453592] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 170.461165] nix-daemon[847]: reaped child process 2005, status = succeeded worker1 # [ 170.465940] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 171.297395] nix-daemon[847]: reaped child process 2014, status = succeeded worker1 # [ 171.376181] nix-daemon[847]: accepted connection from pid 2030, user nix-grpc-daemon (trusted) worker1 # [ 171.388970] nix-daemon[847]: reaped child process 2033, status = succeeded lb # [ 171.395934] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 171.404994] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.612Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 171.406634] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.613Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)" lb # [ 171.433402] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv lb # [ 171.446115] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst lb # [ 171.454352] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 171.459908] niks3-server[1135]: 2026/09/23 05:35:43 INFO Signed narinfos id=79 count=1 worker1 # [ 171.462618] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.669Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.472103] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 171.477624] niks3-server[1135]: 2026/09/23 05:35:43 INFO Completed upload id=79 worker1 # [ 171.477912] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.685Z level=INFO msg="Upload complete. (202ms)" lb # [ 171.479347] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls lb # [ 171.482213] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo worker2 # [ 171.474907] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:43Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 171.484921] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=14 outputs=1 worker2 # [ 171.479934] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:43Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=0 worker1 # [ 171.494327] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 171.503213] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 171.512573] nix-daemon[847]: reaped child process 2012, status = succeeded worker1 # [ 171.514700] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.721Z level=INFO msg="All 1 paths already cached" worker1 # [ 171.517037] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 171.521811] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 171.603538] nix-daemon[847]: accepted connection from pid 2037, user nix-grpc-daemon (trusted) worker1 # [ 171.617075] nix-daemon[847]: reaped child process 2040, status = succeeded lb # [ 171.621990] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 171.631339] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 171.633294] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.840Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)" lb # [ 171.654585] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 171.659477] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls lb # [ 171.665524] niks3-server[1135]: 2026/09/23 05:35:43 INFO Signed narinfos id=80 count=1 worker1 # [ 171.668547] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.875Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.672051] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst lb # [ 171.680678] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 171.686526] niks3-server[1135]: 2026/09/23 05:35:43 INFO Completed upload id=80 worker1 # [ 171.687183] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.894Z level=INFO msg="Upload complete. (160ms)" lb # [ 171.688696] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo worker1 # [ 171.689957] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 171.694646] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 171.703865] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 171.708188] nix-daemon[847]: reaped child process 2035, status = succeeded worker1 # [ 171.712992] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 172.351279] nix-daemon[847]: reaped child process 2044, status = succeeded worker1 # [ 172.432641] nix-daemon[847]: accepted connection from pid 2060, user nix-grpc-daemon (trusted) worker1 # [ 172.445374] nix-daemon[847]: reaped child process 2063, status = succeeded lb # [ 172.452674] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.461866] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.668Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 172.463401] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.670Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)" lb # [ 172.495352] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 172.502132] niks3-server[1135]: 2026/09/23 05:35:44 INFO Signed narinfos id=81 count=1 worker1 # [ 172.506297] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.713Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.509860] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst lb # [ 172.518181] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls lb # [ 172.522772] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv lb # [ 172.531390] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 172.534723] niks3-server[1135]: 2026/09/23 05:35:44 INFO Completed upload id=81 worker1 # [ 172.535312] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.742Z level=INFO msg="Upload complete. (214ms)" lb # [ 172.538365] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo worker2 # [ 172.530851] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:44Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.541520] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=15 outputs=1 worker2 # [ 172.536051] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:44Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=0 worker1 # [ 172.550397] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 172.560893] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 172.569882] nix-daemon[847]: reaped child process 2042, status = succeeded worker1 # [ 172.573631] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.779Z level=INFO msg="All 2 paths already cached" worker1 # [ 172.577844] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 172.581714] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 172.663077] nix-daemon[847]: accepted connection from pid 2067, user nix-grpc-daemon (trusted) worker1 # [ 172.676313] nix-daemon[847]: reaped child process 2070, status = succeeded lb # [ 172.682122] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 172.691038] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.898Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 172.692934] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.899Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)" lb # [ 172.715932] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst lb # [ 172.723955] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 172.729253] niks3-server[1135]: 2026/09/23 05:35:44 INFO Signed narinfos id=82 count=1 worker1 # [ 172.733100] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.940Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.737173] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls lb # [ 172.749181] niks3-server[1135]: 2026/09/23 05:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 172.754094] niks3-server[1135]: 2026/09/23 05:35:45 INFO Completed upload id=82 worker1 # [ 172.754525] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.961Z level=INFO msg="Upload complete. (168ms)" lb # [ 172.757225] niks3-server[1135]: 2026/09/23 05:35:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo worker1 # [ 172.758837] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 172.764685] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 172.775111] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 172.780262] nix-daemon[847]: reaped child process 2065, status = succeeded worker1 # [ 172.785430] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 174.023229] nix-daemon[847]: reaped child process 2074, status = succeeded worker1 # [ 174.099367] nix-daemon[847]: accepted connection from pid 2090, user nix-grpc-daemon (trusted) worker1 # [ 174.111412] nix-daemon[847]: reaped child process 2093, status = succeeded lb # [ 174.121246] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.131396] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.338Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 174.133165] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.340Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)" lb # [ 174.165856] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls lb # [ 174.171262] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst lb # [ 174.177104] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 174.183630] niks3-server[1135]: 2026/09/23 05:35:46 INFO Signed narinfos id=83 count=1 worker1 # [ 174.184377] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.391Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.189980] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv lb # [ 174.195255] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 174.198575] niks3-server[1135]: 2026/09/23 05:35:46 INFO Completed upload id=83 worker1 # [ 174.199244] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.406Z level=INFO msg="Upload complete. (206ms)" lb # [ 174.201274] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo worker2 # [ 174.194898] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:46Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 174.205695] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=16 outputs=1 worker1 # [ 174.219840] nix-daemon[847]: reaped child process 2072, status = succeeded worker1 # [ 174.250585] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 174.263147] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 174.276815] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.483Z level=INFO msg="All 2 paths already cached" worker1 # [ 174.285758] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 174.295132] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 174.388472] nix-daemon[847]: accepted connection from pid 2097, user nix-grpc-daemon (trusted) worker1 # [ 174.401144] nix-daemon[847]: reaped child process 2100, status = succeeded lb # [ 174.407971] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 174.417195] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.624Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 174.418677] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.626Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)" lb # [ 174.444423] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 174.450953] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst lb # [ 174.457127] niks3-server[1135]: 2026/09/23 05:35:46 INFO Signed narinfos id=84 count=1 worker1 # [ 174.460414] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.667Z level=INFO msg="Uploading 1 narinfos" lb # [ 174.464492] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls lb # [ 174.473736] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 174.479555] niks3-server[1135]: 2026/09/23 05:35:46 INFO Completed upload id=84 worker1 # [ 174.482450] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.689Z level=INFO msg="Upload complete. (176ms)" worker1 # [ 174.486606] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 174.491677] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 174.501129] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 174.508926] nix-daemon[847]: reaped child process 2095, status = succeeded worker1 # [ 174.513844] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 175.552558] nix-daemon[847]: reaped child process 2104, status = succeeded worker1 # [ 175.630716] nix-daemon[847]: accepted connection from pid 2120, user nix-grpc-daemon (trusted) worker1 # [ 175.643079] nix-daemon[847]: reaped child process 2123, status = succeeded lb # [ 175.651457] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.660557] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.867Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 175.662138] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.869Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)" lb # [ 175.693232] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 175.699713] niks3-server[1135]: 2026/09/23 05:35:47 INFO Signed narinfos id=85 count=1 worker1 # [ 175.703108] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.910Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.709855] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst lb # [ 175.714977] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv lb # [ 175.720103] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls lb # [ 175.727886] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 175.735383] niks3-server[1135]: 2026/09/23 05:35:47 INFO Completed upload id=85 lb # [ 175.736643] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo worker1 # [ 175.735983] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.943Z level=INFO msg="Upload complete. (213ms)" worker2 # [ 175.731745] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:48Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 175.744468] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=17 outputs=1 worker1 # [ 175.758240] nix-daemon[847]: reaped child process 2102, status = succeeded worker1 # [ 175.787582] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 175.800273] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 175.813414] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.020Z level=INFO msg="All 2 paths already cached" worker1 # [ 175.821856] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 175.831166] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 175.925742] nix-daemon[847]: accepted connection from pid 2127, user nix-grpc-daemon (trusted) worker1 # [ 175.938538] nix-daemon[847]: reaped child process 2130, status = succeeded lb # [ 175.945242] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.955046] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.162Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 175.957139] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.163Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)" lb # [ 175.980519] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst lb # [ 175.990060] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign lb # [ 175.994958] niks3-server[1135]: 2026/09/23 05:35:48 INFO Signed narinfos id=86 count=1 worker1 # [ 175.998908] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.206Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.003253] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls lb # [ 176.013360] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete lb # [ 176.019761] niks3-server[1135]: 2026/09/23 05:35:48 INFO Completed upload id=86 worker1 # [ 176.020272] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.227Z level=INFO msg="Upload complete. (177ms)" lb # [ 176.024330] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo worker1 # [ 176.024076] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 176.031271] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 worker1 # [ 176.040150] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 176.048436] nix-daemon[847]: reaped child process 2125, status = succeeded worker1 # [ 176.053316] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 176.689563] nix-daemon[847]: reaped child process 2134, status = succeeded worker1 # [ 176.771719] nix-daemon[847]: accepted connection from pid 2150, user nix-grpc-daemon (trusted) worker1 # [ 176.784605] nix-daemon[847]: reaped child process 2153, status = succeeded lb # [ 176.791933] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.801404] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 176.803131] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.010Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)" lb # [ 176.837145] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign worker1 # [ 176.845243] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.052Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.845139] niks3-server[1135]: 2026/09/23 05:35:49 INFO Signed narinfos id=87 count=1 lb # [ 176.852512] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls lb # [ 176.858794] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv lb # [ 176.867112] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst lb # [ 176.874112] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 176.878520] niks3-server[1135]: 2026/09/23 05:35:49 INFO Completed upload id=87 worker1 # [ 176.879322] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.086Z level=INFO msg="Upload complete. (219ms)" lb # [ 176.883061] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo worker1 # [ 176.885608] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=18 outputs=1 worker1 # [ 176.900200] nix-daemon[847]: reaped child process 2132, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.60 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 177.022765] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 5376ae82352a44b7996bfcf05090471b client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) worker1 # [ 177.151168] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:51456 client # [ 177.157144] nix[1576]: don't know how to build these paths: client # [ 177.158061] nix[1576]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv worker2 # [ 177.148162] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 177.164277] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:49Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=0 client # [ 177.175737] nix[1576]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker lb # [ 177.985634] envoy[929]: conn peer=[2001:db8:1::1]:42090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27124 tx=12268 ms=13428 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code client # [ 207.178845] nix[1576]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip} client # [ 207.186823] nix[1576]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed client # [ 207.220227] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 207.224168] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 207.229152] systemd[1]: gpu.service: Consumed 136ms CPU time over 30.205s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.41 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.46 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill lb # [ 207.986263] envoy[929]: conn peer=[2001:db8:1::1]:35838 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=30049 worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 209.096252] nix-grpc-daemon[1731]: ts=2026-09-23T05:36:21Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker2 # [ 210.015840] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 210.021170] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 210.041444] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=0 worker2 # [ 210.046382] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 210.090483] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 210.103514] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'... worker2 # [ 210.116140] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 210.127165] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 210.231468] nix-daemon[870]: accepted connection from pid 1963, user nix-grpc-daemon (trusted) worker2 # [ 210.245977] nix-daemon[870]: reaped child process 1966, status = succeeded lb # [ 210.263933] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 210.262190] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 210.264235] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.614Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)" lb # [ 210.304219] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign lb # [ 210.309583] niks3-server[1135]: 2026/09/23 05:36:22 INFO Signed narinfos id=88 count=1 worker2 # [ 210.302311] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.652Z level=INFO msg="Uploading 1 narinfos" lb # [ 210.317952] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls lb # [ 210.324450] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst lb # [ 210.340329] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 210.343043] niks3-server[1135]: 2026/09/23 05:36:22 INFO Completed upload id=88 worker2 # [ 210.332630] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.683Z level=INFO msg="Upload complete. (194ms)" lb # [ 210.345714] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo worker2 # [ 210.337933] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 210.344341] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 210.355200] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 210.363651] nix-daemon[870]: reaped child process 1961, status = succeeded worker2 # [ 210.371871] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051... client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 211.807232] nix-daemon[870]: reaped child process 1970, status = succeeded worker2 # [ 211.893272] nix-daemon[870]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker2 # [ 211.904416] nix-daemon[870]: reaped child process 1989, status = succeeded lb # [ 211.923883] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 211.924561] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.274Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 211.926193] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.276Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)" lb # [ 211.967935] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv lb # [ 211.970354] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign lb # [ 211.973425] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 211.975757] niks3-server[1135]: 2026/09/23 05:36:24 INFO Signed narinfos id=89 count=1 worker2 # [ 211.966601] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.316Z level=INFO msg="Uploading 1 narinfos" lb # [ 211.980107] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls lb # [ 211.983673] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete lb # [ 211.988784] niks3-server[1135]: 2026/09/23 05:36:24 INFO Completed upload id=89 worker2 # [ 211.978338] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.328Z level=INFO msg="Upload complete. (198ms)" lb # [ 211.991678] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo worker2 # [ 211.983647] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=19 outputs=1 worker2 # [ 211.993663] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=0 worker2 # [ 211.995968] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 212.001134] nix-daemon[870]: reaped child process 1968, status = succeeded worker2 # [ 212.038192] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 212.049183] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 212.067957] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.418Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'... worker2 # [ 212.075944] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 212.086265] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 212.181514] nix-daemon[870]: accepted connection from pid 1995, user nix-grpc-daemon (trusted) worker2 # [ 212.193804] nix-daemon[870]: reaped child process 1998, status = succeeded lb # [ 212.212205] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 212.214329] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.564Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 212.215876] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.566Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)" lb # [ 212.249323] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign lb # [ 212.252681] niks3-server[1135]: 2026/09/23 05:36:24 INFO Signed narinfos id=90 count=1 worker2 # [ 212.242150] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.592Z level=INFO msg="Uploading 1 narinfos" lb # [ 212.257584] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls lb # [ 212.260254] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst lb # [ 212.267846] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete lb # [ 212.271633] niks3-server[1135]: 2026/09/23 05:36:24 INFO Completed upload id=90 worker2 # [ 212.261126] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.611Z level=INFO msg="Upload complete. (163ms)" lb # [ 212.273728] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo worker2 # [ 212.263374] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 212.267521] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 212.273761] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 212.275982] nix-daemon[870]: reaped child process 1993, status = succeeded worker2 # [ 212.281202] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051... client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 213.318143] nix-daemon[870]: reaped child process 2002, status = succeeded worker2 # [ 213.402635] nix-daemon[870]: accepted connection from pid 2018, user nix-grpc-daemon (trusted) worker2 # [ 213.413728] nix-daemon[870]: reaped child process 2021, status = succeeded lb # [ 213.433787] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 213.432719] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.783Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 213.434272] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.784Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)" lb # [ 213.480214] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls lb # [ 213.484994] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst lb # [ 213.491510] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign worker2 # [ 213.485168] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.835Z level=INFO msg="Uploading 1 narinfos" lb # [ 213.495483] niks3-server[1135]: 2026/09/23 05:36:25 INFO Signed narinfos id=91 count=1 lb # [ 213.505989] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv lb # [ 213.511216] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 213.515080] niks3-server[1135]: 2026/09/23 05:36:25 INFO Completed upload id=91 worker2 # [ 213.504633] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.855Z level=INFO msg="Upload complete. (216ms)" lb # [ 213.517974] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo worker2 # [ 213.510984] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=20 outputs=1 worker2 # [ 213.522565] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=0 worker2 # [ 213.526989] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 213.532081] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 213.540307] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 213.542937] nix-daemon[870]: reaped child process 2000, status = succeeded worker2 # [ 213.547764] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.898Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 213.553340] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'... worker2 # [ 213.559344] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 213.644050] nix-daemon[870]: accepted connection from pid 2027, user nix-grpc-daemon (trusted) worker2 # [ 213.657964] nix-daemon[870]: reaped child process 2030, status = succeeded lb # [ 213.673991] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 213.672219] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.022Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 213.674268] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.024Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)" lb # [ 213.707241] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign worker2 # [ 213.702962] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.053Z level=INFO msg="Uploading 1 narinfos" lb # [ 213.714549] niks3-server[1135]: 2026/09/23 05:36:25 INFO Signed narinfos id=92 count=1 lb # [ 213.720183] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls lb # [ 213.726143] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst lb # [ 213.738147] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 213.742796] niks3-server[1135]: 2026/09/23 05:36:26 INFO Completed upload id=92 worker2 # [ 213.732223] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.082Z level=INFO msg="Upload complete. (168ms)" lb # [ 213.745114] niks3-server[1135]: 2026/09/23 05:36:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo worker2 # [ 213.735773] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 213.740793] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 worker2 # [ 213.752105] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) worker2 # [ 213.755622] nix-daemon[870]: reaped child process 2025, status = succeeded worker2 # [ 213.760724] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted) client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051... client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 214.575209] nix-daemon[870]: reaped child process 2034, status = succeeded worker2 # [ 214.650121] nix-daemon[870]: accepted connection from pid 2050, user nix-grpc-daemon (trusted) worker2 # [ 214.662406] nix-daemon[870]: reaped child process 2053, status = succeeded lb # [ 214.683421] niks3-server[1135]: 2026/09/23 05:36:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 214.680725] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.031Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 214.682383] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.032Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)" lb # [ 214.722222] niks3-server[1135]: 2026/09/23 05:36:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign worker2 # [ 214.718112] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.068Z level=INFO msg="Uploading 1 narinfos" lb # [ 214.728324] niks3-server[1135]: 2026/09/23 05:36:26 INFO Signed narinfos id=93 count=1 lb # [ 214.739618] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls lb # [ 214.744430] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv lb # [ 214.756348] niks3-server[1135]: 2026/09/23 05:36:27 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 214.762769] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst lb # [ 214.764723] niks3-server[1135]: 2026/09/23 05:36:27 INFO Completed upload id=93 worker2 # [ 214.754362] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.104Z level=INFO msg="Upload complete. (187ms)" lb # [ 214.767470] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo worker2 # [ 214.760809] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=21 outputs=1 worker2 # [ 214.775678] nix-daemon[870]: reaped child process 2032, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.93 seconds) client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 216.099157] nix-grpc-daemon[1731]: ts=2026-09-23T05:36:28Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.38 seconds) (finished: run the VM test script, in 217.07 seconds) test script finished in 217.25s cleanup kill QemuMachine (pid 50) client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 49) lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.10 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh