vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #468
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13lb: QEMU running (pid 50)14client # Formatting '/build/vm-state-client/tmp.16aZ17dVGp', fmt=raw size=107374182415client: QEMU running (pid 49)16client # mke2fs 1.47.4 (6-Mar-2025)17lb # Disk image does not exist, creating the virtualisation disk image...18client # Discarding device blocks: 0/262144 done19lb # Formatting '/build/vm-state-lb/tmp.PxVZTdwSeR', fmt=raw size=107374182420client # Creating filesystem with 262144 4k blocks and 65536 inodes21lb # mke2fs 1.47.4 (6-Mar-2025)22client # Filesystem UUID: d0c5862a-4101-431f-b032-4c311f0a254a23lb # Discarding device blocks: 0/262144 done24client # Superblock backups stored on blocks:25lb # Creating filesystem with 262144 4k blocks and 65536 inodes26client # 32768, 98304, 163840, 22937627lb # Filesystem UUID: ebf465ef-d18e-4a73-8ead-ee4f8ee874b728client # 29lb # Superblock backups stored on blocks:30client # Allocating group tables: 0/8 done31lb # 32768, 98304, 163840, 22937632client # Writing inode tables: 0/8 done33lb # 34client # Creating journal (8192 blocks): done35lb # Allocating group tables: 0/8 done36worker2 # Disk image does not exist, creating the virtualisation disk image...37client # Writing superblocks and filesystem accounting information: 0/8 done38worker2: QEMU running (pid 52)39lb # Writing inode tables: 0/8 done40worker2 # Formatting '/build/vm-state-worker2/tmp.I8hYD4lyL6', fmt=raw size=107374182441lb # Creating journal (8192 blocks): done42worker2 # mke2fs 1.47.4 (6-Mar-2025)43lb # Writing superblocks and filesystem accounting information: 0/8 done44worker2 # Discarding device blocks: 0/262144 done45lb # 46worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes47lb # Virtualisation disk image created.48worker2 # Filesystem UUID: 363fbb41-1e68-4c15-8c4a-17bba5411e6c49lb # Starting virtiofs daemons...50worker2 # Superblock backups stored on blocks:51lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)52worker2 # 32768, 98304, 163840, 22937653lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether54worker2 # 55lb # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...56worker2 # Allocating group tables: 0/8 done57lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)58worker2 # Writing inode tables: 0/8 done59lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether60worker2 # Creating journal (8192 blocks): done61lb # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...62worker2 # Writing superblocks and filesystem accounting information: 0/8 done63lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64worker2 # 65lb # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether66worker2 # Virtualisation disk image created.67lb # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...68worker2 # Starting virtiofs daemons...69lb # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests70worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71lb # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests72worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73lb # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests74worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...75client # 76worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77client # Virtualisation disk image created.78worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # Starting virtiofs daemons...80worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...81client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker2 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85client # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...87client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests89client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests91client # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker2 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests93client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95client # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...96client # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests97client # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests99worker1: QEMU running (pid 51)100worker1 # Disk image does not exist, creating the virtualisation disk image...101(finished: start all VMs, in 0.25 seconds)102worker1 # Formatting '/build/vm-state-worker1/tmp.fahzVy2rK0', fmt=raw size=1073741824103client: waiting for unit network-addresses-eth1.service104worker1 # mke2fs 1.47.4 (6-Mar-2025)105client: waiting for the VM to finish booting106worker1 # Discarding device blocks: 0/262144 done107worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes108worker1 # Filesystem UUID: 49303707-863a-4bb7-a573-4964e3047260109worker1 # Superblock backups stored on blocks:110worker1 # 32768, 98304, 163840, 229376111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-21T09:50:47Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-21T09:50:47Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139client # 140client # 141client # 142client # 143client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144worker2 # 145lb # 146worker2 # 147lb # 148worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00149lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00150client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...151client # 152client # 153worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...154worker2 # 155worker2 # 156worker2 # 157worker2 # 158lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...159lb # 160lb # 161lb # 162lb # 163worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00164lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...167worker2 # 168lb # 169worker2 # 170lb # 171worker2 # Booting from ROM...172worker1 # 173worker1 # 174worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00175worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...176worker1 # 177worker1 # 178worker1 # 179worker1 # 180worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00181worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...182worker1 # 183worker1 # 184client # Booting from ROM...185lb # Booting from ROM...186worker1 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test/init regInfo=/nix/store/ni4rfn88cjlndgpninvy69i2kzy5cf9k-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026194client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195worker2 # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026196client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved200lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h7lcrbggbxkddv6z1942qgq2abqdngay-nixos-system-lb-test/init regInfo=/nix/store/dvhl2bbg4bznz7v33ag7p79f03hl4w8z-closure-info/registration console=ttyS0,115200n8 console=tty0201worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5xxqilbz2yan1rkvdhzkgsh6hd6vlx8m-nixos-system-worker2-test/init regInfo=/nix/store/7cd7l6d238csvdfqwa8mh54lm0c675q9-closure-info/registration console=ttyS0,115200n8 console=tty0202lb # [ 0.000000] BIOS-provided physical RAM map:203client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved204worker2 # [ 0.000000] BIOS-provided physical RAM map:205lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable206client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved207worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable208client # [ 0.000000] NX (Execute Disable) protection: active209lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved210worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved211client # [ 0.000000] APIC: Static calls initialized212client # [ 0.000000] SMBIOS 2.8 present.213lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved214worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved215lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable216worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable217client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014218client # [ 0.000000] DMI: Memory slots populated: 1/1219lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved220worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved221client # [ 0.000000] Hypervisor detected: KVM222lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved223worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved224client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000225lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved226client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00227worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved228client # [ 0.000000] kvm-clock: using sched offset of 542565297 cycles229lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved230worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved231lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved232worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved233client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns234lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved235client # [ 0.000005] tsc: Detected 2400.012 MHz processor236worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved237lb # [ 0.000000] NX (Execute Disable) protection: active238worker2 # [ 0.000000] NX (Execute Disable) protection: active239client # [ 0.000817] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000240lb # [ 0.000000] APIC: Static calls initialized241worker2 # [ 0.000000] APIC: Static calls initialized242lb # [ 0.000000] SMBIOS 2.8 present.243worker2 # [ 0.000000] SMBIOS 2.8 present.244client # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs245client # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT246lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014247worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014248lb # [ 0.000000] DMI: Memory slots populated: 1/1249client # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]250worker2 # [ 0.000000] DMI: Memory slots populated: 1/1251lb # [ 0.000000] Hypervisor detected: KVM252worker2 # [ 0.000000] Hypervisor detected: KVM253client # [ 0.002744] Using GB pages for direct mapping254client # [ 0.002870] RAMDISK: [mem 0x5e370000-0x5ffcffff]255worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000256lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000257lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00258worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259client # [ 0.002878] ACPI: Early table checksum verification disabled260worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026261lb # [ 0.000000] kvm-clock: using sched offset of 551502394 cycles262worker2 # [ 0.000000] kvm-clock: using sched offset of 542802506 cycles263client # [ 0.002881] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )264client # [ 0.002884] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns266lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns267lb # [ 0.000005] tsc: Detected 2400.012 MHz processor268worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor269client # [ 0.002888] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)270lb # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000271worker2 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000272client # [ 0.002894] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker2 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs274worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pwycv9j24ld0va1bn93qyiiqiyd01wl0-nixos-system-worker1-test/init regInfo=/nix/store/zjqmg2m2j0lmvlqwrmhp3sy46wn8yhql-closure-info/registration console=ttyS0,115200n8 console=tty0275client # [ 0.002896] ACPI: FACS 0x000000005FFE0000 000040276lb # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs277worker1 # [ 0.000000] BIOS-provided physical RAM map:278lb # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT279worker2 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT280client # [ 0.002898] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)281worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable282lb # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]283worker2 # [ 0.002720] found SMP MP-table at [mem 0x000f5450-0x000f545f]284lb # [ 0.002738] Using GB pages for direct mapping285worker2 # [ 0.002732] Using GB pages for direct mapping286worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved287client # [ 0.002899] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)288lb # [ 0.002824] RAMDISK: [mem 0x5e36e000-0x5ffcffff]289worker2 # [ 0.002840] RAMDISK: [mem 0x5e36f000-0x5ffcffff]290worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved291client # [ 0.002901] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)292lb # [ 0.002832] ACPI: Early table checksum verification disabled293worker2 # [ 0.002848] ACPI: Early table checksum verification disabled294worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable295worker2 # [ 0.002851] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )296lb # [ 0.002834] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )297client # [ 0.002902] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)298worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved299lb # [ 0.002838] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)300worker2 # [ 0.002855] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)301client # [ 0.002904] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]302worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved303client # [ 0.002905] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]304lb # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)305worker2 # [ 0.002858] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)306worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved307client # [ 0.002905] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]308worker2 # [ 0.002865] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)309lb # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved311worker2 # [ 0.002867] ACPI: FACS 0x000000005FFE0000 000040312lb # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040313client # [ 0.002906] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]314worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved315client # [ 0.002906] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]316worker2 # [ 0.002869] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)317lb # [ 0.002852] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)318worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved319worker1 # [ 0.000000] NX (Execute Disable) protection: active320client # [ 0.002907] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]321worker2 # [ 0.002870] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)322lb # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)323worker1 # [ 0.000000] APIC: Static calls initialized324client # [ 0.002907] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]325worker1 # [ 0.000000] SMBIOS 2.8 present.326lb # [ 0.002855] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)327worker2 # [ 0.002872] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)328client # [ 0.003134] No NUMA configuration found329client # [ 0.003135] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]330worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014331worker2 # [ 0.002873] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)332lb # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker1 # [ 0.000000] DMI: Memory slots populated: 1/1334client # [ 0.003137] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]335worker1 # [ 0.000000] Hypervisor detected: KVM336worker2 # [ 0.002874] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]337lb # [ 0.002859] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]338client # [ 0.005393] Zone ranges:339worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000340client # [ 0.005394] DMA [mem 0x0000000000001000-0x0000000000ffffff]341worker2 # [ 0.002875] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]342lb # [ 0.002860] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]343worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00344client # [ 0.005396] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]345worker2 # [ 0.002876] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]346lb # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]347client # [ 0.005397] Normal empty348worker1 # [ 0.000000] kvm-clock: using sched offset of 548693766 cycles349client # [ 0.005397] Device empty350lb # [ 0.002861] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]351worker2 # [ 0.002876] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]352client # [ 0.005398] Movable zone start for each node353client # [ 0.005399] Early memory node ranges354worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns355worker2 # [ 0.002877] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]356lb # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]357worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor358client # [ 0.005399] node 0: [mem 0x0000000000001000-0x000000000009efff]359worker2 # [ 0.002877] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]360lb # [ 0.002862] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]361worker1 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000362client # [ 0.005400] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]363worker2 # [ 0.002878] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]364lb # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]365client # [ 0.005401] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]366worker2 # [ 0.003108] No NUMA configuration found367lb # [ 0.003086] No NUMA configuration found368worker1 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs369worker1 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT370client # [ 0.005419] On node 0, zone DMA: 1 pages in unavailable ranges371worker2 # [ 0.003109] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]372lb # [ 0.003087] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]373worker1 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]374worker2 # [ 0.003111] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]375client # [ 0.005674] On node 0, zone DMA: 97 pages in unavailable ranges376lb # [ 0.003089] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]377worker2 # [ 0.005345] Zone ranges:378lb # [ 0.005315] Zone ranges:379worker1 # [ 0.002741] Using GB pages for direct mapping380client # [ 0.031678] On node 0, zone DMA32: 40 pages in unavailable ranges381worker1 # [ 0.002813] RAMDISK: [mem 0x5e370000-0x5ffcffff]382worker2 # [ 0.005346] DMA [mem 0x0000000000001000-0x0000000000ffffff]383lb # [ 0.005316] DMA [mem 0x0000000000001000-0x0000000000ffffff]384client # [ 0.032134] ACPI: PM-Timer IO Port: 0x608385worker1 # [ 0.002821] ACPI: Early table checksum verification disabled386worker2 # [ 0.005348] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]387lb # [ 0.005318] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]388client # [ 0.032146] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])389worker2 # [ 0.005349] Normal empty390lb # [ 0.005319] Normal empty391worker1 # [ 0.002823] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )392worker2 # [ 0.005349] Device empty393lb # [ 0.005320] Device empty394client # [ 0.032171] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23395lb # [ 0.005320] Movable zone start for each node396worker2 # [ 0.005350] Movable zone start for each node397worker2 # [ 0.005351] Early memory node ranges398client # [ 0.032174] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)399lb # [ 0.005321] Early memory node ranges400worker1 # [ 0.002827] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)401worker2 # [ 0.005351] node 0: [mem 0x0000000000001000-0x000000000009efff]402client # [ 0.032175] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)403lb # [ 0.005321] node 0: [mem 0x0000000000001000-0x000000000009efff]404worker1 # [ 0.002831] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)405worker2 # [ 0.005352] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]406client # [ 0.032177] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)407lb # [ 0.005322] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]408worker1 # [ 0.002838] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)409worker2 # [ 0.005353] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]410client # [ 0.032177] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)411worker1 # [ 0.002840] ACPI: FACS 0x000000005FFE0000 000040412lb # [ 0.005323] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]413worker2 # [ 0.005371] On node 0, zone DMA: 1 pages in unavailable ranges414lb # [ 0.005342] On node 0, zone DMA: 1 pages in unavailable ranges415client # [ 0.032178] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)416worker1 # [ 0.002841] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)417worker2 # [ 0.005628] On node 0, zone DMA: 97 pages in unavailable ranges418client # [ 0.032181] ACPI: Using ACPI (MADT) for SMP configuration information419lb # [ 0.005598] On node 0, zone DMA: 97 pages in unavailable ranges420worker2 # [ 0.031308] On node 0, zone DMA32: 40 pages in unavailable ranges421lb # [ 0.031488] On node 0, zone DMA32: 40 pages in unavailable ranges422worker1 # [ 0.002843] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)423client # [ 0.032182] ACPI: HPET id: 0x8086a201 base: 0xfed00000424worker2 # [ 0.031766] ACPI: PM-Timer IO Port: 0x608425lb # [ 0.031942] ACPI: PM-Timer IO Port: 0x608426client # [ 0.032186] TSC deadline timer available427worker1 # [ 0.002844] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)428worker2 # [ 0.031776] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])429lb # [ 0.031952] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])430client # [ 0.032191] CPU topo: Max. logical packages: 1431client # [ 0.032192] CPU topo: Max. logical dies: 1432worker2 # [ 0.031802] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23433worker1 # [ 0.002846] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)434lb # [ 0.031977] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23435client # [ 0.032192] CPU topo: Max. dies per package: 1436worker2 # [ 0.031804] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)437lb # [ 0.031980] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)438client # [ 0.032197] CPU topo: Max. threads per core: 1439worker1 # [ 0.002847] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]440client # [ 0.032198] CPU topo: Num. cores per package: 1441worker2 # [ 0.031806] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)442lb # [ 0.031982] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)443worker1 # [ 0.002848] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]444client # [ 0.032198] CPU topo: Num. threads per package: 1445worker2 # [ 0.031807] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)446lb # [ 0.031983] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)447client # [ 0.032199] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs448worker1 # [ 0.002849] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]449worker2 # [ 0.031808] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)450lb # [ 0.031984] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)451client # [ 0.032216] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()452worker1 # [ 0.002849] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]453worker2 # [ 0.031808] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)454lb # [ 0.031984] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)455worker1 # [ 0.002850] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]456client # [ 0.032246] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]457worker2 # [ 0.031811] ACPI: Using ACPI (MADT) for SMP configuration information458lb # [ 0.031987] ACPI: Using ACPI (MADT) for SMP configuration information459worker1 # [ 0.002850] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]460worker2 # [ 0.031812] ACPI: HPET id: 0x8086a201 base: 0xfed00000461client # [ 0.032247] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]462lb # [ 0.031988] ACPI: HPET id: 0x8086a201 base: 0xfed00000463worker2 # [ 0.031815] TSC deadline timer available464lb # [ 0.031991] TSC deadline timer available465worker1 # [ 0.002851] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]466client # [ 0.032249] [mem 0x60000000-0xafffffff] available for PCI devices467worker2 # [ 0.031820] CPU topo: Max. logical packages: 1468lb # [ 0.031995] CPU topo: Max. logical packages: 1469worker1 # [ 0.003083] No NUMA configuration found470client # [ 0.032250] Booting paravirtualized kernel on KVM471worker2 # [ 0.031820] CPU topo: Max. logical dies: 1472lb # [ 0.031996] CPU topo: Max. logical dies: 1473worker1 # [ 0.003084] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]474worker2 # [ 0.031821] CPU topo: Max. dies per package: 1475lb # [ 0.031996] CPU topo: Max. dies per package: 1476client # [ 0.032253] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns477worker1 # [ 0.003087] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]478worker2 # [ 0.031824] CPU topo: Max. threads per core: 1479lb # [ 0.031999] CPU topo: Max. threads per core: 1480worker1 # [ 0.005338] Zone ranges:481worker2 # [ 0.031825] CPU topo: Num. cores per package: 1482lb # [ 0.032000] CPU topo: Num. cores per package: 1483client # [ 0.036764] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1484worker1 # [ 0.005339] DMA [mem 0x0000000000001000-0x0000000000ffffff]485worker2 # [ 0.031825] CPU topo: Num. threads per package: 1486lb # [ 0.032000] CPU topo: Num. threads per package: 1487client # [ 0.038867] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152488worker1 # [ 0.005341] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]489worker2 # [ 0.031825] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs490lb # [ 0.032001] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs491worker1 # [ 0.005342] Normal empty492client # [ 0.038910] kvm-guest: PV spinlocks disabled, single CPU493worker1 # [ 0.005343] Device empty494worker2 # [ 0.031842] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()495lb # [ 0.032017] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()496worker1 # [ 0.005343] Movable zone start for each node497worker1 # [ 0.005344] Early memory node ranges498worker2 # [ 0.031870] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]499lb # [ 0.032046] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]500worker1 # [ 0.005344] node 0: [mem 0x0000000000001000-0x000000000009efff]501worker2 # [ 0.031872] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]502lb # [ 0.032047] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]503worker1 # [ 0.005345] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]504worker2 # [ 0.031873] [mem 0x60000000-0xafffffff] available for PCI devices505lb # [ 0.032049] [mem 0x60000000-0xafffffff] available for PCI devices506worker1 # [ 0.005346] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]507worker2 # [ 0.031874] Booting paravirtualized kernel on KVM508client # [ 0.038911] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test/init regInfo=/nix/store/ni4rfn88cjlndgpninvy69i2kzy5cf9k-closure-info/registration console=ttyS0,115200n8 console=tty0509lb # [ 0.032050] Booting paravirtualized kernel on KVM510worker1 # [ 0.005366] On node 0, zone DMA: 1 pages in unavailable ranges511worker2 # [ 0.031877] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns512worker1 # [ 0.005621] On node 0, zone DMA: 97 pages in unavailable ranges513lb # [ 0.032053] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns514client # [ 0.039001] Unknown kernel command line parameters "regInfo=/nix/store/ni4rfn88cjlndgpninvy69i2kzy5cf9k-closure-info/registration", will be passed to user space.515worker1 # [ 0.031327] On node 0, zone DMA32: 40 pages in unavailable ranges516worker2 # [ 0.036376] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1517client # [ 0.039014] random: crng init done518worker1 # [ 0.031787] ACPI: PM-Timer IO Port: 0x608519lb # [ 0.036537] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1520worker2 # [ 0.038503] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152521lb # [ 0.038646] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152522worker1 # [ 0.031797] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])523client # [ 0.039014] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes524worker2 # [ 0.038543] kvm-guest: PV spinlocks disabled, single CPU525lb # [ 0.038690] kvm-guest: PV spinlocks disabled, single CPU526worker1 # [ 0.031823] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23527client # [ 0.041257] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)528worker1 # [ 0.031826] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)529client # [ 0.042325] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)530client # [ 0.042356] Fallback order for Node 0: 0531worker1 # [ 0.031827] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)532client # [ 0.042359] Built 1 zonelists, mobility grouping on. Total pages: 393078533worker1 # [ 0.031828] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)534client # [ 0.042360] Policy zone: DMA32535worker1 # [ 0.031830] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)536worker2 # [ 0.038545] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5xxqilbz2yan1rkvdhzkgsh6hd6vlx8m-nixos-system-worker2-test/init regInfo=/nix/store/7cd7l6d238csvdfqwa8mh54lm0c675q9-closure-info/registration console=ttyS0,115200n8 console=tty0537lb # [ 0.038691] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h7lcrbggbxkddv6z1942qgq2abqdngay-nixos-system-lb-test/init regInfo=/nix/store/dvhl2bbg4bznz7v33ag7p79f03hl4w8z-closure-info/registration console=ttyS0,115200n8 console=tty0538client # [ 0.044780] mem auto-init: stack:all(zero), heap alloc:on, heap free:off539worker1 # [ 0.031830] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)540client # [ 0.047812] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1541worker1 # [ 0.031833] ACPI: Using ACPI (MADT) for SMP configuration information542client # [ 0.051056] allocated 3145728 bytes of page_ext543worker2 # [ 0.038635] Unknown kernel command line parameters "regInfo=/nix/store/7cd7l6d238csvdfqwa8mh54lm0c675q9-closure-info/registration", will be passed to user space.544lb # [ 0.038782] Unknown kernel command line parameters "regInfo=/nix/store/dvhl2bbg4bznz7v33ag7p79f03hl4w8z-closure-info/registration", will be passed to user space.545worker1 # [ 0.031833] ACPI: HPET id: 0x8086a201 base: 0xfed00000546client # [ 0.061122] ftrace: allocating 48787 entries in 192 pages547worker2 # [ 0.038647] random: crng init done548lb # [ 0.038795] random: crng init done549worker1 # [ 0.031837] TSC deadline timer available550client # [ 0.061124] ftrace: allocated 192 pages with 2 groups551worker2 # [ 0.038648] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes552worker1 # [ 0.031841] CPU topo: Max. logical packages: 1553lb # [ 0.038795] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes554client # [ 0.061924] Dynamic Preempt: lazy555worker1 # [ 0.031841] CPU topo: Max. logical dies: 1556client # [ 0.062092] rcu: Preemptible hierarchical RCU implementation.557worker2 # [ 0.040909] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)558worker1 # [ 0.031841] CPU topo: Max. dies per package: 1559lb # [ 0.040855] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)560client # [ 0.062092] rcu: RCU event tracing is enabled.561worker1 # [ 0.031845] CPU topo: Max. threads per core: 1562worker2 # [ 0.041962] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)563lb # [ 0.042071] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)564client # [ 0.062093] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.565worker1 # [ 0.031845] CPU topo: Num. cores per package: 1566worker2 # [ 0.041991] Fallback order for Node 0: 0567lb # [ 0.042101] Fallback order for Node 0: 0568client # [ 0.062094] Trampoline variant of Tasks RCU enabled.569worker1 # [ 0.031846] CPU topo: Num. threads per package: 1570worker2 # [ 0.041995] Built 1 zonelists, mobility grouping on. Total pages: 393078571lb # [ 0.042104] Built 1 zonelists, mobility grouping on. Total pages: 393078572client # [ 0.062095] Rude variant of Tasks RCU enabled.573worker2 # [ 0.041996] Policy zone: DMA32574worker1 # [ 0.031846] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs575lb # [ 0.042105] Policy zone: DMA32576client # [ 0.062095] Tracing variant of Tasks RCU enabled.577worker2 # [ 0.044413] mem auto-init: stack:all(zero), heap alloc:on, heap free:off578worker1 # [ 0.031870] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()579lb # [ 0.044530] mem auto-init: stack:all(zero), heap alloc:on, heap free:off580client # [ 0.062096] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.581worker2 # [ 0.047410] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1582lb # [ 0.047509] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1583worker1 # [ 0.031900] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]584worker2 # [ 0.050634] allocated 3145728 bytes of page_ext585client # [ 0.062096] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1586lb # [ 0.050746] allocated 3145728 bytes of page_ext587worker1 # [ 0.031902] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]588worker2 # [ 0.060449] ftrace: allocating 48787 entries in 192 pages589lb # [ 0.060499] ftrace: allocating 48787 entries in 192 pages590client # [ 0.062111] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker1 # [ 0.031903] [mem 0x60000000-0xafffffff] available for PCI devices592worker2 # [ 0.060451] ftrace: allocated 192 pages with 2 groups593lb # [ 0.060501] ftrace: allocated 192 pages with 2 groups594worker2 # [ 0.061400] Dynamic Preempt: lazy595worker1 # [ 0.031904] Booting paravirtualized kernel on KVM596lb # [ 0.061487] Dynamic Preempt: lazy597client # [ 0.062113] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.598worker2 # [ 0.061515] rcu: Preemptible hierarchical RCU implementation.599lb # [ 0.061637] rcu: Preemptible hierarchical RCU implementation.600worker2 # [ 0.061516] rcu: RCU event tracing is enabled.601worker1 # [ 0.031907] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns602lb # [ 0.061637] rcu: RCU event tracing is enabled.603client # [ 0.062114] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604worker2 # [ 0.061516] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.605lb # [ 0.061638] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.606worker1 # [ 0.036375] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1607client # [ 0.066448] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16608worker2 # [ 0.061518] Trampoline variant of Tasks RCU enabled.609lb # [ 0.061639] Trampoline variant of Tasks RCU enabled.610worker1 # [ 0.038589] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152611worker2 # [ 0.061518] Rude variant of Tasks RCU enabled.612client # [ 0.066709] rcu: srcu_init: Setting srcu_struct sizes based on contention.613lb # [ 0.061639] Rude variant of Tasks RCU enabled.614worker1 # [ 0.038632] kvm-guest: PV spinlocks disabled, single CPU615worker2 # [ 0.061518] Tracing variant of Tasks RCU enabled.616lb # [ 0.061640] Tracing variant of Tasks RCU enabled.617client # [ 0.066715] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns618worker2 # [ 0.061519] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.619lb # [ 0.061640] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.620worker2 # [ 0.061520] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1621client # [ 0.066815] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)622lb # [ 0.061641] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1623client # [ 0.070288] Console: colour VGA+ 80x25624client # [ 0.070298] printk: legacy console [tty0] enabled625worker2 # [ 0.061576] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.626lb # [ 0.061655] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627client # [ 0.099681] printk: legacy console [ttyS0] enabled628worker1 # [ 0.038633] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pwycv9j24ld0va1bn93qyiiqiyd01wl0-nixos-system-worker1-test/init regInfo=/nix/store/zjqmg2m2j0lmvlqwrmhp3sy46wn8yhql-closure-info/registration console=ttyS0,115200n8 console=tty0629worker2 # [ 0.061578] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.630lb # [ 0.061656] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631client # [ 0.196412] ACPI: Core revision 20250807632worker2 # [ 0.061579] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.633lb # [ 0.061657] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634worker1 # [ 0.038723] Unknown kernel command line parameters "regInfo=/nix/store/zjqmg2m2j0lmvlqwrmhp3sy46wn8yhql-closure-info/registration", will be passed to user space.635worker2 # [ 0.065892] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16636worker1 # [ 0.038736] random: crng init done637client # [ 0.197324] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns638lb # [ 0.065965] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16639worker2 # [ 0.066175] rcu: srcu_init: Setting srcu_struct sizes based on contention.640lb # [ 0.066236] rcu: srcu_init: Setting srcu_struct sizes based on contention.641client # [ 0.198830] APIC: Switch to symmetric I/O mode setup642worker1 # [ 0.038737] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes643client # [ 0.199765] x2apic enabled644worker2 # [ 0.066183] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns645worker1 # [ 0.040765] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)646lb # [ 0.066243] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns647client # [ 0.200497] APIC: Switched APIC routing to: physical x2apic648worker1 # [ 0.041772] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)649worker2 # [ 0.066282] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)650worker1 # [ 0.041802] Fallback order for Node 0: 0651lb # [ 0.066341] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)652worker2 # [ 0.069811] Console: colour VGA+ 80x25653lb # [ 0.069855] Console: colour VGA+ 80x25654worker2 # [ 0.069813] printk: legacy console [tty0] enabled655worker1 # [ 0.041805] Built 1 zonelists, mobility grouping on. Total pages: 393078656lb # [ 0.069858] printk: legacy console [tty0] enabled657worker1 # [ 0.041806] Policy zone: DMA32658worker2 # [ 0.099329] printk: legacy console [ttyS0] enabled659client # [ 0.202281] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1660lb # [ 0.099283] printk: legacy console [ttyS0] enabled661worker1 # [ 0.044175] mem auto-init: stack:all(zero), heap alloc:on, heap free:off662worker2 # [ 0.194743] ACPI: Core revision 20250807663lb # [ 0.195411] ACPI: Core revision 20250807664worker1 # [ 0.047173] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1665client # [ 0.203231] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns666worker1 # [ 0.050469] allocated 3145728 bytes of page_ext667worker2 # [ 0.195590] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns668worker1 # [ 0.060097] ftrace: allocating 48787 entries in 192 pages669lb # [ 0.196285] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns670client # [ 0.204888] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)671worker1 # [ 0.060099] ftrace: allocated 192 pages with 2 groups672worker2 # [ 0.197159] APIC: Switch to symmetric I/O mode setup673worker1 # [ 0.060890] Dynamic Preempt: lazy674lb # [ 0.197784] APIC: Switch to symmetric I/O mode setup675worker2 # [ 0.198128] x2apic enabled676client # [ 0.205971] x86/cpu: User Mode Instruction Prevention (UMIP) activated677worker1 # [ 0.061015] rcu: Preemptible hierarchical RCU implementation.678lb # [ 0.198784] x2apic enabled679worker1 # [ 0.061016] rcu: RCU event tracing is enabled.680worker2 # [ 0.198895] APIC: Switched APIC routing to: physical x2apic681client # [ 0.207014] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127682worker1 # [ 0.061017] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.683lb # [ 0.199571] APIC: Switched APIC routing to: physical x2apic684worker1 # [ 0.061018] Trampoline variant of Tasks RCU enabled.685client # [ 0.207886] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0686worker1 # [ 0.061018] Rude variant of Tasks RCU enabled.687worker2 # [ 0.200709] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1688worker1 # [ 0.061019] Tracing variant of Tasks RCU enabled.689lb # [ 0.201421] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1690client # [ 0.208889] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto691worker1 # [ 0.061019] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.692worker2 # [ 0.201678] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns693worker1 # [ 0.061020] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1694lb # [ 0.202407] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns695client # [ 0.209886] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl696worker2 # [ 0.203312] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)697worker1 # [ 0.061040] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.698client # [ 0.210886] Transient Scheduler Attacks: Vulnerable: No microcode699lb # [ 0.204058] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)700worker2 # [ 0.204395] x86/cpu: User Mode Instruction Prevention (UMIP) activated701worker1 # [ 0.061041] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.702client # [ 0.211886] Spectre V2 : Mitigation: Enhanced / Automatic IBRS703lb # [ 0.205142] x86/cpu: User Mode Instruction Prevention (UMIP) activated704worker2 # [ 0.205425] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127705worker1 # [ 0.061042] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.706client # [ 0.212886] Speculative Return Stack Overflow: Mitigation: Safe RET707lb # [ 0.206205] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127708worker1 # [ 0.065381] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16709worker2 # [ 0.206309] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0710worker1 # [ 0.065649] rcu: srcu_init: Setting srcu_struct sizes based on contention.711lb # [ 0.207738] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0712worker1 # [ 0.065655] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns713worker2 # [ 0.207313] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto714lb # [ 0.208059] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto715worker1 # [ 0.065755] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)716client # [ 0.213885] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization717worker2 # [ 0.209309] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl718worker1 # [ 0.069277] Console: colour VGA+ 80x25719lb # [ 0.210055] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl720client # [ 0.215892] Spectre V2 : Enabling IBPB for BPF721worker1 # [ 0.069280] printk: legacy console [tty0] enabled722worker2 # [ 0.210309] Transient Scheduler Attacks: Vulnerable: No microcode723worker1 # [ 0.098720] printk: legacy console [ttyS0] enabled724lb # [ 0.211056] Transient Scheduler Attacks: Vulnerable: No microcode725client # [ 0.216607] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier726worker2 # [ 0.211309] Spectre V2 : Mitigation: Enhanced / Automatic IBRS727worker1 # [ 0.194177] ACPI: Core revision 20250807728lb # [ 0.212055] Spectre V2 : Mitigation: Enhanced / Automatic IBRS729client # [ 0.217887] active return thunk: srso_alias_return_thunk730worker2 # [ 0.212309] Speculative Return Stack Overflow: Mitigation: Safe RET731lb # [ 0.213055] Speculative Return Stack Overflow: Mitigation: Safe RET732worker1 # [ 0.195057] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns733client # [ 0.218906] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'734worker1 # [ 0.196613] APIC: Switch to symmetric I/O mode setup735client # [ 0.219885] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'736worker1 # [ 0.197621] x2apic enabled737client # [ 0.220885] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'738worker1 # [ 0.198411] APIC: Switched APIC routing to: physical x2apic739worker2 # [ 0.213309] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization740client # [ 0.221885] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'741lb # [ 0.214055] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization742worker2 # [ 0.215316] Spectre V2 : Enabling IBPB for BPF743client # [ 0.222885] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'744lb # [ 0.216062] Spectre V2 : Enabling IBPB for BPF745worker1 # [ 0.200238] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1746client # [ 0.223885] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'747worker2 # [ 0.216309] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier748lb # [ 0.217056] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier749worker2 # [ 0.217309] active return thunk: srso_alias_return_thunk750worker1 # [ 0.201215] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns751client # [ 0.224885] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'752lb # [ 0.218056] active return thunk: srso_alias_return_thunk753worker2 # [ 0.218330] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'754client # [ 0.225886] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256755worker1 # [ 0.202866] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)756lb # [ 0.219077] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'757client # [ 0.226886] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64758worker2 # [ 0.219309] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'759lb # [ 0.220055] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'760worker1 # [ 0.203949] x86/cpu: User Mode Instruction Prevention (UMIP) activated761client # [ 0.227885] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512762worker2 # [ 0.220309] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'763lb # [ 0.221786] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'764worker1 # [ 0.205014] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127765client # [ 0.228885] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024766worker2 # [ 0.221309] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'767lb # [ 0.222787] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'768worker1 # [ 0.206533] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0769client # [ 0.229885] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8770worker2 # [ 0.222308] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'771lb # [ 0.223055] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'772worker2 # [ 0.223309] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'773worker1 # [ 0.206867] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto774client # [ 0.230885] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.775lb # [ 0.224055] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776worker2 # [ 0.224309] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'777worker1 # [ 0.208864] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl778lb # [ 0.225055] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'779worker2 # [ 0.226218] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256780worker1 # [ 0.209863] Transient Scheduler Attacks: Vulnerable: No microcode781lb # [ 0.227055] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256782worker2 # [ 0.227049] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64783worker1 # [ 0.211601] Spectre V2 : Mitigation: Enhanced / Automatic IBRS784lb # [ 0.227998] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64785worker2 # [ 0.228025] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512786worker1 # [ 0.212541] Speculative Return Stack Overflow: Mitigation: Safe RET787lb # [ 0.229055] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512788worker2 # [ 0.229039] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024789lb # [ 0.229982] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024790worker2 # [ 0.230032] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8791lb # [ 0.230800] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8792worker2 # [ 0.231063] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.793lb # [ 0.231803] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.794worker1 # [ 0.212863] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization795worker1 # [ 0.214871] Spectre V2 : Enabling IBPB for BPF796worker1 # [ 0.215864] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier797worker1 # [ 0.216864] active return thunk: srso_alias_return_thunk798worker1 # [ 0.218572] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799worker1 # [ 0.219863] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800worker1 # [ 0.220863] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801worker1 # [ 0.221863] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802worker1 # [ 0.222863] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803worker1 # [ 0.223863] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804worker1 # [ 0.224863] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805worker1 # [ 0.225863] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806worker1 # [ 0.226863] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker1 # [ 0.227860] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker1 # [ 0.228687] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker1 # [ 0.229863] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker1 # [ 0.230863] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.265366] Freeing SMP alternatives memory: 44K812client # [ 0.265888] pid_max: default: 32768 minimum: 301813client # [ 0.266971] LSM: initializing lsm=capability,landlock,yama,bpf,ima814client # [ 0.267995] landlock: Up and running.815client # [ 0.268643] Yama: becoming mindful.816client # [ 0.269544] LSM support for eBPF active817client # [ 0.270487] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.270926] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker2 # [ 0.265096] Freeing SMP alternatives memory: 44K820worker2 # [ 0.265859] pid_max: default: 32768 minimum: 301821lb # [ 0.266392] Freeing SMP alternatives memory: 44K822lb # [ 0.267058] pid_max: default: 32768 minimum: 301823worker2 # [ 0.266397] LSM: initializing lsm=capability,landlock,yama,bpf,ima824client # [ 0.274018] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)825worker2 # [ 0.267414] landlock: Up and running.826lb # [ 0.268060] LSM: initializing lsm=capability,landlock,yama,bpf,ima827worker2 # [ 0.268310] Yama: becoming mindful.828lb # [ 0.269131] landlock: Up and running.829client # [ 0.275423] Performance Events: Fam17h+ core perfctr, AMD PMU driver.830worker2 # [ 0.269138] LSM support for eBPF active831lb # [ 0.269760] Yama: becoming mindful.832client # [ 0.275895] ... version: 2833lb # [ 0.270275] LSM support for eBPF active834worker2 # [ 0.269877] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)835client # [ 0.276604] ... bit width: 48836client # [ 0.276888] ... generic counters: 6837lb # [ 0.271196] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)838worker2 # [ 0.271204] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)839client # [ 0.277590] ... generic bitmap: 000000000000003f840client # [ 0.277888] ... fixed-purpose counters: 0841lb # [ 0.272093] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842client # [ 0.278594] ... fixed-purpose bitmap: 0000000000000000843worker2 # [ 0.273357] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)844client # [ 0.278888] ... value mask: 0000ffffffffffff845lb # [ 0.274929] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)846client # [ 0.279778] ... max period: 00007fffffffffff847worker2 # [ 0.274851] Performance Events: Fam17h+ core perfctr, AMD PMU driver.848client # [ 0.280585] ... global_ctrl mask: 000000000000003f849worker2 # [ 0.275317] ... version: 2850lb # [ 0.275540] Performance Events: Fam17h+ core perfctr, AMD PMU driver.851worker2 # [ 0.276007] ... bit width: 48852client # [ 0.280992] signal: max sigframe size: 3376853lb # [ 0.276060] ... version: 2854worker2 # [ 0.276311] ... generic counters: 6855client # [ 0.281768] rcu: Hierarchical SRCU implementation.856lb # [ 0.276755] ... bit width: 48857worker2 # [ 0.277021] ... generic bitmap: 000000000000003f858client # [ 0.282492] rcu: Max phase no-delay instances is 400.859lb # [ 0.277058] ... generic counters: 6860worker2 # [ 0.277311] ... fixed-purpose counters: 0861lb # [ 0.277790] ... generic bitmap: 000000000000003f862worker2 # [ 0.278011] ... fixed-purpose bitmap: 0000000000000000863lb # [ 0.278102] ... fixed-purpose counters: 0864worker2 # [ 0.278311] ... value mask: 0000ffffffffffff865lb # [ 0.278844] ... fixed-purpose bitmap: 0000000000000000866worker2 # [ 0.279195] ... max period: 00007fffffffffff867lb # [ 0.279058] ... value mask: 0000ffffffffffff868worker2 # [ 0.280012] ... global_ctrl mask: 000000000000003f869lb # [ 0.279961] ... max period: 00007fffffffffff870worker2 # [ 0.280416] signal: max sigframe size: 3376871client # [ 0.287447] smp: Bringing up secondary CPUs ...872worker1 # [ 0.265371] Freeing SMP alternatives memory: 44K873lb # [ 0.280761] ... global_ctrl mask: 000000000000003f874worker2 # [ 0.281198] rcu: Hierarchical SRCU implementation.875client # [ 0.287901] smp: Brought up 1 node, 1 CPU876worker1 # [ 0.265866] pid_max: default: 32768 minimum: 301877lb # [ 0.281166] signal: max sigframe size: 3376878worker2 # [ 0.281879] rcu: Max phase no-delay instances is 400.879client # [ 0.288551] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)880lb # [ 0.281937] rcu: Hierarchical SRCU implementation.881worker1 # [ 0.266956] LSM: initializing lsm=capability,landlock,yama,bpf,ima882lb # [ 0.282622] rcu: Max phase no-delay instances is 400.883worker1 # [ 0.267964] landlock: Up and running.884worker1 # [ 0.268570] Yama: becoming mindful.885client # [ 0.289048] Memory: 1454628K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)886worker1 # [ 0.269078] LSM support for eBPF active887client # [ 0.290115] devtmpfs: initialized888worker1 # [ 0.269965] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)889client # [ 0.290935] x86/mm: Memory block size: 128MB890worker2 # [ 0.286800] smp: Bringing up secondary CPUs ...891worker1 # [ 0.270899] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)892worker2 # [ 0.287325] smp: Brought up 1 node, 1 CPU893lb # [ 0.287552] smp: Bringing up secondary CPUs ...894client # [ 0.292725] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)895worker2 # [ 0.288006] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)896lb # [ 0.288071] smp: Brought up 1 node, 1 CPU897worker1 # [ 0.273764] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)898client # [ 0.293793] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).899lb # [ 0.288728] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)900client # [ 0.294968] pinctrl core: initialized pinctrl subsystem901worker2 # [ 0.288440] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)902worker1 # [ 0.274369] Performance Events: Fam17h+ core perfctr, AMD PMU driver.903worker2 # [ 0.289534] devtmpfs: initialized904client # [ 0.296048] PM: RTC time: 09:50:47, date: 2026-09-21905worker1 # [ 0.274868] ... version: 2906lb # [ 0.289284] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)907worker2 # [ 0.290330] x86/mm: Memory block size: 128MB908worker1 # [ 0.275544] ... bit width: 48909lb # [ 0.290279] devtmpfs: initialized910worker1 # [ 0.275866] ... generic counters: 6911lb # [ 0.291026] x86/mm: Memory block size: 128MB912worker1 # [ 0.276605] ... generic bitmap: 000000000000003f913worker2 # [ 0.292074] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)914worker1 # [ 0.276911] ... fixed-purpose counters: 0915client # [ 0.299461] NET: Registered PF_NETLINK/PF_ROUTE protocol family916worker1 # [ 0.277652] ... fixed-purpose bitmap: 0000000000000000917worker2 # [ 0.293240] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).918lb # [ 0.292618] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)919worker1 # [ 0.277866] ... value mask: 0000ffffffffffff920client # [ 0.300373] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations921worker2 # [ 0.294398] pinctrl core: initialized pinctrl subsystem922lb # [ 0.293090] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).923worker1 # [ 0.278741] ... max period: 00007fffffffffff924worker2 # [ 0.295464] PM: RTC time: 09:50:47, date: 2026-09-21925client # [ 0.300909] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations926worker1 # [ 0.279550] ... global_ctrl mask: 000000000000003f927lb # [ 0.294129] pinctrl core: initialized pinctrl subsystem928worker1 # [ 0.279974] signal: max sigframe size: 3376929lb # [ 0.295235] PM: RTC time: 09:50:47, date: 2026-09-21930client # [ 0.302160] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations931worker1 # [ 0.280759] rcu: Hierarchical SRCU implementation.932client # [ 0.302900] audit: initializing netlink subsys (disabled)933worker1 # [ 0.281448] rcu: Max phase no-delay instances is 400.934worker2 # [ 0.298824] NET: Registered PF_NETLINK/PF_ROUTE protocol family935client # [ 0.304061] thermal_sys: Registered thermal governor 'fair_share'936client # [ 0.304064] thermal_sys: Registered thermal governor 'bang_bang'937worker2 # [ 0.299793] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations938lb # [ 0.298688] NET: Registered PF_NETLINK/PF_ROUTE protocol family939client # [ 0.304889] thermal_sys: Registered thermal governor 'step_wise'940worker2 # [ 0.300332] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations941client # [ 0.305831] thermal_sys: Registered thermal governor 'user_space'942lb # [ 0.299541] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations943worker2 # [ 0.301581] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations944client # [ 0.306613] audit: type=2000 audit(1789984248.663:1): state=initialized audit_enabled=0 res=1945lb # [ 0.300080] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations946worker1 # [ 0.286376] smp: Bringing up secondary CPUs ...947worker2 # [ 0.302320] audit: initializing netlink subsys (disabled)948client # [ 0.307891] thermal_sys: Registered thermal governor 'power_allocator'949worker1 # [ 0.286880] smp: Brought up 1 node, 1 CPU950lb # [ 0.301319] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations951client # [ 0.307914] cpuidle: using governor menu952worker2 # [ 0.303453] thermal_sys: Registered thermal governor 'fair_share'953worker1 # [ 0.287513] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)954lb # [ 0.302067] audit: initializing netlink subsys (disabled)955worker2 # [ 0.303455] thermal_sys: Registered thermal governor 'bang_bang'956client # [ 0.310090] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5957worker2 # [ 0.304312] thermal_sys: Registered thermal governor 'step_wise'958lb # [ 0.303060] audit: type=2000 audit(1789984248.671:1): state=initialized audit_enabled=0 res=1959worker1 # [ 0.288020] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)960worker2 # [ 0.305240] thermal_sys: Registered thermal governor 'user_space'961client # [ 0.311128] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]962lb # [ 0.304220] thermal_sys: Registered thermal governor 'fair_share'963worker1 # [ 0.289114] devtmpfs: initialized964client # [ 0.311892] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry965lb # [ 0.304222] thermal_sys: Registered thermal governor 'bang_bang'966worker1 # [ 0.289934] x86/mm: Memory block size: 128MB967worker2 # [ 0.306035] audit: type=2000 audit(1789984248.672:1): state=initialized audit_enabled=0 res=1968client # [ 0.312900] PCI: Using configuration type 1 for base access969lb # [ 0.305059] thermal_sys: Registered thermal governor 'step_wise'970worker2 # [ 0.307315] thermal_sys: Registered thermal governor 'power_allocator'971worker2 # [ 0.307328] cpuidle: using governor menu972lb # [ 0.305993] thermal_sys: Registered thermal governor 'user_space'973worker1 # [ 0.291698] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)974client # [ 0.313989] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.975lb # [ 0.306772] thermal_sys: Registered thermal governor 'power_allocator'976worker2 # [ 0.309501] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5977lb # [ 0.307071] cpuidle: using governor menu978worker1 # [ 0.292737] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).979worker1 # [ 0.293957] pinctrl core: initialized pinctrl subsystem980worker2 # [ 0.310558] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]981lb # [ 0.309284] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5982worker1 # [ 0.295077] PM: RTC time: 09:50:47, date: 2026-09-21983worker2 # [ 0.311314] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry984lb # [ 0.310291] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]985worker2 # [ 0.312323] PCI: Using configuration type 1 for base access986client # [ 0.319118] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages987lb # [ 0.311061] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry988worker2 # [ 0.313402] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.989client # [ 0.319889] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page990lb # [ 0.312070] PCI: Using configuration type 1 for base access991worker1 # [ 0.298641] NET: Registered PF_NETLINK/PF_ROUTE protocol family992lb # [ 0.313158] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.993worker1 # [ 0.300117] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations994worker1 # [ 0.300886] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations995client # [ 0.324889] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages996worker2 # [ 0.318575] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages997worker1 # [ 0.302125] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations998client # [ 0.325888] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page999worker2 # [ 0.319312] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1000worker1 # [ 0.302875] audit: initializing netlink subsys (disabled)1001worker1 # [ 0.303868] audit: type=2000 audit(1789984248.691:1): state=initialized audit_enabled=0 res=11002worker1 # [ 0.304978] thermal_sys: Registered thermal governor 'fair_share'1003lb # [ 0.320071] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1004worker1 # [ 0.304980] thermal_sys: Registered thermal governor 'bang_bang'1005lb # [ 0.321058] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1006worker1 # [ 0.305866] thermal_sys: Registered thermal governor 'step_wise'1007worker2 # [ 0.324312] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1008worker1 # [ 0.306830] thermal_sys: Registered thermal governor 'user_space'1009worker2 # [ 0.325312] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1010worker1 # [ 0.307628] thermal_sys: Registered thermal governor 'power_allocator'1011worker1 # [ 0.307879] cpuidle: using governor menu1012worker1 # [ 0.310104] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51013lb # [ 0.326059] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1014client # [ 0.336262] ACPI: Added _OSI(Module Device)1015lb # [ 0.327060] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1016worker1 # [ 0.311089] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1017client # [ 0.336890] ACPI: Added _OSI(Processor Device)1018client # [ 0.337619] ACPI: Added _OSI(Processor Aggregator Device)1019worker1 # [ 0.311868] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1020worker1 # [ 0.312877] PCI: Using configuration type 1 for base access1021worker1 # [ 0.314008] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1022worker2 # [ 0.335719] ACPI: Added _OSI(Module Device)1023worker2 # [ 0.336313] ACPI: Added _OSI(Processor Device)1024worker2 # [ 0.337086] ACPI: Added _OSI(Processor Aggregator Device)1025lb # [ 0.338342] ACPI: Added _OSI(Module Device)1026client # [ 0.346468] ACPI: 1 ACPI AML tables successfully acquired and loaded1027worker1 # [ 0.320881] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1028lb # [ 0.339044] ACPI: Added _OSI(Processor Device)1029worker1 # [ 0.321866] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1030lb # [ 0.339599] ACPI: Added _OSI(Processor Aggregator Device)1031client # [ 0.347758] ACPI: Interpreter enabled1032client # [ 0.348360] ACPI: PM: (supports S0 S3 S4 S5)1033client # [ 0.350889] ACPI: Using IOAPIC for interrupt routing1034worker2 # [ 0.345853] ACPI: 1 ACPI AML tables successfully acquired and loaded1035worker1 # [ 0.326867] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1036client # [ 0.351768] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1037worker1 # [ 0.327867] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1038lb # [ 0.345358] ACPI: 1 ACPI AML tables successfully acquired and loaded1039worker2 # [ 0.347265] ACPI: Interpreter enabled1040worker2 # [ 0.347803] ACPI: PM: (supports S0 S3 S4 S5)1041client # [ 0.354888] PCI: Using E820 reservations for host bridge windows1042client # [ 0.356005] ACPI: Enabled 2 GPEs in block 00 to 3F1043worker2 # [ 0.350313] ACPI: Using IOAPIC for interrupt routing1044lb # [ 0.349475] ACPI: Interpreter enabled1045lb # [ 0.350073] ACPI: PM: (supports S0 S3 S4 S5)1046worker2 # [ 0.351190] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1047lb # [ 0.350779] ACPI: Using IOAPIC for interrupt routing1048worker2 # [ 0.354311] PCI: Using E820 reservations for host bridge windows1049worker2 # [ 0.355437] ACPI: Enabled 2 GPEs in block 00 to 3F1050lb # [ 0.353112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1051lb # [ 0.354059] PCI: Using E820 reservations for host bridge windows1052worker1 # [ 0.339105] ACPI: Added _OSI(Module Device)1053worker1 # [ 0.339816] ACPI: Added _OSI(Processor Device)1054client # [ 0.364654] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1055worker1 # [ 0.340392] ACPI: Added _OSI(Processor Aggregator Device)1056lb # [ 0.357180] ACPI: Enabled 2 GPEs in block 00 to 3F1057client # [ 0.365580] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1058client # [ 0.366965] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1059client # [ 0.368013] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1060client # [ 0.369244] PCI host bridge to bus 0000:001061worker2 # [ 0.364033] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1062worker1 # [ 0.346149] ACPI: 1 ACPI AML tables successfully acquired and loaded1063client # [ 0.369893] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1064client # [ 0.370888] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1065worker2 # [ 0.364857] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1066lb # [ 0.364535] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1067client # [ 0.371888] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1068worker2 # [ 0.365389] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1069client # [ 0.372888] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1070worker1 # [ 0.350274] ACPI: Interpreter enabled1071worker2 # [ 0.366433] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1072lb # [ 0.365064] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1073worker1 # [ 0.350883] ACPI: PM: (supports S0 S3 S4 S5)1074worker2 # [ 0.367658] PCI host bridge to bus 0000:001075client # [ 0.373889] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1076lb # [ 0.366136] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1077worker1 # [ 0.351559] ACPI: Using IOAPIC for interrupt routing1078worker2 # [ 0.368315] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1079client # [ 0.374888] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1080lb # [ 0.367181] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker2 # [ 0.369312] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1082client # [ 0.375888] pci_bus 0000:00: root bus resource [bus 00-ff]1083lb # [ 0.368432] PCI host bridge to bus 0000:001084worker2 # [ 0.370311] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1085lb # [ 0.369063] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1086client # [ 0.376895] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1087worker1 # [ 0.353913] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1088worker2 # [ 0.371311] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1089lb # [ 0.370059] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1090worker1 # [ 0.354866] PCI: Using E820 reservations for host bridge windows1091client # [ 0.378384] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1092worker2 # [ 0.372311] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093lb # [ 0.371058] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1094worker2 # [ 0.373311] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1095lb # [ 0.372059] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1096worker1 # [ 0.357899] ACPI: Enabled 2 GPEs in block 00 to 3F1097worker2 # [ 0.374312] pci_bus 0000:00: root bus resource [bus 00-ff]1098lb # [ 0.373059] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1099worker2 # [ 0.375291] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1100lb # [ 0.374058] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1101lb # [ 0.375062] pci_bus 0000:00: root bus resource [bus 00-ff]1102worker2 # [ 0.376807] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1103lb # [ 0.376039] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1104client # [ 0.380966] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1105lb # [ 0.377501] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1106client # [ 0.381900] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1107worker1 # [ 0.365212] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1108client # [ 0.382845] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1109worker1 # [ 0.365873] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1110client # [ 0.383881] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1111worker1 # [ 0.366955] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1112worker2 # [ 0.379368] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1113worker1 # [ 0.367990] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1114client # [ 0.385624] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115worker2 # [ 0.380324] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1116worker1 # [ 0.369213] PCI host bridge to bus 0000:001117worker2 # [ 0.381285] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1118worker1 # [ 0.369870] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1119lb # [ 0.380116] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1120client # [ 0.388930] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1121worker1 # [ 0.370866] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1122worker2 # [ 0.382294] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1123lb # [ 0.381070] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1124client # [ 0.389776] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1125worker1 # [ 0.371866] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1126lb # [ 0.382009] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1127client # [ 0.390604] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1128worker1 # [ 0.372866] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1129worker2 # [ 0.384012] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1130client # [ 0.390894] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1131lb # [ 0.383033] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1132worker1 # [ 0.373866] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1133worker1 # [ 0.374866] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1134client # [ 0.392426] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1135worker2 # [ 0.386320] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1136worker1 # [ 0.375866] pci_bus 0000:00: root bus resource [bus 00-ff]1137lb # [ 0.384862] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138worker2 # [ 0.387188] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1139worker1 # [ 0.376839] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1140worker2 # [ 0.388093] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1141client # [ 0.393896] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1142worker2 # [ 0.389172] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1143lb # [ 0.387100] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1144client # [ 0.394793] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1145worker1 # [ 0.378340] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1146lb # [ 0.387956] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1147client # [ 0.395596] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1148lb # [ 0.388832] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1149worker2 # [ 0.390646] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1150lb # [ 0.389960] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1151client # [ 0.396477] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1152worker2 # [ 0.393187] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1153lb # [ 0.391566] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1154worker2 # [ 0.393957] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1155worker2 # [ 0.394334] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1156client # [ 0.398929] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1157lb # [ 0.394108] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1158worker1 # [ 0.380917] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1159client # [ 0.399895] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1160lb # [ 0.395011] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1161worker2 # [ 0.395846] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1162worker1 # [ 0.381878] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1163client # [ 0.400844] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1164lb # [ 0.395793] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1165worker1 # [ 0.382820] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1166client # [ 0.402290] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1167lb # [ 0.396628] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1168worker1 # [ 0.383931] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1169worker2 # [ 0.398345] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1170client # [ 0.403896] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1171worker2 # [ 0.400023] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1172worker1 # [ 0.385754] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1173client # [ 0.404769] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1174worker2 # [ 0.400334] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1175lb # [ 0.399660] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1176client # [ 0.405622] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1177lb # [ 0.400065] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1178worker1 # [ 0.387903] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1179worker2 # [ 0.401901] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1180client # [ 0.406733] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1181lb # [ 0.401038] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1182worker1 # [ 0.388716] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1183worker1 # [ 0.389618] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1184worker2 # [ 0.403319] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1185lb # [ 0.402496] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186client # [ 0.408454] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187worker1 # [ 0.389872] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1188worker2 # [ 0.404159] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1189worker2 # [ 0.405066] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1190client # [ 0.409901] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1191lb # [ 0.404066] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1192worker1 # [ 0.391403] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1193client # [ 0.410874] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1194lb # [ 0.405005] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195worker2 # [ 0.406150] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1196lb # [ 0.405990] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1197client # [ 0.412307] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1198worker1 # [ 0.393873] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1199worker2 # [ 0.407844] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1200lb # [ 0.406929] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1201worker1 # [ 0.394704] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1202client # [ 0.413901] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1203worker1 # [ 0.395665] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1204worker2 # [ 0.409324] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1205lb # [ 0.408649] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1206worker2 # [ 0.410263] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1207client # [ 0.414872] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1208worker1 # [ 0.397271] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1209lb # [ 0.409889] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1210worker2 # [ 0.411819] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1211client # [ 0.416477] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1212lb # [ 0.410860] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1213worker2 # [ 0.413325] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1214client # [ 0.417901] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1215lb # [ 0.412531] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker1 # [ 0.400142] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1217client # [ 0.418904] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1218worker2 # [ 0.414277] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1219worker1 # [ 0.401536] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1220lb # [ 0.414104] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1221client # [ 0.420438] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222worker1 # [ 0.401887] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1223lb # [ 0.415081] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1224worker2 # [ 0.415853] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1225worker1 # [ 0.403434] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1226client # [ 0.421901] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1227lb # [ 0.416641] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1228worker2 # [ 0.417323] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1229worker2 # [ 0.418285] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1230client # [ 0.422856] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1231lb # [ 0.417854] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1232worker1 # [ 0.404874] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1233worker1 # [ 0.405707] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1234lb # [ 0.418807] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1235worker2 # [ 0.419739] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1236client # [ 0.424419] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1237worker1 # [ 0.406591] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1238lb # [ 0.419629] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239worker2 # [ 0.421324] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1240worker1 # [ 0.406872] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1241client # [ 0.425939] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1242lb # [ 0.421071] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1243client # [ 0.426798] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1244worker2 # [ 0.422322] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1245worker1 # [ 0.408411] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1246client # [ 0.427630] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1247lb # [ 0.422027] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1248worker2 # [ 0.423879] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1249worker1 # [ 0.409891] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1250client # [ 0.429321] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1251lb # [ 0.423601] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1252worker1 # [ 0.410871] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1253worker2 # [ 0.425328] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1254worker2 # [ 0.426210] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1255worker1 # [ 0.412420] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1256lb # [ 0.425078] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1257lb # [ 0.425948] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1258worker2 # [ 0.427056] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1259client # [ 0.431852] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1260lb # [ 0.426810] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1261worker1 # [ 0.413908] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1262client # [ 0.432531] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1263worker2 # [ 0.428694] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1264client # [ 0.432909] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1265worker1 # [ 0.414882] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1266lb # [ 0.427589] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1267client # [ 0.434474] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1268worker1 # [ 0.416447] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1269worker2 # [ 0.430352] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1270client # [ 0.435612] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1271lb # [ 0.429066] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1272worker2 # [ 0.431973] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1273lb # [ 0.430065] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1274worker1 # [ 0.417880] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1275client # [ 0.436099] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1276worker2 # [ 0.432333] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1277lb # [ 0.431030] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1278worker1 # [ 0.418841] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1279client # [ 0.437537] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1280worker2 # [ 0.433864] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1281lb # [ 0.432459] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1282worker1 # [ 0.420290] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1283worker2 # [ 0.435057] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1284client # [ 0.438078] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1285lb # [ 0.433594] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1286worker1 # [ 0.421879] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1287worker2 # [ 0.435509] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1288lb # [ 0.434252] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1289client # [ 0.439577] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1290worker2 # [ 0.436950] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1291lb # [ 0.435564] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1292worker1 # [ 0.422876] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1293lb # [ 0.436258] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1294worker1 # [ 0.424396] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1295worker2 # [ 0.437507] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1296lb # [ 0.437621] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1297worker2 # [ 0.438989] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1298worker1 # [ 0.426413] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1299worker1 # [ 0.426901] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1300worker1 # [ 0.427876] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1301worker1 # [ 0.429400] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1302client # [ 0.440111] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1303worker1 # [ 0.431884] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1304worker1 # [ 0.432753] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1305client # [ 0.442513] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1306worker1 # [ 0.433663] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1307client # [ 0.443183] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1308worker1 # [ 0.435328] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1309lb # [ 0.438313] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1310client # [ 0.444184] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1311worker2 # [ 0.439607] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1312worker1 # [ 0.436623] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1313lb # [ 0.440607] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1314worker2 # [ 0.441939] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1315client # [ 0.445428] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1316worker1 # [ 0.437092] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1317lb # [ 0.441302] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1318worker2 # [ 0.442542] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1319client # [ 0.446916] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1320client # [ 0.447764] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1321lb # [ 0.442381] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1322worker1 # [ 0.438495] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1323worker2 # [ 0.443606] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1324client # [ 0.448946] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1325worker1 # [ 0.439058] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1326lb # [ 0.443309] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1327worker2 # [ 0.444553] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1328worker1 # [ 0.440528] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1329worker2 # [ 0.446340] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1330worker2 # [ 0.447215] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1331lb # [ 0.444953] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1332client # [ 0.451564] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1333worker2 # [ 0.448412] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1334lb # [ 0.445793] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1335lb # [ 0.446410] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1336worker2 # [ 0.449977] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1337lb # [ 0.448459] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1338client # [ 0.456054] ACPI: PCI: Interrupt link LNKA configured for IRQ 101339client # [ 0.457004] ACPI: PCI: Interrupt link LNKB configured for IRQ 101340client # [ 0.457988] ACPI: PCI: Interrupt link LNKC configured for IRQ 111341client # [ 0.458988] ACPI: PCI: Interrupt link LNKD configured for IRQ 111342worker1 # [ 0.441080] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1343client # [ 0.459986] ACPI: PCI: Interrupt link LNKE configured for IRQ 101344lb # [ 0.453152] ACPI: PCI: Interrupt link LNKA configured for IRQ 101345worker2 # [ 0.455378] ACPI: PCI: Interrupt link LNKA configured for IRQ 101346client # [ 0.460983] ACPI: PCI: Interrupt link LNKF configured for IRQ 101347lb # [ 0.454167] ACPI: PCI: Interrupt link LNKB configured for IRQ 101348worker1 # [ 0.443526] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1349worker2 # [ 0.456418] ACPI: PCI: Interrupt link LNKB configured for IRQ 101350lb # [ 0.455155] ACPI: PCI: Interrupt link LNKC configured for IRQ 111351client # [ 0.461987] ACPI: PCI: Interrupt link LNKG configured for IRQ 111352worker2 # [ 0.457421] ACPI: PCI: Interrupt link LNKC configured for IRQ 111353lb # [ 0.456153] ACPI: PCI: Interrupt link LNKD configured for IRQ 111354client # [ 0.462989] ACPI: PCI: Interrupt link LNKH configured for IRQ 111355worker2 # [ 0.458427] ACPI: PCI: Interrupt link LNKD configured for IRQ 111356lb # [ 0.457154] ACPI: PCI: Interrupt link LNKE configured for IRQ 101357client # [ 0.463940] ACPI: PCI: Interrupt link GSIA configured for IRQ 161358worker2 # [ 0.459408] ACPI: PCI: Interrupt link LNKE configured for IRQ 101359lb # [ 0.458191] ACPI: PCI: Interrupt link LNKF configured for IRQ 101360client # [ 0.464904] ACPI: PCI: Interrupt link GSIB configured for IRQ 171361worker1 # [ 0.444151] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1362worker2 # [ 0.460408] ACPI: PCI: Interrupt link LNKF configured for IRQ 101363lb # [ 0.459157] ACPI: PCI: Interrupt link LNKG configured for IRQ 111364client # [ 0.465904] ACPI: PCI: Interrupt link GSIC configured for IRQ 181365worker2 # [ 0.461413] ACPI: PCI: Interrupt link LNKG configured for IRQ 111366client # [ 0.466876] ACPI: PCI: Interrupt link GSID configured for IRQ 191367lb # [ 0.460158] ACPI: PCI: Interrupt link LNKH configured for IRQ 111368worker1 # [ 0.445217] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1369client # [ 0.467636] ACPI: PCI: Interrupt link GSIE configured for IRQ 201370worker2 # [ 0.462407] ACPI: PCI: Interrupt link LNKH configured for IRQ 111371lb # [ 0.461096] ACPI: PCI: Interrupt link GSIA configured for IRQ 161372worker1 # [ 0.446144] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1373client # [ 0.467901] ACPI: PCI: Interrupt link GSIF configured for IRQ 211374worker2 # [ 0.463352] ACPI: PCI: Interrupt link GSIA configured for IRQ 161375lb # [ 0.462070] ACPI: PCI: Interrupt link GSIB configured for IRQ 171376client # [ 0.468846] ACPI: PCI: Interrupt link GSIG configured for IRQ 221377worker2 # [ 0.464326] ACPI: PCI: Interrupt link GSIB configured for IRQ 171378lb # [ 0.463032] ACPI: PCI: Interrupt link GSIC configured for IRQ 181379client # [ 0.469636] ACPI: PCI: Interrupt link GSIH configured for IRQ 231380worker2 # [ 0.465323] ACPI: PCI: Interrupt link GSIC configured for IRQ 181381lb # [ 0.463867] ACPI: PCI: Interrupt link GSID configured for IRQ 191382worker1 # [ 0.448571] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1383worker2 # [ 0.466326] ACPI: PCI: Interrupt link GSID configured for IRQ 191384lb # [ 0.464809] ACPI: PCI: Interrupt link GSIE configured for IRQ 201385worker1 # [ 0.448873] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1386client # [ 0.471003] iommu: Default domain type: Translated1387worker2 # [ 0.467290] ACPI: PCI: Interrupt link GSIE configured for IRQ 201388lb # [ 0.465071] ACPI: PCI: Interrupt link GSIF configured for IRQ 211389client # [ 0.471803] iommu: DMA domain TLB invalidation policy: lazy mode1390worker1 # [ 0.450110] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1391worker2 # [ 0.468042] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392lb # [ 0.466045] ACPI: PCI: Interrupt link GSIG configured for IRQ 221393client # [ 0.472869] ACPI: bus type USB registered1394worker2 # [ 0.468323] ACPI: PCI: Interrupt link GSIG configured for IRQ 221395lb # [ 0.466805] ACPI: PCI: Interrupt link GSIH configured for IRQ 231396client # [ 0.473442] usbcore: registered new interface driver usbfs1397worker2 # [ 0.469277] ACPI: PCI: Interrupt link GSIH configured for IRQ 231398worker1 # [ 0.451722] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1399client # [ 0.473905] usbcore: registered new interface driver hub1400lb # [ 0.468037] iommu: Default domain type: Translated1401client # [ 0.474777] usbcore: registered new device driver usb1402worker2 # [ 0.471148] iommu: Default domain type: Translated1403lb # [ 0.468668] iommu: DMA domain TLB invalidation policy: lazy mode1404worker2 # [ 0.471901] iommu: DMA domain TLB invalidation policy: lazy mode1405lb # [ 0.469324] ACPI: bus type USB registered1406client # [ 0.476376] NetLabel: Initializing1407client # [ 0.476889] NetLabel: domain hash size = 1281408worker2 # [ 0.472584] ACPI: bus type USB registered1409lb # [ 0.470092] usbcore: registered new interface driver usbfs1410client # [ 0.477630] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1411worker1 # [ 0.456015] ACPI: PCI: Interrupt link LNKA configured for IRQ 101412lb # [ 0.471012] usbcore: registered new interface driver hub1413worker2 # [ 0.473318] usbcore: registered new interface driver usbfs1414client # [ 0.477928] NetLabel: unlabeled traffic allowed by default1415lb # [ 0.471667] usbcore: registered new device driver usb1416worker2 # [ 0.474192] usbcore: registered new interface driver hub1417worker1 # [ 0.456968] ACPI: PCI: Interrupt link LNKB configured for IRQ 101418client # [ 0.478800] PCI: Using ACPI for IRQ routing1419worker2 # [ 0.474957] usbcore: registered new device driver usb1420worker1 # [ 0.457973] ACPI: PCI: Interrupt link LNKC configured for IRQ 111421lb # [ 0.472902] NetLabel: Initializing1422lb # [ 0.473528] NetLabel: domain hash size = 1281423worker1 # [ 0.458975] ACPI: PCI: Interrupt link LNKD configured for IRQ 111424worker2 # [ 0.476169] NetLabel: Initializing1425lb # [ 0.474058] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1426worker2 # [ 0.476721] NetLabel: domain hash size = 1281427worker1 # [ 0.459965] ACPI: PCI: Interrupt link LNKE configured for IRQ 101428lb # [ 0.474971] NetLabel: unlabeled traffic allowed by default1429worker2 # [ 0.477312] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1430worker1 # [ 0.460962] ACPI: PCI: Interrupt link LNKF configured for IRQ 101431lb # [ 0.475751] PCI: Using ACPI for IRQ routing1432worker2 # [ 0.478283] NetLabel: unlabeled traffic allowed by default1433worker2 # [ 0.479039] PCI: Using ACPI for IRQ routing1434worker1 # [ 0.461975] ACPI: PCI: Interrupt link LNKG configured for IRQ 111435worker1 # [ 0.462974] ACPI: PCI: Interrupt link LNKH configured for IRQ 111436worker1 # [ 0.463906] ACPI: PCI: Interrupt link GSIA configured for IRQ 161437worker1 # [ 0.464880] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438worker1 # [ 0.465878] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439worker1 # [ 0.466879] ACPI: PCI: Interrupt link GSID configured for IRQ 191440worker1 # [ 0.467881] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441worker1 # [ 0.468879] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442worker1 # [ 0.469887] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443worker1 # [ 0.470878] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker1 # [ 0.472940] iommu: Default domain type: Translated1445worker1 # [ 0.473827] iommu: DMA domain TLB invalidation policy: lazy mode1446worker1 # [ 0.474933] ACPI: bus type USB registered1447worker1 # [ 0.475777] usbcore: registered new interface driver usbfs1448worker1 # [ 0.476617] usbcore: registered new interface driver hub1449worker1 # [ 0.476875] usbcore: registered new device driver usb1450worker1 # [ 0.478637] NetLabel: Initializing1451worker1 # [ 0.478872] NetLabel: domain hash size = 1281452worker1 # [ 0.479663] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker1 # [ 0.480669] NetLabel: unlabeled traffic allowed by default1454worker1 # [ 0.481625] PCI: Using ACPI for IRQ routing1455client # [ 0.522521] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.522884] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.522884] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.522894] vgaarb: loaded1459client # [ 0.523655] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460lb # [ 0.519506] pci 0000:00:01.0: vgaarb: setting as boot VGA device1461client # [ 0.523890] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1462worker2 # [ 0.522166] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463lb # [ 0.520054] pci 0000:00:01.0: vgaarb: bridge control possible1464worker2 # [ 0.522308] pci 0000:00:01.0: vgaarb: bridge control possible1465lb # [ 0.520054] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1466worker2 # [ 0.522308] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1467lb # [ 0.520063] vgaarb: loaded1468worker2 # [ 0.522316] vgaarb: loaded1469lb # [ 0.520838] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470worker2 # [ 0.523051] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01471lb # [ 0.521058] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1472worker2 # [ 0.523311] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1473client # [ 0.527978] clocksource: Switched to clocksource kvm-clock1474client # [ 0.531471] VFS: Disk quotas dquot_6.6.01475lb # [ 0.524153] clocksource: Switched to clocksource kvm-clock1476client # [ 0.532194] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1477lb # [ 0.525799] VFS: Disk quotas dquot_6.6.01478client # [ 0.533600] pnp: PnP ACPI init1479worker2 # [ 0.529432] clocksource: Switched to clocksource kvm-clock1480client # [ 0.534438] ACPI: IRQ 4 override to edge(!), high(!)1481worker2 # [ 0.531037] VFS: Disk quotas dquot_6.6.01482client # [ 0.535423] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1483lb # [ 0.528272] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1484lb # [ 0.529620] pnp: PnP ACPI init1485client # [ 0.536958] pnp: PnP ACPI: found 5 devices1486lb # [ 0.530433] ACPI: IRQ 4 override to edge(!), high(!)1487worker2 # [ 0.533487] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1488worker2 # [ 0.534833] pnp: PnP ACPI init1489lb # [ 0.531429] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1490worker2 # [ 0.535668] ACPI: IRQ 4 override to edge(!), high(!)1491lb # [ 0.533021] pnp: PnP ACPI: found 5 devices1492worker2 # [ 0.536647] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1493worker2 # [ 0.538263] pnp: PnP ACPI: found 5 devices1494client # [ 0.544525] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1495worker1 # [ 0.525212] pci 0000:00:01.0: vgaarb: setting as boot VGA device1496client # [ 0.546029] clocksource: Switched to clocksource acpi_pm1497worker1 # [ 0.525862] pci 0000:00:01.0: vgaarb: bridge control possible1498client # [ 0.546983] NET: Registered PF_INET protocol family1499worker1 # [ 0.525862] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1500worker1 # [ 0.525873] vgaarb: loaded1501client # [ 0.548148] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1502lb # [ 0.540616] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503worker1 # [ 0.526651] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01504lb # [ 0.542105] clocksource: Switched to clocksource acpi_pm1505worker1 # [ 0.527458] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1506lb # [ 0.543092] NET: Registered PF_INET protocol family1507worker2 # [ 0.545806] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1508worker2 # [ 0.547319] clocksource: Switched to clocksource acpi_pm1509lb # [ 0.544244] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1510worker2 # [ 0.548276] NET: Registered PF_INET protocol family1511worker1 # [ 0.531041] clocksource: Switched to clocksource kvm-clock1512worker2 # [ 0.549445] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1513worker1 # [ 0.534776] VFS: Disk quotas dquot_6.6.01514worker1 # [ 0.535508] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1515worker1 # [ 0.536972] pnp: PnP ACPI init1516worker1 # [ 0.537823] ACPI: IRQ 4 override to edge(!), high(!)1517worker1 # [ 0.538829] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1518worker1 # [ 0.540445] pnp: PnP ACPI: found 5 devices1519client # [ 0.564582] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1520client # [ 0.566215] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521client # [ 0.567588] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522lb # [ 0.560482] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1523client # [ 0.568971] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1524lb # [ 0.562070] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1525client # [ 0.570262] TCP: Hash tables configured (established 16384 bind 16384)1526worker2 # [ 0.565734] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1527lb # [ 0.563388] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1528client # [ 0.571382] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1529worker1 # [ 0.548228] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1530worker2 # [ 0.567266] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1531lb # [ 0.564739] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1532client # [ 0.572630] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1533worker1 # [ 0.549617] clocksource: Switched to clocksource acpi_pm1534worker2 # [ 0.568500] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1535lb # [ 0.565980] TCP: Hash tables configured (established 16384 bind 16384)1536worker1 # [ 0.550595] NET: Registered PF_INET protocol family1537client # [ 0.573687] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1538worker2 # [ 0.569786] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1539lb # [ 0.567071] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1540client # [ 0.574866] NET: Registered PF_UNIX/PF_LOCAL protocol family1541worker1 # [ 0.551759] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1542worker2 # [ 0.571012] TCP: Hash tables configured (established 16384 bind 16384)1543client # [ 0.575782] NET: Registered PF_XDP protocol family1544lb # [ 0.568289] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1545client # [ 0.576578] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1546worker2 # [ 0.572057] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1547lb # [ 0.569352] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1548client # [ 0.577580] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1549worker2 # [ 0.573331] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1550lb # [ 0.570569] NET: Registered PF_UNIX/PF_LOCAL protocol family1551client # [ 0.578592] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1552lb # [ 0.571491] NET: Registered PF_XDP protocol family1553worker2 # [ 0.574398] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1554lb # [ 0.572297] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1555client # [ 0.579693] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1556worker2 # [ 0.575577] NET: Registered PF_UNIX/PF_LOCAL protocol family1557lb # [ 0.573314] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1558client # [ 0.580774] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1559worker2 # [ 0.576522] NET: Registered PF_XDP protocol family1560worker2 # [ 0.577360] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1561lb # [ 0.574333] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1562client # [ 0.581833] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563worker2 # [ 0.578350] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1564lb # [ 0.575443] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1565client # [ 0.583483] ACPI: \_SB_.GSIA: Enabled at IRQ 161566worker2 # [ 0.579364] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567lb # [ 0.576534] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568worker2 # [ 0.580475] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1569lb # [ 0.577631] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570client # [ 0.585528] ACPI: \_SB_.GSIB: Enabled at IRQ 171571worker2 # [ 0.581585] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1572lb # [ 0.579345] ACPI: \_SB_.GSIA: Enabled at IRQ 161573worker2 # [ 0.582719] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1574client # [ 0.587689] ACPI: \_SB_.GSIC: Enabled at IRQ 181575lb # [ 0.581389] ACPI: \_SB_.GSIB: Enabled at IRQ 171576worker2 # [ 0.584456] ACPI: \_SB_.GSIA: Enabled at IRQ 161577client # [ 0.589723] ACPI: \_SB_.GSID: Enabled at IRQ 191578lb # [ 0.583381] ACPI: \_SB_.GSIC: Enabled at IRQ 181579worker2 # [ 0.586767] ACPI: \_SB_.GSIB: Enabled at IRQ 171580client # [ 0.591570] PCI: CLS 0 bytes, default 641581lb # [ 0.585332] ACPI: \_SB_.GSID: Enabled at IRQ 191582worker1 # [ 0.568778] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1583worker2 # [ 0.588653] ACPI: \_SB_.GSIC: Enabled at IRQ 181584client # [ 0.592584] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1585worker1 # [ 0.570360] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1586client # [ 0.594312] Trying to unpack rootfs image as initramfs...1587lb # [ 0.587096] PCI: CLS 0 bytes, default 641588worker2 # [ 0.590550] ACPI: \_SB_.GSID: Enabled at IRQ 191589worker1 # [ 0.571707] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1590lb # [ 0.588011] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1591worker2 # [ 0.592321] PCI: CLS 0 bytes, default 641592worker1 # [ 0.573095] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1593lb # [ 0.589786] Trying to unpack rootfs image as initramfs...1594worker2 # [ 0.593250] Trying to unpack rootfs image as initramfs...1595worker1 # [ 0.574350] TCP: Hash tables configured (established 16384 bind 16384)1596worker1 # [ 0.575493] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1597worker1 # [ 0.576807] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1598worker1 # [ 0.577945] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1599worker2 # [ 0.597310] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1600worker1 # [ 0.579182] NET: Registered PF_UNIX/PF_LOCAL protocol family1601worker1 # [ 0.580167] NET: Registered PF_XDP protocol family1602worker1 # [ 0.581067] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1603worker1 # [ 0.582117] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1604worker1 # [ 0.583196] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1605worker1 # [ 0.584363] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1606worker1 # [ 0.585501] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607worker1 # [ 0.586653] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker1 # [ 0.588485] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker1 # [ 0.590666] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker1 # [ 0.592835] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker1 # [ 0.594967] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker1 # [ 0.596856] PCI: CLS 0 bytes, default 641613worker1 # [ 0.597889] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614worker1 # [ 0.599710] Trying to unpack rootfs image as initramfs...1615client # [ 0.638208] Initialise system trusted keyrings1616client # [ 0.641247] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.635242] Initialise system trusted keyrings1618worker2 # [ 0.638814] Initialise system trusted keyrings1619lb # [ 0.638880] workingset: timestamp_bits=40 max_order=19 bucket_order=01620worker2 # [ 0.642200] workingset: timestamp_bits=40 max_order=19 bucket_order=01621client # [ 0.662501] Key type asymmetric registered1622client # [ 0.665208] Asymmetric key parser 'x509' registered1623lb # [ 0.658061] Key type asymmetric registered1624client # [ 0.666145] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625worker1 # [ 0.643614] Initialise system trusted keyrings1626worker2 # [ 0.663447] Key type asymmetric registered1627lb # [ 0.660847] Asymmetric key parser 'x509' registered1628client # [ 0.669337] io scheduler mq-deadline registered1629lb # [ 0.661730] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1630client # [ 0.670136] io scheduler kyber registered1631worker1 # [ 0.646426] workingset: timestamp_bits=40 max_order=19 bucket_order=01632worker2 # [ 0.666160] Asymmetric key parser 'x509' registered1633worker2 # [ 0.667028] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1634lb # [ 0.664941] io scheduler mq-deadline registered1635lb # [ 0.665728] io scheduler kyber registered1636client # [ 0.673797] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637worker2 # [ 0.670248] io scheduler mq-deadline registered1638worker2 # [ 0.671010] io scheduler kyber registered1639client # [ 0.675098] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1640lb # [ 0.670024] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1641client # [ 0.678950] Linux agpgart interface v0.1031642lb # [ 0.671338] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1643worker2 # [ 0.674707] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1644client # [ 0.679682] ACPI: bus type drm_connector registered1645worker2 # [ 0.675971] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1646client # [ 0.682674] usbcore: registered new interface driver usbserial_generic1647lb # [ 0.676624] Linux agpgart interface v0.1031648client # [ 0.683799] usbserial: USB Serial support registered for generic1649lb # [ 0.677377] ACPI: bus type drm_connector registered1650worker2 # [ 0.681290] Linux agpgart interface v0.1031651worker2 # [ 0.682040] ACPI: bus type drm_connector registered1652client # [ 0.687209] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1653client # [ 0.687209] Please enable it if your BIOS has the CPPC option.1654lb # [ 0.681984] usbcore: registered new interface driver usbserial_generic1655client # [ 0.689508] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1656worker2 # [ 0.685417] usbcore: registered new interface driver usbserial_generic1657lb # [ 0.683067] usbserial: USB Serial support registered for generic1658worker2 # [ 0.686493] usbserial: USB Serial support registered for generic1659client # [ 0.692368] drop_monitor: Initializing network drop monitor service1660worker1 # [ 0.670321] Key type asymmetric registered1661client # [ 0.693644] NET: Registered PF_INET6 protocol family1662lb # [ 0.685844] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1663worker2 # [ 0.689160] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1664worker1 # [ 0.671042] Asymmetric key parser 'x509' registered1665lb # [ 0.685844] Please enable it if your BIOS has the CPPC option.1666worker2 # [ 0.689160] Please enable it if your BIOS has the CPPC option.1667worker1 # [ 0.671898] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1668lb # [ 0.688007] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1669worker2 # [ 0.691332] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670client # [ 0.698676] Segment Routing with IPv61671lb # [ 0.690983] drop_monitor: Initializing network drop monitor service1672worker1 # [ 0.675358] io scheduler mq-deadline registered1673client # [ 0.699343] In-situ OAM (IOAM) with IPv61674worker1 # [ 0.676119] io scheduler kyber registered1675lb # [ 0.692131] NET: Registered PF_INET6 protocol family1676worker2 # [ 0.695286] drop_monitor: Initializing network drop monitor service1677worker2 # [ 0.696447] NET: Registered PF_INET6 protocol family1678client # [ 0.701490] IPI shorthand broadcast: enabled1679lb # [ 0.695315] Segment Routing with IPv61680worker1 # [ 0.680249] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1681worker2 # [ 0.699621] Segment Routing with IPv61682lb # [ 0.697886] In-situ OAM (IOAM) with IPv61683worker1 # [ 0.681572] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1684worker2 # [ 0.701162] In-situ OAM (IOAM) with IPv61685lb # [ 0.698890] IPI shorthand broadcast: enabled1686worker2 # [ 0.703431] IPI shorthand broadcast: enabled1687client # [ 0.709739] sched_clock: Marking stable (575014514, 134169900)->(786074710, -76890296)1688worker1 # [ 0.687113] Linux agpgart interface v0.1031689worker1 # [ 0.687889] ACPI: bus type drm_connector registered1690client # [ 0.714284] registered taskstats version 11691worker1 # [ 0.690701] usbcore: registered new interface driver usbserial_generic1692lb # [ 0.707124] sched_clock: Marking stable (573014151, 133811781)->(782976175, -76150243)1693client # [ 0.715239] Loading compiled-in X.509 certificates1694worker1 # [ 0.691797] usbserial: USB Serial support registered for generic1695worker2 # [ 0.711630] sched_clock: Marking stable (578014407, 133125275)->(784160759, -73021077)1696lb # [ 0.711923] registered taskstats version 11697worker1 # [ 0.695265] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1698worker1 # [ 0.695265] Please enable it if your BIOS has the CPPC option.1699lb # [ 0.712867] Loading compiled-in X.509 certificates1700worker1 # [ 0.697530] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1701worker2 # [ 0.717323] registered taskstats version 11702worker2 # [ 0.718317] Loading compiled-in X.509 certificates1703worker1 # [ 0.702407] drop_monitor: Initializing network drop monitor service1704worker1 # [ 0.703635] NET: Registered PF_INET6 protocol family1705worker1 # [ 0.706742] Segment Routing with IPv61706worker1 # [ 0.709254] In-situ OAM (IOAM) with IPv61707client # [ 0.733197] Demotion targets for Node 0: null1708worker1 # [ 0.710419] IPI shorthand broadcast: enabled1709client # [ 0.734104] Key type .fscrypt registered1710client # [ 0.737188] Key type fscrypt-provisioning registered1711lb # [ 0.730840] Demotion targets for Node 0: null1712client # [ 0.738164] ima: No TPM chip found, activating TPM-bypass!1713lb # [ 0.731686] Key type .fscrypt registered1714client # [ 0.739091] ima: Allocated hash algorithm: sha11715worker2 # [ 0.734699] Demotion targets for Node 0: null1716client # [ 0.741225] ima: No architecture policies found1717client # [ 0.742216] PM: Magic number: 10:338:8291718lb # [ 0.734843] Key type fscrypt-provisioning registered1719worker2 # [ 0.738184] Key type .fscrypt registered1720worker1 # [ 0.718808] sched_clock: Marking stable (585015250, 133207674)->(792043990, -73821066)1721lb # [ 0.735764] ima: No TPM chip found, activating TPM-bypass!1722worker2 # [ 0.738816] Key type fscrypt-provisioning registered1723client # [ 0.743766] RAS: Correctable Errors collector initialized.1724lb # [ 0.736685] ima: Allocated hash algorithm: sha11725worker2 # [ 0.739771] ima: No TPM chip found, activating TPM-bypass!1726lb # [ 0.737472] ima: No architecture policies found1727worker2 # [ 0.742157] ima: Allocated hash algorithm: sha11728lb # [ 0.739845] PM: Magic number: 10:338:8291729worker2 # [ 0.742950] ima: No architecture policies found1730worker1 # [ 0.724385] registered taskstats version 11731worker1 # [ 0.725410] Loading compiled-in X.509 certificates1732lb # [ 0.741419] RAS: Correctable Errors collector initialized.1733worker2 # [ 0.745154] PM: Magic number: 10:338:8291734worker2 # [ 0.746729] RAS: Correctable Errors collector initialized.1735client # [ 0.752909] clk: Disabling unused clocks1736client # [ 0.756217] PM: genpd: Disabling unused power domains1737lb # [ 0.751573] clk: Disabling unused clocks1738worker2 # [ 0.755750] clk: Disabling unused clocks1739lb # [ 0.753841] PM: genpd: Disabling unused power domains1740worker2 # [ 0.760158] PM: genpd: Disabling unused power domains1741worker1 # [ 0.744346] Demotion targets for Node 0: null1742worker1 # [ 0.745306] Key type .fscrypt registered1743worker1 # [ 0.746006] Key type fscrypt-provisioning registered1744worker1 # [ 0.749352] ima: No TPM chip found, activating TPM-bypass!1745worker1 # [ 0.750323] ima: Allocated hash algorithm: sha11746worker1 # [ 0.751146] ima: No architecture policies found1747worker1 # [ 0.753468] PM: Magic number: 10:338:8291748worker1 # [ 0.757405] RAS: Correctable Errors collector initialized.1749worker1 # [ 0.766251] clk: Disabling unused clocks1750worker1 # [ 0.766980] PM: genpd: Disabling unused power domains1751lb # [ 0.908879] Freeing initrd memory: 29064K1752lb # [ 0.912084] Freeing unused decrypted memory: 2028K1753lb # [ 0.914711] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.915931] Write protecting the kernel read-only data: 32768k1755lb # [ 0.917855] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker2 # [ 0.922503] Freeing initrd memory: 29060K1757lb # [ 0.919408] Freeing unused kernel image (rodata/data gap) memory: 720K1758worker2 # [ 0.925683] Freeing unused decrypted memory: 2028K1759worker2 # [ 0.928320] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.929457] Write protecting the kernel read-only data: 32768k1761worker2 # [ 0.931346] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker2 # [ 0.932877] Freeing unused kernel image (rodata/data gap) memory: 720K1763client # [ 0.939264] Freeing initrd memory: 29056K1764client # [ 0.942740] Freeing unused decrypted memory: 2028K1765client # [ 0.945670] Freeing unused kernel image (initmem) memory: 3652K1766client # [ 0.946829] Write protecting the kernel read-only data: 32768k1767client # [ 0.948839] Freeing unused kernel image (text/rodata gap) memory: 1184K1768client # [ 0.950460] Freeing unused kernel image (rodata/data gap) memory: 720K1769worker1 # [ 0.938707] Freeing initrd memory: 29056K1770worker1 # [ 0.942120] Freeing unused decrypted memory: 2028K1771worker1 # [ 0.945090] Freeing unused kernel image (initmem) memory: 3652K1772worker1 # [ 0.946260] Write protecting the kernel read-only data: 32768k1773worker1 # [ 0.948253] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker1 # [ 0.949855] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 0.970394] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 0.971500] Run /init as init process1777worker2 # [ 0.983540] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778worker2 # [ 0.984654] Run /init as init process1779lb # [ 0.981963] systemd[1]: Inserted module 'autofs4'1780worker2 # [ 0.995056] systemd[1]: Inserted module 'autofs4'1781client # [ 1.001707] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782client # [ 1.002991] Run /init as init process1783lb # [ 1.001652] fuse: init (API version 7.45)1784lb # [ 1.009077] ACPI: \_SB_.GSIG: Enabled at IRQ 221785client # [ 1.016787] systemd[1]: Inserted module 'autofs4'1786lb # [ 1.011744] ACPI: \_SB_.GSIH: Enabled at IRQ 231787worker2 # [ 1.014707] fuse: init (API version 7.45)1788lb # [ 1.015976] ACPI: \_SB_.GSIE: Enabled at IRQ 201789worker1 # [ 1.000828] x86/mm: Checked W+X mappings: passed, no W+X pages found.1790worker1 # [ 1.001949] Run /init as init process1791lb # [ 1.018319] ACPI: \_SB_.GSIF: Enabled at IRQ 211792worker2 # [ 1.022202] ACPI: \_SB_.GSIG: Enabled at IRQ 221793worker2 # [ 1.024766] ACPI: \_SB_.GSIH: Enabled at IRQ 231794lb # [ 1.023478] virtiofs virtio5: discovered new tag: nix-store1795worker2 # [ 1.028210] ACPI: \_SB_.GSIE: Enabled at IRQ 201796lb # [ 1.025190] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1797worker2 # [ 1.030336] ACPI: \_SB_.GSIF: Enabled at IRQ 211798worker1 # [ 1.013723] systemd[1]: Inserted module 'autofs4'1799client # [ 1.038915] fuse: init (API version 7.45)1800worker2 # [ 1.035698] virtiofs virtio5: discovered new tag: nix-store1801lb # [ 1.033682] virtiofs virtio6: discovered new tag: shared1802worker2 # [ 1.037329] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1803lb # [ 1.035262] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1804lb # [ 1.038880] virtiofs virtio7: discovered new tag: xchg1805client # [ 1.046906] ACPI: \_SB_.GSIG: Enabled at IRQ 221806lb # [ 1.040346] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1807worker2 # [ 1.044818] virtiofs virtio6: discovered new tag: shared1808client # [ 1.049640] ACPI: \_SB_.GSIH: Enabled at IRQ 231809worker2 # [ 1.046363] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1810client # [ 1.052795] ACPI: \_SB_.GSIE: Enabled at IRQ 201811worker2 # [ 1.049797] virtiofs virtio7: discovered new tag: xchg1812client # [ 1.054811] ACPI: \_SB_.GSIF: Enabled at IRQ 211813worker2 # [ 1.051327] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1814worker1 # [ 1.033713] fuse: init (API version 7.45)1815client # [ 1.060371] virtiofs virtio5: discovered new tag: nix-store1816client # [ 1.061969] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1817worker1 # [ 1.041370] ACPI: \_SB_.GSIG: Enabled at IRQ 221818worker1 # [ 1.043962] ACPI: \_SB_.GSIH: Enabled at IRQ 231819client # [ 1.068715] virtiofs virtio6: discovered new tag: shared1820worker1 # [ 1.046927] ACPI: \_SB_.GSIE: Enabled at IRQ 201821client # [ 1.070636] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1822lb # [ 1.063598] systemd[1]: Successfully made /usr/ read-only.1823worker1 # [ 1.048931] ACPI: \_SB_.GSIF: Enabled at IRQ 211824client # [ 1.073889] virtiofs virtio7: discovered new tag: xchg1825client # [ 1.075381] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1826worker2 # [ 1.072926] systemd[1]: Successfully made /usr/ read-only.1827worker1 # [ 1.054243] virtiofs virtio5: discovered new tag: nix-store1828worker1 # [ 1.055833] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1829worker1 # [ 1.063696] virtiofs virtio6: discovered new tag: shared1830worker1 # [ 1.065441] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1831worker1 # [ 1.069810] virtiofs virtio7: discovered new tag: xchg1832worker1 # [ 1.071429] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833client # [ 1.098067] systemd[1]: Successfully made /usr/ read-only.1834worker1 # [ 1.094355] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.400180] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.412132] systemd[1]: Detected virtualization kvm.1837lb # [ 1.414253] systemd[1]: Detected architecture x86-64.1838lb # [ 1.416357] systemd[1]: Running in initrd.1839worker2 # [ 1.409436] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840worker2 # [ 1.421451] systemd[1]: Detected virtualization kvm.1841lb # [ 1.418743] systemd[1]: Initializing machine ID from random generator.1842worker2 # [ 1.423580] systemd[1]: Detected architecture x86-64.1843lb # [ 1.421621] systemd[1]: Hostname set to <lb>.1844worker2 # [ 1.425804] systemd[1]: Running in initrd.1845worker2 # [ 1.428347] systemd[1]: Initializing machine ID from random generator.1846worker2 # [ 1.431321] systemd[1]: Hostname set to <worker2>.1847client # [ 1.434700] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.446597] systemd[1]: Detected virtualization kvm.1849client # [ 1.448674] systemd[1]: Detected architecture x86-64.1850client # [ 1.450824] systemd[1]: Running in initrd.1851client # [ 1.453317] systemd[1]: Initializing machine ID from random generator.1852client # [ 1.456121] systemd[1]: Hostname set to <client>.1853worker1 # [ 1.431058] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker1 # [ 1.443197] systemd[1]: Detected virtualization kvm.1855worker1 # [ 1.445376] systemd[1]: Detected architecture x86-64.1856worker1 # [ 1.447483] systemd[1]: Running in initrd.1857worker1 # [ 1.449924] systemd[1]: Initializing machine ID from random generator.1858worker1 # [ 1.452831] systemd[1]: Hostname set to <worker1>.1859lb # [ 1.640695] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker2 # [ 1.656205] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.681005] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862lb # [ 1.693723] systemd[1]: Queued start job for default target Initrd Default Target.1863worker1 # [ 1.679949] systemd[1]: bpf-restrict-fs: LSM BPF program attached1864lb # [ 1.698105] systemd[1]: Created slice Slice /system/modprobe.1865lb # [ 1.699315] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866lb # [ 1.700831] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867lb # [ 1.701961] systemd[1]: Reached target Path Units.1868lb # [ 1.702923] systemd[1]: Reached target Slice Units.1869lb # [ 1.703840] systemd[1]: Reached target Swaps.1870lb # [ 1.704632] systemd[1]: Reached target Timer Units.1871lb # [ 1.705592] systemd[1]: Listening on D-Bus System Message Bus Socket.1872worker2 # [ 1.709309] systemd[1]: Queued start job for default target Initrd Default Target.1873lb # [ 1.706906] systemd[1]: Listening on Journal Socket (/dev/log).1874lb # [ 1.708074] systemd[1]: Listening on Journal Sockets.1875lb # [ 1.709121] systemd[1]: Listening on udev Control Socket.1876worker2 # [ 1.712433] systemd[1]: Created slice Slice /system/modprobe.1877lb # [ 1.710193] systemd[1]: Listening on udev Kernel Socket.1878worker2 # [ 1.713617] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1879lb # [ 1.711181] systemd[1]: Reached target Socket Units.1880worker2 # [ 1.715040] systemd[1]: Expecting device /dev/disk/by-label/nixos...1881worker2 # [ 1.716175] systemd[1]: Reached target Path Units.1882lb # [ 1.712972] systemd[1]: Starting Create List of Static Device Nodes...1883worker2 # [ 1.717021] systemd[1]: Reached target Slice Units.1884worker2 # [ 1.717904] systemd[1]: Reached target Swaps.1885worker2 # [ 1.718735] systemd[1]: Reached target Timer Units.1886worker2 # [ 1.719735] systemd[1]: Listening on D-Bus System Message Bus Socket.1887lb # [ 1.716899] systemd[1]: Starting Load Kernel Module configfs...1888worker2 # [ 1.720973] systemd[1]: Listening on Journal Socket (/dev/log).1889worker2 # [ 1.722189] systemd[1]: Listening on Journal Sockets.1890worker2 # [ 1.723189] systemd[1]: Listening on udev Control Socket.1891worker2 # [ 1.724253] systemd[1]: Listening on udev Kernel Socket.1892worker2 # [ 1.725247] systemd[1]: Reached target Socket Units.1893worker2 # [ 1.726961] systemd[1]: Starting Create List of Static Device Nodes...1894client # [ 1.734335] systemd[1]: Queued start job for default target Initrd Default Target.1895worker2 # [ 1.730938] systemd[1]: Starting Load Kernel Module configfs...1896client # [ 1.738520] systemd[1]: Created slice Slice /system/modprobe.1897client # [ 1.739738] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1898client # [ 1.741197] systemd[1]: Expecting device /dev/disk/by-label/nixos...1899client # [ 1.742346] systemd[1]: Reached target Path Units.1900client # [ 1.743256] systemd[1]: Reached target Slice Units.1901client # [ 1.744195] systemd[1]: Reached target Swaps.1902lb # [ 1.725921] systemd[1]: Starting Journal Service...1903client # [ 1.745026] systemd[1]: Reached target Timer Units.1904client # [ 1.745973] systemd[1]: Listening on D-Bus System Message Bus Socket.1905client # [ 1.747265] systemd[1]: Listening on Journal Socket (/dev/log).1906client # [ 1.748456] systemd[1]: Listening on Journal Sockets.1907client # [ 1.749510] systemd[1]: Listening on udev Control Socket.1908client # [ 1.750586] systemd[1]: Listening on udev Kernel Socket.1909client # [ 1.751598] systemd[1]: Reached target Socket Units.1910client # [ 1.753463] systemd[1]: Starting Create List of Static Device Nodes...1911worker2 # [ 1.737809] systemd[1]: Starting Journal Service...1912lb # [ 1.749427] systemd[1]: Starting Load Kernel Modules...1913client # [ 1.757254] systemd[1]: Starting Load Kernel Module configfs...1914worker1 # [ 1.733839] systemd[1]: Queued start job for default target Initrd Default Target.1915worker1 # [ 1.737594] systemd[1]: Created slice Slice /system/modprobe.1916lb # [ 1.753955] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1917worker1 # [ 1.738778] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1918worker1 # [ 1.740243] systemd[1]: Expecting device /dev/disk/by-label/nixos...1919worker1 # [ 1.741395] systemd[1]: Reached target Path Units.1920worker1 # [ 1.742346] systemd[1]: Reached target Slice Units.1921worker1 # [ 1.743263] systemd[1]: Reached target Swaps.1922worker1 # [ 1.744086] systemd[1]: Reached target Timer Units.1923lb # [ 1.760602] systemd[1]: Starting Coldplug All udev Devices...1924worker1 # [ 1.745096] systemd[1]: Listening on D-Bus System Message Bus Socket.1925worker1 # [ 1.746341] systemd[1]: Listening on Journal Socket (/dev/log).1926worker2 # [ 1.765321] systemd[1]: Starting Load Kernel Modules...1927worker1 # [ 1.747518] systemd[1]: Listening on Journal Sockets.1928worker1 # [ 1.748554] systemd[1]: Listening on udev Control Socket.1929worker1 # [ 1.749622] systemd[1]: Listening on udev Kernel Socket.1930worker1 # [ 1.750582] systemd[1]: Reached target Socket Units.1931worker2 # [ 1.769236] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1932worker1 # [ 1.752455] systemd[1]: Starting Create List of Static Device Nodes...1933client # [ 1.766360] systemd[1]: Starting Journal Service...1934worker1 # [ 1.756296] systemd[1]: Starting Load Kernel Module configfs...1935worker2 # [ 1.776207] systemd[1]: Starting Coldplug All udev Devices...1936lb # [ 1.774375] systemd-journald[66]: Collecting audit messages is disabled.1937lb # [ 1.775643] systemd[1]: Finished Create List of Static Device Nodes.1938lb # [ 1.782225] systemd[1]: modprobe@configfs.service: Deactivated successfully.1939client # [ 1.792362] systemd[1]: Starting Load Kernel Modules...1940worker2 # [ 1.788813] systemd-journald[66]: Collecting audit messages is disabled.1941lb # [ 1.789201] systemd[1]: Finished Load Kernel Module configfs.1942client # [ 1.796282] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1943worker2 # [ 1.792515] systemd[1]: Finished Create List of Static Device Nodes.1944worker1 # [ 1.765870] systemd[1]: Starting Journal Service...1945worker2 # [ 1.797743] systemd[1]: modprobe@configfs.service: Deactivated successfully.1946lb # [ 1.795230] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1947client # [ 1.804257] systemd[1]: Starting Coldplug All udev Devices...1948worker2 # [ 1.803504] systemd[1]: Finished Load Kernel Module configfs.1949lb # [ 1.804605] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1950worker1 # [ 1.791401] systemd[1]: Starting Load Kernel Modules...1951client # [ 1.814746] systemd-journald[66]: Collecting audit messages is disabled.1952worker2 # [ 1.809548] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1953client # [ 1.818479] systemd[1]: Finished Create List of Static Device Nodes.1954worker1 # [ 1.795324] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1955client # [ 1.822031] systemd[1]: modprobe@configfs.service: Deactivated successfully.1956lb # [ 1.815076] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1957worker1 # [ 1.802409] systemd[1]: Starting Coldplug All udev Devices...1958worker2 # [ 1.821206] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1959client # [ 1.829251] systemd[1]: Finished Load Kernel Module configfs.1960worker2 # [ 1.825011] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1961lb # [ 1.824858] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1962client # [ 1.835658] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1963worker1 # [ 1.815007] systemd-journald[66]: Collecting audit messages is disabled.1964worker1 # [ 1.818605] systemd[1]: Finished Create List of Static Device Nodes.1965worker2 # [ 1.838318] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966worker1 # [ 1.820865] systemd[1]: modprobe@configfs.service: Deactivated successfully.1967client # [ 1.848266] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1968lb # [ 1.843366] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1969worker1 # [ 1.829650] systemd[1]: Finished Load Kernel Module configfs.1970client # [ 1.853799] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1971worker1 # [ 1.834693] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1972lb # [ 1.852024] systemd[1]: Starting Create Static Device Nodes in /dev...1973worker2 # [ 1.858687] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1974client # [ 1.865538] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1975worker1 # [ 1.847307] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1976lb # [ 1.865363] systemd[1]: Finished Load Kernel Modules.1977worker2 # [ 1.869345] systemd[1]: Starting Create Static Device Nodes in /dev...1978lb # [ 1.874056] systemd[1]: Starting Apply Kernel Variables...1979worker1 # [ 1.859666] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1980worker2 # [ 1.881214] systemd[1]: Finished Load Kernel Modules.1981client # [ 1.888274] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1982worker1 # [ 1.869278] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1983worker2 # [ 1.890357] systemd[1]: Starting Apply Kernel Variables...1984client # [ 1.896382] systemd[1]: Starting Create Static Device Nodes in /dev...1985lb # [ 1.892387] systemd[1]: Finished Create Static Device Nodes in /dev.1986lb # [ 1.897031] systemd[1]: Reached target Preparation for Local File Systems.1987lb # [ 1.764876] systemd-modules-load[67]: Inserted module 'dm_mod'1988client # [ 1.908268] systemd[1]: Finished Load Kernel Modules.1989lb # [ 1.767861] systemd-modules-load[67]: Inserted module 'virtio_balloon'1990worker1 # [ 1.885803] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1991lb # [ 1.768954] systemd-modules-load[67]: Inserted module 'virtio_gpu'1992worker2 # [ 1.908278] systemd[1]: Finished Create Static Device Nodes in /dev.1993lb # [ 1.906905] systemd[1]: Reached target Local File Systems.1994client # [ 1.915425] systemd[1]: Starting Apply Kernel Variables...1995lb # [ 1.910340] systemd[1]: Starting Rule-based Manager for Device Events and Files...1996worker2 # [ 1.780148] systemd-modules-load[68]: Inserted module 'dm_mod'1997worker2 # [ 1.914551] systemd[1]: Started Journal Service.1998worker2 # [ 1.782267] systemd-modules-load[68]: Inserted module 'virtio_balloon'1999worker1 # [ 1.896437] systemd[1]: Starting Create Static Device Nodes in /dev...2000worker2 # [ 1.783362] systemd-modules-load[68]: Inserted module 'virtio_gpu'
loading 1,976 hidden lines…
3977worker1 # [ 10.848772] dhcpcd[684]: eth0: adding default route via fe80::23978lb # [ 11.369418] dhcpcd[716]: eth0: soliciting an IPv6 router3979lb # [ 11.370814] dhcpcd[716]: eth0: Router Advertisement from fe80::23980lb # [ 11.371757] dhcpcd[716]: eth0: adding address fec0::5054:ff:fe12:3456/643981lb # [ 11.373284] dhcpcd[716]: eth0: adding route to fec0::/643982lb # [ 11.373993] dhcpcd[716]: eth0: adding default route via fe80::23983worker2 # [ 11.783921] dhcpcd[694]: eth0: soliciting an IPv6 router3984worker2 # [ 11.786949] dhcpcd[694]: eth0: Router Advertisement from fe80::23985worker2 # [ 11.789259] dhcpcd[694]: eth0: adding address fec0::5054:ff:fe12:3456/643986worker2 # [ 11.791491] dhcpcd[694]: eth0: adding route to fec0::/643987worker2 # [ 11.793390] dhcpcd[694]: eth0: adding default route via fe80::23988client # [ 14.603648] dhcpcd[562]: eth0: leased 10.0.2.15 for 86400 seconds3989client # [ 14.604620] dhcpcd[562]: eth0: adding route to 10.0.2.0/243990client # [ 14.605597] dhcpcd[562]: eth0: adding default route via 10.0.2.23991client # [ 14.666992] systemd[1]: Started DHCP Client.3992client # [ 14.668982] systemd[1]: Reached target Multi-User System.3993client # [ 14.670365] systemd[1]: Startup finished in 873ms (kernel) + 3.697s (initrd) + 10.098s (userspace) = 14.670s.3994worker2 # [ 14.789586] dhcpcd[694]: eth0: leased 10.0.2.15 for 86400 seconds3995worker2 # [ 14.791401] dhcpcd[694]: eth0: adding route to 10.0.2.0/243996worker2 # [ 14.792374] dhcpcd[694]: eth0: adding default route via 10.0.2.23997worker2 # [ 14.848739] systemd[1]: Started DHCP Client.3998worker2 # [ 14.850765] systemd[1]: Reached target Multi-User System.3999worker2 # [ 14.851761] systemd[1]: Startup finished in 854ms (kernel) + 4.734s (initrd) + 9.261s (userspace) = 14.850s.4000worker1 # [ 15.382934] dhcpcd[684]: eth0: leased 10.0.2.15 for 86400 seconds4001worker1 # [ 15.384212] dhcpcd[684]: eth0: adding route to 10.0.2.0/244002worker1 # [ 15.385737] dhcpcd[684]: eth0: adding default route via 10.0.2.24003worker1 # [ 15.447807] systemd[1]: Started DHCP Client.4004worker1 # [ 15.449869] systemd[1]: Reached target Multi-User System.4005worker1 # [ 15.450965] systemd[1]: Startup finished in 872ms (kernel) + 3.686s (initrd) + 10.890s (userspace) = 15.449s.4006lb # [ 15.882770] dhcpcd[716]: eth0: leased 10.0.2.15 for 86400 seconds4007lb # [ 15.883769] dhcpcd[716]: eth0: adding route to 10.0.2.0/244008lb # [ 15.884665] dhcpcd[716]: eth0: adding default route via 10.0.2.24009lb # [ 15.943263] systemd[1]: Started DHCP Client.4010lb # [ 15.946155] systemd[1]: Reached target Network is Online.4011lb # [ 15.949919] systemd[1]: Started Envoy reverse proxy.4012lb # [ 15.953710] systemd[1]: Started mock-oidc.service.4013lb # [ 16.000407] mock-oidc-server[915]: Mock OIDC Server running4014lb # [ 16.002185] mock-oidc-server[915]: OIDC Address: 192.168.1.2:80804015lb # [ 16.003270] mock-oidc-server[915]: Issue Address: 0.0.0.0:80814016lb # [ 16.004490] mock-oidc-server[915]: Issuer: http://192.168.1.2:8080/oidc4017lb # [ 16.005523] mock-oidc-server[915]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json4018lb # [ 16.006696] mock-oidc-server[915]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration4019lb # [ 16.008118] mock-oidc-server[915]: Issue tokens: http://0.0.0.0:8081/issue?sub=...4020lb # [ 16.298925] envoy[914]: [2026-09-21 09:51:03.668][914][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)4021lb # [ 16.301848] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:469] statically linked extensions:4022lb # [ 16.303303] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4023lb # [ 16.306228] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4024lb # [ 16.308553] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4025lb # [ 16.311152] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4026lb # [ 16.313867] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4027lb # [ 16.315510] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4028lb # [ 16.317078] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24029lb # [ 16.318588] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4030lb # [ 16.320605] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4031lb # [ 16.322963] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4032lb # [ 16.325343] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4033lb # [ 16.327249] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4034lb # [ 16.328935] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4035lb # [ 16.345716] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port4036lb # [ 16.347852] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4037lb # [ 16.349518] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default4038lb # [ 16.351331] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4039lb # [ 16.357485] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4040lb # [ 16.359821] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4041lb # [ 16.362145] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4042lb # [ 16.363835] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4043lb # [ 16.366038] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4044lb # [ 16.367922] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4045lb # [ 16.369665] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4046lb # [ 16.373671] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default4047lb # [ 16.375884] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4048lb # [ 16.377862] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4049lb # [ 16.381595] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4050lb # [ 16.383286] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4051lb # [ 16.384936] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4052lb # [ 16.388618] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4053lb # [ 16.390682] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4054lb # [ 16.393452] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4055lb # [ 16.395198] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4056lb # [ 16.397699] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4057lb # [ 16.400224] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4058lb # [ 16.402155] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4059lb # [ 16.405373] envoy[914]: [2026-09-21 09:51:03.670][914][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4060lb # [ 16.411341] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4061lb # [ 16.414832] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4062lb # [ 16.417252] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4063lb # [ 16.419516] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4064lb # [ 16.423068] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4065lb # [ 16.426085] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4066lb # [ 16.428029] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality4067lb # [ 16.435313] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4068lb # [ 16.437503] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4069lb # [ 16.444450] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor4070lb # [ 16.447245] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4071lb # [ 16.453360] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata4072lb # [ 16.455907] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4073lb # [ 16.458709] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4074lb # [ 16.463717] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4075lb # [ 16.465575] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4076lb # [ 16.467131] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4077lb # [ 16.468878] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4078lb # [ 16.470590] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4079lb # [ 16.473470] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4080lb # [ 16.476183] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4081lb # [ 16.478130] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4082lb # [ 16.484323] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4083lb # [ 16.485988] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options4084lb # [ 16.490257] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4085lb # [ 16.492251] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4086lb # [ 16.495467] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4087lb # [ 16.502100] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4088lb # [ 16.504493] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4089lb # [ 16.507590] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4090lb # [ 16.513102] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4091lb # [ 16.514987] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation4092lb # [ 16.517702] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4093lb # [ 16.519803] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4094lb # [ 16.521716] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4095lb # [ 16.523713] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4096lb # [ 16.528119] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4097lb # [ 16.533772] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84098lb # [ 16.535617] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4099lb # [ 16.538966] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4100lb # [ 16.567241] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4101lb # [ 16.569803] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4102lb # [ 16.572562] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4103lb # [ 16.584734] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24104lb # [ 16.586429] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4105lb # [ 16.588098] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4106lb # [ 16.590043] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4107lb # [ 16.592383] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4108lb # [ 16.594585] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http14109lb # [ 16.596752] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4110lb # [ 16.598759] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4111lb # [ 16.602147] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4112lb # [ 16.604631] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4113lb # [ 16.607838] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4114lb # [ 16.612503] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24115lb # [ 16.614746] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4116lb # [ 16.616968] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4117lb # [ 16.618875] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities4118lb # [ 16.620764] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4119lb # [ 16.624096] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4120lb # [ 16.626809] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap4121lb # [ 16.631295] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4122lb # [ 16.634186] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4123lb # [ 16.640588] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4124lb # [ 16.642345] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4125lb # [ 16.644476] envoy[914]: [2026-09-21 09:51:03.671][914][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4126lb # [ 16.647055] envoy[914]: [2026-09-21 09:51:03.713][914][info][main] [source/server/server.cc:531] HTTP header map info:4127lb # [ 16.648426] envoy[914]: [2026-09-21 09:51:03.715][914][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4128lb # [ 16.662689] envoy[914]: [2026-09-21 09:51:03.715][914][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4129lb # [ 16.664242] envoy[914]: [2026-09-21 09:51:03.715][914][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4130lb # [ 16.673302] envoy[914]: [2026-09-21 09:51:03.715][914][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4131lb # [ 16.678361] envoy[914]: [2026-09-21 09:51:04.048][914][info][main] [source/server/server.cc:968] runtime: {}4132lb # [ 16.681751] envoy[914]: [2026-09-21 09:51:04.051][914][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4133lb # [ 16.683637] envoy[914]: [2026-09-21 09:51:04.053][914][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014134lb # [ 16.685420] envoy[914]: [2026-09-21 09:51:04.055][914][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014135lb # [ 16.687775] envoy[914]: [2026-09-21 09:51:04.057][914][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4136lb # [ 16.689898] envoy[914]: [2026-09-21 09:51:04.059][914][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4137lb # [ 16.691866] envoy[914]: [2026-09-21 09:51:04.059][914][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4138lb # [ 16.712765] envoy[914]: [2026-09-21 09:51:04.082][914][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4139worker2 # [ 16.717690] systemd[1]: Reached target Network is Online.4140worker2 # [ 16.720963] systemd[1]: Starting scheduler-token.service...4141lb # [ 16.718462] envoy[914]: [2026-09-21 09:51:04.087][914][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4142worker1 # [ 16.701236] systemd[1]: Starting Nix worker-protocol over gRPC...4143lb # [ 16.720923] envoy[914]: [2026-09-21 09:51:04.090][914][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4144lb # [ 16.724199] envoy[914]: [2026-09-21 09:51:04.090][914][info][main] [source/server/server.cc:1077] starting main dispatch loop4145worker2 # [ 16.758861] systemd[1]: scheduler-token.service: Deactivated successfully.4146worker2 # [ 16.761066] systemd[1]: Finished scheduler-token.service.4147worker2 # [ 16.761772] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 39ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4148worker2 # [ 16.765939] systemd[1]: Starting Nix worker-protocol over gRPC...4149worker1 # [ 16.800921] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4150worker1 # [ 16.812221] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4151worker1 # [ 16.814596] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14152worker1 # [ 16.826979] nix-grpc-daemon[779]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4153worker1 # [ 16.828600] nix-grpc-daemon[779]: E0000 00:00:1789984264.831362 779 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4154worker1 # [ 16.834709] nix-grpc-daemon[779]: E0000 00:00:1789984264.839455 779 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4155worker2 # [ 16.854351] nix-grpc-daemon[785]: ts=2026-09-21T09:51:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4156worker1 # [ 16.838077] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4157worker1 # [ 16.842908] systemd[1]: Started Nix worker-protocol over gRPC.4158worker2 # [ 16.866273] nix-grpc-daemon[785]: ts=2026-09-21T09:51:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4159worker2 # [ 16.869274] nix-grpc-daemon[785]: ts=2026-09-21T09:51:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14160worker1 # [ 16.860375] nix-grpc-daemon[779]: E0000 00:00:1789984264.864982 821 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4161worker2 # [ 16.883529] nix-grpc-daemon[785]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4162worker2 # [ 16.885184] nix-grpc-daemon[785]: E0000 00:00:1789984264.946602 785 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4163worker2 # [ 16.892044] nix-grpc-daemon[785]: E0000 00:00:1789984264.955256 785 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4164lb # [ 16.892133] envoy[914]: [2026-09-21 09:51:04.261][914][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4165worker2 # [ 16.896032] nix-grpc-daemon[785]: ts=2026-09-21T09:51:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4166lb # [ 16.894232] envoy[914]: [2026-09-21 09:51:04.261][914][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4167worker2 # [ 16.899490] systemd[1]: Started Nix worker-protocol over gRPC.4168lb # [ 16.897965] envoy[914]: [2026-09-21 09:51:04.261][914][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4169worker1 # [ 16.880198] nix-grpc-daemon[779]: E0000 00:00:1789984264.884831 821 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4170worker1 # [ 16.885059] nix-grpc-daemon[779]: E0000 00:00:1789984264.887203 821 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4171lb # [ 16.903796] envoy[914]: [2026-09-21 09:51:04.273][914][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4172worker2 # [ 16.913801] nix-grpc-daemon[785]: E0000 00:00:1789984264.976561 827 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4173worker1 # [ 16.896405] nix-grpc-daemon[779]: E0000 00:00:1789984264.901035 821 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4174worker1 # [ 16.911298] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 16.913350] nix-grpc-daemon[779]: ts=2026-09-21T09:51:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker2 # [ 16.935074] nix-grpc-daemon[785]: E0000 00:00:1789984264.998445 827 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4177worker2 # [ 16.941077] nix-grpc-daemon[785]: E0000 00:00:1789984265.001218 827 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4178worker2 # [ 16.953741] nix-grpc-daemon[785]: E0000 00:00:1789984265.017119 827 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4179worker2 # [ 16.967441] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 16.969228] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 17.114246] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514182worker1 # [ 17.116189] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker2 # [ 17.170268] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514184worker2 # [ 17.172211] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker1 # [ 17.316959] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 17.319588] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4187worker2 # [ 17.372936] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514188worker2 # [ 17.374621] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker1 # [ 17.520534] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 17.523653] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4191worker2 # [ 17.575885] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514192worker2 # [ 17.579623] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker1 # [ 17.724748] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 17.728867] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4195worker2 # [ 17.780291] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514196worker2 # [ 17.782045] nix-grpc-daemon[785]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197lb # [ 17.931111] rustfs-bucket-start[950]: mb s3://farm4198lb # [ 17.939490] systemd[1]: Finished rustfs-bucket.service.4199lb # [ 17.946947] systemd[1]: Starting niks3 server...4200worker1 # [ 17.930916] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 17.935177] nix-grpc-daemon[779]: ts=2026-09-21T09:51:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 17.984251] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 17.987457] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204lb # [ 18.088422] postgres[963]: [963] ERROR: relation "goose_db_version" does not exist at character 364205lb # [ 18.090113] postgres[963]: [963] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4206lb # [ 18.110981] niks3-server[957]: 2026/09/21 09:51:05 OK 20241026095416_initial_model.sql (12.14ms)4207lb # [ 18.114496] niks3-server[957]: 2026/09/21 09:51:05 OK 20251210153512_drop_unused_gin_index.sql (3.81ms)4208lb # [ 18.119855] niks3-server[957]: 2026/09/21 09:51:05 OK 20251218171726_add_pins.sql (5.33ms)4209lb # [ 18.124109] niks3-server[957]: 2026/09/21 09:51:05 OK 20260628120000_add_object_size_and_stats.sql (4.2ms)4210lb # [ 18.128512] niks3-server[957]: 2026/09/21 09:51:05 OK 20260905000000_add_claims.sql (4.48ms)4211lb # [ 18.132415] niks3-server[957]: 2026/09/21 09:51:05 OK 20260920000000_drop_claims.sql (2.72ms)4212lb # [ 18.133754] niks3-server[957]: 2026/09/21 09:51:05 goose: successfully migrated database to version: 202609200000004213lb # [ 18.138224] niks3-server[957]: 2026/09/21 09:51:05 OK 1_commit_pending_closure.sql (4.45ms)4214lb # [ 18.141276] niks3-server[957]: 2026/09/21 09:51:05 OK 2_object_stats_trigger.sql (2.97ms)4215lb # [ 18.142351] niks3-server[957]: 2026/09/21 09:51:05 goose: up to current file version: 24216lb # [ 18.145773] niks3-server[957]: 2026/09/21 09:51:05 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key4217worker1 # [ 18.131475] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 18.133587] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219lb # [ 18.178664] niks3-server[957]: 2026/09/21 09:51:05 INFO Read proxy enabled — serving cache objects from S34220lb # [ 18.182542] niks3-server[957]: 2026/09/21 09:51:05 INFO Using socket-activated listener address=0.0.0.0:57514221worker2 # [ 18.185349] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514222lb # [ 18.186146] niks3-server[957]: 2026/09/21 09:51:05 INFO systemd watchdog enabled interval=15s4223lb # [ 18.187302] systemd[1]: Started niks3 server.4224lb # [ 18.187900] systemd[1]: Reached target Multi-User System.4225worker2 # [ 18.189199] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226lb # [ 18.190062] niks3-server[957]: 2026/09/21 09:51:05 INFO Starting HTTP server address=0.0.0.0:57514227lb # [ 18.191795] systemd[1]: Startup finished in 841ms (kernel) + 3.633s (initrd) + 13.717s (userspace) = 18.191s.4228lb # [ 18.204854] niks3-server[957]: 2026/09/21 09:51:05 INFO lead: acquired remote=192.168.1.3:335164229worker1 # [ 18.192438] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_take_over4230lb: (finished: waiting for unit niks3.service, in 8.62 seconds)4231lb: waiting for TCP port 5751 on localhost4232lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4233lb: (finished: waiting for TCP port 5751 on localhost, in 0.04 seconds)4234lb: waiting for unit envoy.service4235lb: (finished: waiting for unit envoy.service, in 0.04 seconds)4236lb: waiting for TCP port 50051 on localhost4237lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4238lb # [ 18.343660] envoy[914]: conn peer=[::1]:55802 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24239lb: (finished: waiting for TCP port 50051 on localhost, in 0.02 seconds)4240worker1 # [ 18.333890] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 18.336283] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 18.391265] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 18.447496] systemd[1]: Starting scheduler-token.service...4244worker2 # [ 18.492250] systemd[1]: scheduler-token.service: Deactivated successfully.4245worker2 # [ 18.493879] systemd[1]: Finished scheduler-token.service.4246worker2 # [ 18.495297] systemd[1]: scheduler-token.service: Consumed 26ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4247lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4248lb: must succeed: curl -sf localhost:9901/clusters4249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4250lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.03 seconds)4251subtest: one node schedules and both builders hold a WorkerSession on it4252lb: must succeed: curl -sf localhost:9901/clusters4253worker1 # [ 18.536587] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4255worker1 # [ 18.538483] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 18.739494] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 18.742868] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 18.792505] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 18.795871] nix-grpc-daemon[785]: ts=2026-09-21T09:51:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 18.944148] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 18.950810] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:357284262worker1 # [ 18.952865] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04263worker2 # [ 18.997095] nix-grpc-daemon[785]: ts=2026-09-21T09:51:07Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 18.986088] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=oidc_jwks provider=mock keys=14265worker1 # [ 18.988377] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:357284266worker1 # [ 18.990224] nix-grpc-daemon[779]: ts=2026-09-21T09:51:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04267lb: must succeed: curl -sf localhost:9901/clusters4268lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4269worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4270worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4271lb: must succeed: curl -sf localhost:9901/clusters4272lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4273lb: must succeed: curl -sf localhost:9901/clusters4274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4275lb: must succeed: curl -sf localhost:9901/clusters4276lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4277lb: must succeed: curl -sf localhost:9901/clusters4278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4279worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4280worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.02 seconds)4281worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4282worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4283lb: must succeed: curl -sf localhost:9901/clusters4284lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4285lb: must succeed: curl -sf localhost:9901/clusters4286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4287worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4288worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4289worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4290worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4291worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4292worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4293worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4294worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4295(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.39 seconds)4296subtest: balancer auth: client cert, bearer token, nothing, unknown CN4297worker1 # [ 20.072855] nix-grpc-daemon[779]: ts=2026-09-21T09:51:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357344298worker1 # [ 20.081542] systemd[1]: Started Nix Daemon.4299worker1 # [ 20.216371] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4300worker1 # [ 20.236906] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 1/5)4301worker1 # [ 20.318256] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)4302worker1 # [ 20.475169] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 3/5)4303worker1 # [ 20.537395] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 4/5)4304worker1 # [ 20.545247] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4305worker1 # [ 20.548453] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4306worker2 # [ 20.589315] nix-grpc-daemon[785]: ts=2026-09-21T09:51:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412464307worker2 # [ 20.593420] systemd[1]: Started Nix Daemon.4308worker2 # [ 20.678216] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4309worker2 # [ 20.702554] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)4310worker2 # [ 20.743194] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 2/5)4311worker2 # [ 20.773200] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 227 ms (attempt 3/5)4312worker2 # [ 21.002339] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 497 ms (attempt 4/5)4313worker2 # [ 21.502764] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4314worker2 # [ 21.506767] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4315worker2 # [ 21.522391] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=path_info_miss path=b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test error="error: path '/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test' is required, but there is no substituter that can build it"4316worker2 # [ 21.541662] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412464317worker2 # [ 21.550263] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=path_info_miss path=b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test error="error: path '/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test' is required, but there is no substituter that can build it"4318client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4319client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds)4320client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4321client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds)4322worker2 # [ 21.787732] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=info event=oidc_jwks provider=mock keys=14323worker2 # [ 21.791385] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:412464324worker1 # [ 21.787882] nix-grpc-daemon[779]: ts=2026-09-21T09:51:09Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:357344325worker2 # [ 21.821658] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:412464326worker1 # [ 21.798989] nix-grpc-daemon[779]: ts=2026-09-21T09:51:09Z level=debug event=path_info_miss path=b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test error="error: path '/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test' is required, but there is no substituter that can build it"4327worker2 # [ 21.830426] nix-grpc-daemon[785]: ts=2026-09-21T09:51:09Z level=debug event=path_info_miss path=b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test error="error: path '/nix/store/b7nxd6x3j7rw815w3p2dgfhfplmqalr8-nixos-system-client-test' is required, but there is no substituter that can build it"4328client: must succeed: rm -r /var/lib/nix-grpc-store4329client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds)4330worker2 # [ 21.983819] nix-grpc-daemon[785]: ts=2026-09-21T09:51:10Z level=info event=denied method=QueryMissing cn=- role=none4331worker1 # [ 22.111280] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=info event=denied method=QueryMissing cn=stranger role=none4332(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.23 seconds)4333subtest: DAG build is scheduled across workers and lands in the cache4334client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24335worker1 # [ 22.256677] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357344336client # don't know how to build these paths:4337client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4338worker1 # [ 22.266061] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344339worker1 # [ 22.294644] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04340worker1 # [ 22.299599] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14341worker1 # [ 22.341869] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344342worker1 # [ 22.354439] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344343client # copying 1 paths to 'lb:50051'...4344worker1 # [ 22.367152] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344345client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4346worker1 # [ 22.391208] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4347worker1 # [ 22.422419] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=info event=niks3_push_started pid=8724348worker1 # [ 22.595832] nix-grpc-daemon[872]: time=2026-09-21T09:51:10.599Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4349worker1 # [ 22.700982] nix-daemon[857]: accepted connection from pid 885, user nix-grpc-daemon (trusted)4350worker1 # [ 22.716296] nix-daemon[857]: reaped child process 888, status = succeeded4351worker1 # [ 22.814616] nix-daemon[857]: accepted connection from pid 890, user nix-grpc-daemon (trusted)4352worker1 # [ 22.832752] nix-daemon[857]: reaped child process 893, status = succeeded4353lb # [ 22.848077] niks3-server[957]: 2026/09/21 09:51:10 INFO Received uploads request method=POST path=/api/pending_closures4354worker1 # [ 22.848541] nix-grpc-daemon[872]: time=2026-09-21T09:51:10.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4355worker1 # [ 22.850323] nix-grpc-daemon[872]: time=2026-09-21T09:51:10.854Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4356lb # [ 22.902361] niks3-server[957]: 2026/09/21 09:51:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4357lb # [ 22.922066] niks3-server[957]: 2026/09/21 09:51:10 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4358lb # [ 22.937474] niks3-server[957]: 2026/09/21 09:51:10 INFO Signed narinfos id=1 count=14359worker1 # [ 22.924478] nix-grpc-daemon[872]: time=2026-09-21T09:51:10.928Z level=INFO msg="Uploading 1 narinfos"4360lb # [ 22.946941] niks3-server[957]: 2026/09/21 09:51:10 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4361lb # [ 22.950897] niks3-server[957]: 2026/09/21 09:51:10 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4362lb # [ 22.960434] niks3-server[957]: 2026/09/21 09:51:10 INFO Completed upload id=14363worker1 # [ 22.945415] nix-grpc-daemon[872]: time=2026-09-21T09:51:10.950Z level=INFO msg="Upload complete. (345ms)"4364worker1 # [ 22.947746] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=5044365lb # [ 22.968219] niks3-server[957]: 2026/09/21 09:51:10 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4366worker1 # [ 22.951422] nix-grpc-daemon[779]: ts=2026-09-21T09:51:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344367worker1 # [ 22.961868] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4368worker1 # [ 22.970496] nix-daemon[857]: reaped child process 870, status = succeeded4369worker1 # [ 22.976766] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4370client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4371client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4372client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4373client # farm-a> LOG-farm-a-t14374worker1 # [ 24.407467] nix-daemon[857]: reaped child process 897, status = succeeded4375worker1 # [ 24.495713] nix-daemon[857]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4376worker1 # [ 24.507723] nix-daemon[857]: reaped child process 916, status = succeeded4377lb # [ 24.530928] niks3-server[957]: 2026/09/21 09:51:11 INFO Received uploads request method=POST path=/api/pending_closures4378worker1 # [ 24.523601] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.528Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4379worker1 # [ 24.525159] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.529Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4380lb # [ 24.571134] niks3-server[957]: 2026/09/21 09:51:11 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4381lb # [ 24.583835] niks3-server[957]: 2026/09/21 09:51:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4382lb # [ 24.591117] niks3-server[957]: 2026/09/21 09:51:11 INFO Signed narinfos id=2 count=14383worker1 # [ 24.579022] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.583Z level=INFO msg="Uploading 1 narinfos"4384lb # [ 24.595055] niks3-server[957]: 2026/09/21 09:51:11 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4385lb # [ 24.600253] niks3-server[957]: 2026/09/21 09:51:11 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4386lb # [ 24.606065] niks3-server[957]: 2026/09/21 09:51:11 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4387lb # [ 24.612401] niks3-server[957]: 2026/09/21 09:51:11 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4388lb # [ 24.616492] niks3-server[957]: 2026/09/21 09:51:11 INFO Completed upload id=24389worker1 # [ 24.601106] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.605Z level=INFO msg="Upload complete. (215ms)"4390worker1 # [ 24.607851] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14391worker1 # [ 24.612411] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04392worker1 # [ 24.615733] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14393worker1 # [ 24.621698] nix-daemon[857]: reaped child process 895, status = succeeded4394worker1 # [ 24.658753] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344395worker1 # [ 24.665845] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344396worker1 # [ 24.675302] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.679Z level=INFO msg="All 1 paths already cached"4397client # copying 1 paths to 'lb:50051'...4398worker1 # [ 24.678419] nix-grpc-daemon[779]: ts=2026-09-21T09:51:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344399client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4400worker1 # [ 24.688282] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4401worker1 # [ 24.798339] nix-daemon[857]: accepted connection from pid 922, user nix-grpc-daemon (trusted)4402worker1 # [ 24.815785] nix-daemon[857]: reaped child process 926, status = succeeded4403worker1 # [ 24.926155] nix-daemon[857]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4404worker1 # [ 24.940630] nix-daemon[857]: reaped child process 931, status = succeeded4405lb # [ 24.960902] niks3-server[957]: 2026/09/21 09:51:12 INFO Received uploads request method=POST path=/api/pending_closures4406worker1 # [ 24.950149] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.954Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4407worker1 # [ 24.951920] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.956Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4408lb # [ 24.986082] niks3-server[957]: 2026/09/21 09:51:12 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4409lb # [ 24.999135] niks3-server[957]: 2026/09/21 09:51:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4410worker1 # [ 24.990327] nix-grpc-daemon[872]: time=2026-09-21T09:51:12.995Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 25.005723] niks3-server[957]: 2026/09/21 09:51:12 INFO Signed narinfos id=3 count=14412lb # [ 25.008745] niks3-server[957]: 2026/09/21 09:51:12 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4413lb # [ 25.022298] niks3-server[957]: 2026/09/21 09:51:12 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4414lb # [ 25.026109] niks3-server[957]: 2026/09/21 09:51:12 INFO Completed upload id=34415worker1 # [ 25.010883] nix-grpc-daemon[872]: time=2026-09-21T09:51:13.015Z level=INFO msg="Upload complete. (308ms)"4416worker1 # [ 25.012807] nix-grpc-daemon[779]: ts=2026-09-21T09:51:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=6244417lb # [ 25.028861] niks3-server[957]: 2026/09/21 09:51:12 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4418worker1 # [ 25.016497] nix-grpc-daemon[779]: ts=2026-09-21T09:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344419worker1 # [ 25.023709] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4420worker1 # [ 25.029868] nix-daemon[857]: reaped child process 920, status = succeeded4421worker1 # [ 25.032669] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4422client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4423client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4424client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4425client # farm-b> LOG-farm-b-t14426worker1 # [ 26.249274] nix-daemon[857]: reaped child process 935, status = succeeded4427worker1 # [ 26.326127] nix-daemon[857]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4428worker1 # [ 26.339412] nix-daemon[857]: reaped child process 954, status = succeeded4429lb # [ 26.360464] niks3-server[957]: 2026/09/21 09:51:13 INFO Received uploads request method=POST path=/api/pending_closures4430worker1 # [ 26.350682] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.355Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4431worker1 # [ 26.352858] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.356Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4432lb # [ 26.379427] niks3-server[957]: 2026/09/21 09:51:13 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4433lb # [ 26.385511] niks3-server[957]: 2026/09/21 09:51:13 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4434worker1 # [ 26.375165] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.379Z level=INFO msg="Uploading 1 narinfos"4435lb # [ 26.390087] niks3-server[957]: 2026/09/21 09:51:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4436lb # [ 26.391973] niks3-server[957]: 2026/09/21 09:51:13 INFO Signed narinfos id=4 count=14437lb # [ 26.397298] niks3-server[957]: 2026/09/21 09:51:13 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4438lb # [ 26.405072] niks3-server[957]: 2026/09/21 09:51:13 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4439lb # [ 26.408679] niks3-server[957]: 2026/09/21 09:51:13 INFO Completed upload id=44440worker1 # [ 26.393585] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.398Z level=INFO msg="Upload complete. (153ms)"4441worker1 # [ 26.395978] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14442lb # [ 26.411784] niks3-server[957]: 2026/09/21 09:51:13 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4443worker1 # [ 26.402557] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04444worker1 # [ 26.404464] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14445worker1 # [ 26.409200] nix-daemon[857]: reaped child process 933, status = succeeded4446worker1 # [ 26.446466] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344447worker1 # [ 26.455383] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344448worker1 # [ 26.464295] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.468Z level=INFO msg="All 2 paths already cached"4449client # copying 1 paths to 'lb:50051'...4450client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4451worker1 # [ 26.470938] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344452worker1 # [ 26.484275] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4453worker1 # [ 26.584284] nix-daemon[857]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4454worker1 # [ 26.599378] nix-daemon[857]: reaped child process 963, status = succeeded4455worker1 # [ 26.688918] nix-daemon[857]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4456lb # [ 26.715945] envoy[914]: conn peer=[2001:db8:1::1]:35698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=14854457worker1 # [ 26.702555] nix-daemon[857]: reaped child process 968, status = succeeded4458lb # [ 26.717768] envoy[914]: conn peer=[2001:db8:1::1]:35708 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=594459lb # [ 26.719436] envoy[914]: conn peer=[2001:db8:1::1]:35710 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74460lb # [ 26.721169] envoy[914]: conn peer=[2001:db8:1::1]:35718 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84461lb # [ 26.725113] niks3-server[957]: 2026/09/21 09:51:14 INFO Received uploads request method=POST path=/api/pending_closures4462worker1 # [ 26.715153] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.719Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4463worker1 # [ 26.716583] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.719Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4464lb # [ 26.740115] niks3-server[957]: 2026/09/21 09:51:14 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4465lb # [ 26.742347] niks3-server[957]: 2026/09/21 09:51:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4466lb # [ 26.745734] niks3-server[957]: 2026/09/21 09:51:14 INFO Signed narinfos id=5 count=14467worker1 # [ 26.730516] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.735Z level=INFO msg="Uploading 1 narinfos"4468lb # [ 26.748980] niks3-server[957]: 2026/09/21 09:51:14 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4469lb # [ 26.752988] niks3-server[957]: 2026/09/21 09:51:14 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4470lb # [ 26.756742] niks3-server[957]: 2026/09/21 09:51:14 INFO Completed upload id=54471worker1 # [ 26.741654] nix-grpc-daemon[872]: time=2026-09-21T09:51:14.746Z level=INFO msg="Upload complete. (246ms)"4472worker1 # [ 26.743804] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=7524473worker1 # [ 26.746842] nix-grpc-daemon[779]: ts=2026-09-21T09:51:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344474lb # [ 26.762710] niks3-server[957]: 2026/09/21 09:51:14 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4475worker1 # [ 26.753441] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4476worker1 # [ 26.759149] nix-daemon[857]: reaped child process 958, status = succeeded4477worker1 # [ 26.762155] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4478client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4479client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4480client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4481client # farm-top> LOG-farm-top-t14482worker1 # [ 28.188527] nix-daemon[857]: reaped child process 972, status = succeeded4483worker1 # [ 28.267096] nix-daemon[857]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4484worker1 # [ 28.280684] nix-daemon[857]: reaped child process 992, status = succeeded4485lb # [ 28.302901] niks3-server[957]: 2026/09/21 09:51:15 INFO Received uploads request method=POST path=/api/pending_closures4486worker1 # [ 28.293177] nix-grpc-daemon[872]: time=2026-09-21T09:51:16.297Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4487worker1 # [ 28.294637] nix-grpc-daemon[872]: time=2026-09-21T09:51:16.299Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4488lb # [ 28.323284] niks3-server[957]: 2026/09/21 09:51:15 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4489lb # [ 28.326785] niks3-server[957]: 2026/09/21 09:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4490lb # [ 28.329322] niks3-server[957]: 2026/09/21 09:51:15 INFO Signed narinfos id=6 count=14491worker1 # [ 28.315524] nix-grpc-daemon[872]: time=2026-09-21T09:51:16.320Z level=INFO msg="Uploading 1 narinfos"4492lb # [ 28.333249] niks3-server[957]: 2026/09/21 09:51:15 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4493lb # [ 28.338252] niks3-server[957]: 2026/09/21 09:51:15 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4494lb # [ 28.341369] niks3-server[957]: 2026/09/21 09:51:15 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4495lb # [ 28.346279] niks3-server[957]: 2026/09/21 09:51:15 INFO Completed upload id=64496worker1 # [ 28.330941] nix-grpc-daemon[872]: time=2026-09-21T09:51:16.335Z level=INFO msg="Upload complete. (159ms)"4497lb # [ 28.348859] niks3-server[957]: 2026/09/21 09:51:15 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4498worker1 # [ 28.335119] nix-grpc-daemon[779]: ts=2026-09-21T09:51:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14499worker1 # [ 28.341558] nix-daemon[857]: reaped child process 970, status = succeeded4500client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 6.24 seconds)4501client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4502client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4503client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4504client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4505client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4506client # copying 3 paths...4507client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4508client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4509client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4510client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4511(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.58 seconds)4512subtest: repeat is answered Cached by the scheduler without building4513worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4514worker1 # finding garbage collector roots...4515worker1 # deleting specified paths...4516worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4517worker1 # deleting unused links...4518worker1 # note: hard linking is currently saving 0.0 KiB4519worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4520worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4521worker2 # finding garbage collector roots...4522worker2 # deleting specified paths...4523worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4524worker2 # deleting unused links...4525worker2 # note: hard linking is currently saving 0.0 KiB4526worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.19 seconds)4527client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24528worker2 # [ 29.233431] nix-grpc-daemon[785]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:412464529client # don't know how to build these paths:4530client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4531worker2 # [ 29.241689] nix-grpc-daemon[785]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464532worker1 # [ 29.251251] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14533client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4534client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4535client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4536lb: must succeed: curl -sf localhost:9901/clusters4537lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4538lb: must succeed: curl -sf localhost:9901/clusters4539lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4540worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4541worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4542(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.81 seconds)4543subtest: inputs already in the cache are not sent to the scheduler4544lb: must succeed: curl -sf localhost:9901/clusters4545lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4546lb: must succeed: curl -sf localhost:9901/clusters4547lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4548worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4549worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4550client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24551worker1 # [ 29.709723] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357344552client # don't know how to build these paths:4553client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4554worker2 # [ 29.738141] nix-grpc-daemon[785]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464555worker1 # [ 29.732202] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04556worker1 # [ 29.737966] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14557worker1 # [ 29.744495] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344558worker1 # [ 29.756166] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344559worker1 # [ 29.770374] nix-grpc-daemon[872]: time=2026-09-21T09:51:17.774Z level=INFO msg="All 2 paths already cached"4560client # copying 1 paths to 'lb:50051'...4561client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4562worker1 # [ 29.778918] nix-grpc-daemon[779]: ts=2026-09-21T09:51:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344563worker1 # [ 29.791679] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4564worker1 # [ 29.890261] nix-daemon[857]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4565worker1 # [ 29.907703] nix-daemon[857]: reaped child process 1024, status = succeeded4566worker1 # [ 29.994758] nix-daemon[857]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4567worker1 # [ 30.009849] nix-daemon[857]: reaped child process 1029, status = succeeded4568lb # [ 30.032360] niks3-server[957]: 2026/09/21 09:51:17 INFO Received uploads request method=POST path=/api/pending_closures4569worker1 # [ 30.025164] nix-grpc-daemon[872]: time=2026-09-21T09:51:18.029Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4570worker1 # [ 30.026697] nix-grpc-daemon[872]: time=2026-09-21T09:51:18.029Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4571lb # [ 30.061509] niks3-server[957]: 2026/09/21 09:51:17 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4572lb # [ 30.072295] niks3-server[957]: 2026/09/21 09:51:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4573lb # [ 30.077803] niks3-server[957]: 2026/09/21 09:51:17 INFO Signed narinfos id=7 count=14574worker1 # [ 30.067048] nix-grpc-daemon[872]: time=2026-09-21T09:51:18.071Z level=INFO msg="Uploading 1 narinfos"4575lb # [ 30.091330] niks3-server[957]: 2026/09/21 09:51:17 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4576lb # [ 30.100250] niks3-server[957]: 2026/09/21 09:51:17 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4577lb # [ 30.107563] niks3-server[957]: 2026/09/21 09:51:17 INFO Completed upload id=74578worker1 # [ 30.092211] nix-grpc-daemon[872]: time=2026-09-21T09:51:18.096Z level=INFO msg="Upload complete. (290ms)"4579lb # [ 30.109400] niks3-server[957]: 2026/09/21 09:51:17 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4580worker1 # [ 30.094318] nix-grpc-daemon[779]: ts=2026-09-21T09:51:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=7524581worker1 # [ 30.099739] nix-grpc-daemon[779]: ts=2026-09-21T09:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344582worker1 # [ 30.107571] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4583worker1 # [ 30.109162] nix-daemon[857]: reaped child process 1019, status = succeeded4584client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4585worker1 # [ 30.115769] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4586client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4587client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4588client # farm-top> LOG-farm-top-t1b4589worker1 # [ 31.333248] nix-daemon[857]: reaped child process 1033, status = succeeded4590worker1 # [ 31.411583] nix-daemon[857]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4591worker1 # [ 31.423990] nix-daemon[857]: reaped child process 1053, status = succeeded4592lb # [ 31.447837] niks3-server[957]: 2026/09/21 09:51:18 INFO Received uploads request method=POST path=/api/pending_closures4593worker1 # [ 31.441471] nix-grpc-daemon[872]: time=2026-09-21T09:51:19.445Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4594worker1 # [ 31.443045] nix-grpc-daemon[872]: time=2026-09-21T09:51:19.447Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4595lb # [ 31.483667] niks3-server[957]: 2026/09/21 09:51:18 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4596lb # [ 31.496859] niks3-server[957]: 2026/09/21 09:51:18 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4597lb # [ 31.508619] niks3-server[957]: 2026/09/21 09:51:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4598lb # [ 31.514096] niks3-server[957]: 2026/09/21 09:51:18 INFO Signed narinfos id=8 count=14599worker1 # [ 31.502648] nix-grpc-daemon[872]: time=2026-09-21T09:51:19.507Z level=INFO msg="Uploading 1 narinfos"4600lb # [ 31.521242] niks3-server[957]: 2026/09/21 09:51:18 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4601lb # [ 31.531058] niks3-server[957]: 2026/09/21 09:51:18 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4602lb # [ 31.536058] niks3-server[957]: 2026/09/21 09:51:18 INFO Completed upload id=84603worker1 # [ 31.522272] nix-grpc-daemon[872]: time=2026-09-21T09:51:19.527Z level=INFO msg="Upload complete. (199ms)"4604lb # [ 31.537844] niks3-server[957]: 2026/09/21 09:51:18 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4605worker1 # [ 31.525433] nix-grpc-daemon[779]: ts=2026-09-21T09:51:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14606worker1 # [ 31.533144] nix-daemon[857]: reaped child process 1031, status = succeeded4607client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.97 seconds)4608client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4609client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4610lb: must succeed: curl -sf localhost:9901/clusters4611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4612lb: must succeed: curl -sf localhost:9901/clusters4613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4614worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4615worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4616(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.25 seconds)4617subtest: two clients wanting the same drv build it once4618client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4619client # [ 31.844940] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4620client # Running as unit: dedup1.service; invocation ID: 661be514a00d467b81f6ddb7528b90114621client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.06 seconds)4622client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4623worker1 # [ 31.965477] nix-grpc-daemon[779]: ts=2026-09-21T09:51:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357344624client # [ 31.994647] nix[886]: don't know how to build these paths:4625client # [ 31.995580] nix[886]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4626worker1 # [ 31.978771] nix-grpc-daemon[779]: ts=2026-09-21T09:51:19Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04627worker1 # [ 31.982967] nix-grpc-daemon[779]: ts=2026-09-21T09:51:19Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14628worker2 # [ 32.009972] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412464629worker2 # [ 32.022927] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464630client # [ 32.037072] nix[886]: copying 1 paths to 'lb:50051'...4631client # [ 32.037799] nix[886]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4632worker2 # [ 32.033619] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412464633worker2 # [ 32.044962] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4634worker2 # [ 32.065218] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=info event=niks3_push_started pid=8774635worker2 # [ 32.222811] nix-grpc-daemon[877]: time=2026-09-21T09:51:20.285Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4636worker2 # [ 32.316789] nix-daemon[849]: accepted connection from pid 891, user nix-grpc-daemon (trusted)4637worker2 # [ 32.330396] nix-daemon[849]: reaped child process 894, status = succeeded4638worker2 # [ 32.403682] nix-daemon[849]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4639worker2 # [ 32.417728] nix-daemon[849]: reaped child process 899, status = succeeded4640lb # [ 32.416928] niks3-server[957]: 2026/09/21 09:51:19 INFO Received uploads request method=POST path=/api/pending_closures4641worker2 # [ 32.430480] nix-grpc-daemon[877]: time=2026-09-21T09:51:20.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4642worker2 # [ 32.431855] nix-grpc-daemon[877]: time=2026-09-21T09:51:20.493Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4643lb # [ 32.455478] niks3-server[957]: 2026/09/21 09:51:19 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4644worker2 # [ 32.468030] nix-grpc-daemon[877]: time=2026-09-21T09:51:20.531Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 32.464154] niks3-server[957]: 2026/09/21 09:51:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4646lb # [ 32.468224] niks3-server[957]: 2026/09/21 09:51:19 INFO Signed narinfos id=9 count=14647lb # [ 32.483148] niks3-server[957]: 2026/09/21 09:51:19 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4648lb # [ 32.492460] niks3-server[957]: 2026/09/21 09:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4649lb # [ 32.499060] niks3-server[957]: 2026/09/21 09:51:19 INFO Completed upload id=94650worker2 # [ 32.502935] nix-grpc-daemon[877]: time=2026-09-21T09:51:20.566Z level=INFO msg="Upload complete. (275ms)"4651lb # [ 32.503886] niks3-server[957]: 2026/09/21 09:51:19 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4652worker2 # [ 32.507169] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=1 nar_bytes_in=4324653worker2 # [ 32.512412] nix-grpc-daemon[785]: ts=2026-09-21T09:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412464654worker2 # [ 32.520580] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4655worker2 # [ 32.527927] nix-daemon[849]: reaped child process 875, status = succeeded4656worker2 # [ 32.532516] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4657client # [ 32.535972] nix[886]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4658client # [ 32.909459] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4659client # Running as unit: dedup2.service; invocation ID: c229f9656f874191a0d254fc36182e914660client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4661worker1 # [ 33.076269] nix-grpc-daemon[779]: ts=2026-09-21T09:51:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357344662client # [ 33.100299] nix[902]: don't know how to build these paths:4663client # [ 33.101324] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4664worker1 # [ 33.083263] nix-grpc-daemon[779]: ts=2026-09-21T09:51:21Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04665worker2 # [ 33.109172] nix-grpc-daemon[785]: ts=2026-09-21T09:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412464666client # [ 33.117796] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4667client: must succeed: sleep 24668client: (finished: must succeed: sleep 2, in 2.03 seconds)4669client: waiting for success: ! systemctl is-active dedup1 dedup24670worker2 # [ 36.367525] nix-daemon[849]: reaped child process 903, status = succeeded4671worker2 # [ 36.453251] nix-daemon[849]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4672worker2 # [ 36.464416] nix-daemon[849]: reaped child process 937, status = succeeded4673lb # [ 36.468248] niks3-server[957]: 2026/09/21 09:51:23 INFO Received uploads request method=POST path=/api/pending_closures4674worker2 # [ 36.480343] nix-grpc-daemon[877]: time=2026-09-21T09:51:24.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4675worker2 # [ 36.482229] nix-grpc-daemon[877]: time=2026-09-21T09:51:24.545Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4676lb # [ 36.507265] niks3-server[957]: 2026/09/21 09:51:23 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4677lb # [ 36.520789] niks3-server[957]: 2026/09/21 09:51:23 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4678lb # [ 36.537580] niks3-server[957]: 2026/09/21 09:51:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4679lb # [ 36.541554] niks3-server[957]: 2026/09/21 09:51:23 INFO Signed narinfos id=10 count=14680worker2 # [ 36.549773] nix-grpc-daemon[877]: time=2026-09-21T09:51:24.611Z level=INFO msg="Uploading 1 narinfos"4681lb # [ 36.549325] niks3-server[957]: 2026/09/21 09:51:23 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4682lb # [ 36.555221] niks3-server[957]: 2026/09/21 09:51:23 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4683lb # [ 36.558843] niks3-server[957]: 2026/09/21 09:51:23 INFO Completed upload id=104684worker2 # [ 36.562883] nix-grpc-daemon[877]: time=2026-09-21T09:51:24.626Z level=INFO msg="Upload complete. (218ms)"4685lb # [ 36.561624] niks3-server[957]: 2026/09/21 09:51:23 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4686worker2 # [ 36.570380] nix-grpc-daemon[785]: ts=2026-09-21T09:51:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14687worker2 # [ 36.576453] nix-grpc-daemon[785]: ts=2026-09-21T09:51:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14688worker2 # [ 36.588675] nix-daemon[849]: reaped child process 901, status = succeeded4689client # [ 36.611502] systemd[1]: dedup2.service: Deactivated successfully.4690client # [ 36.614692] systemd[1]: dedup2.service: Consumed 165ms CPU time over 3.702s wall clock time, 35.4M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4691client # [ 36.653747] systemd[1]: dedup1.service: Deactivated successfully.4692client # [ 36.655333] systemd[1]: dedup1.service: Consumed 131ms CPU time over 4.810s wall clock time, 43.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.4693lb # [ 36.716752] envoy[914]: conn peer=[2001:db8:1::1]:35728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9477 tx=4062 ms=61024694lb # [ 36.720717] envoy[914]: conn peer=[2001:db8:1::1]:53662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=524695lb # [ 36.724498] envoy[914]: conn peer=[2001:db8:1::1]:53678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1782 ms=18294696lb # [ 36.728922] envoy[914]: conn peer=[2001:db8:1::1]:42074 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=34954697lb # [ 36.732537] envoy[914]: conn peer=[2001:db8:1::1]:53692 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1427 ms=46424698client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4699client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4700client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.03 seconds)4701worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4702worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4703worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4704worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4705worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4706worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4707worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4708worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4709(finished: subtest: two clients wanting the same drv build it once, in 5.71 seconds)4710subtest: build hook: nix-daemon with builders = grpc://lb4711client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log4712client # [ 37.626490] systemd[1]: Started Nix Daemon.4713client # [ 37.715827] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4714worker1 # [ 37.813540] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:357344715worker2 # [ 37.836482] nix-grpc-daemon[785]: ts=2026-09-21T09:51:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464716worker1 # [ 37.833534] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04717worker1 # [ 37.838624] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14718worker1 # [ 37.877646] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344719worker1 # [ 37.890113] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344720worker1 # [ 37.901879] nix-grpc-daemon[779]: ts=2026-09-21T09:51:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344721worker1 # [ 37.910795] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4722worker1 # [ 38.026287] nix-daemon[857]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4723worker1 # [ 38.041673] nix-daemon[857]: reaped child process 1094, status = succeeded4724worker1 # [ 38.125837] nix-daemon[857]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)4725worker1 # [ 38.138542] nix-daemon[857]: reaped child process 1099, status = succeeded4726lb # [ 38.163445] niks3-server[957]: 2026/09/21 09:51:25 INFO Received uploads request method=POST path=/api/pending_closures4727worker1 # [ 38.159729] nix-grpc-daemon[872]: time=2026-09-21T09:51:26.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4728worker1 # [ 38.161607] nix-grpc-daemon[872]: time=2026-09-21T09:51:26.166Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4729lb # [ 38.201305] niks3-server[957]: 2026/09/21 09:51:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4730lb # [ 38.206260] niks3-server[957]: 2026/09/21 09:51:25 INFO Signed narinfos id=11 count=14731worker1 # [ 38.194313] nix-grpc-daemon[872]: time=2026-09-21T09:51:26.198Z level=INFO msg="Uploading 1 narinfos"4732lb # [ 38.220141] niks3-server[957]: 2026/09/21 09:51:25 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4733lb # [ 38.247090] niks3-server[957]: 2026/09/21 09:51:25 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4734lb # [ 38.255390] niks3-server[957]: 2026/09/21 09:51:25 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4735lb # [ 38.264227] niks3-server[957]: 2026/09/21 09:51:25 INFO Completed upload id=114736worker1 # [ 38.249248] nix-grpc-daemon[872]: time=2026-09-21T09:51:26.253Z level=INFO msg="Upload complete. (320ms)"4737lb # [ 38.267273] niks3-server[957]: 2026/09/21 09:51:25 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4738worker1 # [ 38.254156] nix-grpc-daemon[779]: ts=2026-09-21T09:51:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=5124739worker1 # [ 38.261366] nix-grpc-daemon[779]: ts=2026-09-21T09:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344740worker1 # [ 38.272738] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4741worker1 # [ 38.281690] nix-daemon[857]: reaped child process 1089, status = succeeded4742worker1 # [ 38.290249] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4743worker1 # [ 39.134939] nix-daemon[857]: reaped child process 1103, status = succeeded4744worker1 # [ 39.219102] nix-daemon[857]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4745worker1 # [ 39.234304] nix-daemon[857]: reaped child process 1122, status = succeeded4746lb # [ 39.249585] niks3-server[957]: 2026/09/21 09:51:26 INFO Received uploads request method=POST path=/api/pending_closures4747worker1 # [ 39.239970] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.244Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4748worker1 # [ 39.241485] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.246Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4749lb # [ 39.288786] niks3-server[957]: 2026/09/21 09:51:26 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4750lb # [ 39.299099] niks3-server[957]: 2026/09/21 09:51:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4751lb # [ 39.306865] niks3-server[957]: 2026/09/21 09:51:26 INFO Signed narinfos id=12 count=14752worker1 # [ 39.294713] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.299Z level=INFO msg="Uploading 1 narinfos"4753lb # [ 39.314562] niks3-server[957]: 2026/09/21 09:51:26 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4754lb # [ 39.319468] niks3-server[957]: 2026/09/21 09:51:26 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4755lb # [ 39.331367] niks3-server[957]: 2026/09/21 09:51:26 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4756lb # [ 39.337419] niks3-server[957]: 2026/09/21 09:51:26 INFO Completed upload id=124757worker1 # [ 39.322531] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.327Z level=INFO msg="Upload complete. (225ms)"4758lb # [ 39.340157] niks3-server[957]: 2026/09/21 09:51:26 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4759worker1 # [ 39.329455] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14760worker2 # [ 39.362198] nix-grpc-daemon[785]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412464761worker1 # [ 39.346767] nix-daemon[857]: reaped child process 1101, status = succeeded4762worker2 # [ 39.367220] nix-grpc-daemon[785]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4763worker2 # [ 39.386924] nix-grpc-daemon[785]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:412464764worker2 # [ 39.389292] nix-grpc-daemon[785]: ts=2026-09-21T09:51:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=1 nar_bytes_out=1284765worker2 # [ 39.606187] nix-grpc-daemon[785]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:412464766worker1 # [ 39.597926] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344767worker1 # [ 39.675320] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.679Z level=INFO msg="All 1 paths already cached"4768worker1 # [ 39.689866] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04769worker1 # [ 39.695191] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14770worker1 # [ 39.700975] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344771worker1 # [ 39.712671] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344772worker1 # [ 39.720325] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.725Z level=INFO msg="All 1 paths already cached"4773worker1 # [ 39.724162] nix-grpc-daemon[779]: ts=2026-09-21T09:51:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344774worker1 # [ 39.727847] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4775worker1 # [ 39.816512] nix-daemon[857]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4776worker1 # [ 39.832088] nix-daemon[857]: reaped child process 1131, status = succeeded4777worker1 # [ 39.911466] nix-daemon[857]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)4778worker1 # [ 39.926144] nix-daemon[857]: reaped child process 1136, status = succeeded4779lb # [ 39.946619] niks3-server[957]: 2026/09/21 09:51:27 INFO Received uploads request method=POST path=/api/pending_closures4780worker1 # [ 39.937972] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.942Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4781worker1 # [ 39.939372] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.942Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4782lb # [ 39.973462] niks3-server[957]: 2026/09/21 09:51:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4783lb # [ 39.979974] niks3-server[957]: 2026/09/21 09:51:27 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4784worker1 # [ 39.970961] nix-grpc-daemon[872]: time=2026-09-21T09:51:27.975Z level=INFO msg="Uploading 1 narinfos"4785lb # [ 39.985964] niks3-server[957]: 2026/09/21 09:51:27 INFO Signed narinfos id=13 count=14786lb # [ 39.994692] niks3-server[957]: 2026/09/21 09:51:27 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4787lb # [ 40.005256] niks3-server[957]: 2026/09/21 09:51:27 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4788lb # [ 40.014060] niks3-server[957]: 2026/09/21 09:51:27 INFO Completed upload id=134789worker1 # [ 39.999182] nix-grpc-daemon[872]: time=2026-09-21T09:51:28.003Z level=INFO msg="Upload complete. (265ms)"4790worker1 # [ 40.001167] nix-grpc-daemon[779]: ts=2026-09-21T09:51:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=6244791lb # [ 40.017285] niks3-server[957]: 2026/09/21 09:51:27 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4792worker1 # [ 40.004565] nix-grpc-daemon[779]: ts=2026-09-21T09:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344793worker1 # [ 40.011276] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4794worker1 # [ 40.016133] nix-daemon[857]: reaped child process 1126, status = succeeded4795worker1 # [ 40.019338] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4796worker1 # [ 41.456702] nix-daemon[857]: reaped child process 1140, status = succeeded4797worker1 # [ 41.538184] nix-daemon[857]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4798worker1 # [ 41.551830] nix-daemon[857]: reaped child process 1160, status = succeeded4799lb # [ 41.573777] niks3-server[957]: 2026/09/21 09:51:28 INFO Received uploads request method=POST path=/api/pending_closures4800worker1 # [ 41.567600] nix-grpc-daemon[872]: time=2026-09-21T09:51:29.572Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4801worker1 # [ 41.569046] nix-grpc-daemon[872]: time=2026-09-21T09:51:29.573Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4802lb # [ 41.616510] niks3-server[957]: 2026/09/21 09:51:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4803worker1 # [ 41.609285] nix-grpc-daemon[872]: time=2026-09-21T09:51:29.614Z level=INFO msg="Uploading 1 narinfos"4804lb # [ 41.624374] niks3-server[957]: 2026/09/21 09:51:28 INFO Signed narinfos id=14 count=14805lb # [ 41.633540] niks3-server[957]: 2026/09/21 09:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4806lb # [ 41.690117] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4807lb # [ 41.701060] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4808lb # [ 41.707826] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4809lb # [ 41.715474] niks3-server[957]: 2026/09/21 09:51:29 INFO Completed upload id=144810worker1 # [ 41.700566] nix-grpc-daemon[872]: time=2026-09-21T09:51:29.705Z level=INFO msg="Upload complete. (273ms)"4811lb # [ 41.722176] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4812worker1 # [ 41.708431] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14813worker2 # [ 41.740171] nix-grpc-daemon[785]: ts=2026-09-21T09:51:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412464814worker1 # [ 41.723414] nix-daemon[857]: reaped child process 1138, status = succeeded4815worker2 # [ 41.755239] nix-grpc-daemon[785]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4816worker1 # [ 41.769150] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:357344817worker1 # [ 41.773546] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_out=1764818worker2 # [ 41.965181] nix-grpc-daemon[785]: ts=2026-09-21T09:51:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:412464819worker1 # [ 41.949497] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344820worker1 # [ 41.958467] nix-grpc-daemon[872]: time=2026-09-21T09:51:29.963Z level=INFO msg="All 2 paths already cached"4821worker1 # [ 41.972543] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04822worker1 # [ 41.977714] nix-grpc-daemon[779]: ts=2026-09-21T09:51:29Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14823worker1 # [ 42.017133] nix-grpc-daemon[779]: ts=2026-09-21T09:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344824worker1 # [ 42.030103] nix-grpc-daemon[779]: ts=2026-09-21T09:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357344825worker1 # [ 42.042742] nix-grpc-daemon[872]: time=2026-09-21T09:51:30.047Z level=INFO msg="All 2 paths already cached"4826worker1 # [ 42.050564] nix-grpc-daemon[779]: ts=2026-09-21T09:51:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344827worker1 # [ 42.059399] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4828worker1 # [ 42.158856] nix-daemon[857]: accepted connection from pid 1166, user nix-grpc-daemon (trusted)4829worker1 # [ 42.172855] nix-daemon[857]: reaped child process 1169, status = succeeded4830worker1 # [ 42.252357] nix-daemon[857]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)4831worker1 # [ 42.265445] nix-daemon[857]: reaped child process 1174, status = succeeded4832lb # [ 42.287662] niks3-server[957]: 2026/09/21 09:51:29 INFO Received uploads request method=POST path=/api/pending_closures4833worker1 # [ 42.280708] nix-grpc-daemon[872]: time=2026-09-21T09:51:30.285Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4834worker1 # [ 42.282285] nix-grpc-daemon[872]: time=2026-09-21T09:51:30.287Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4835lb # [ 42.316571] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4836lb # [ 42.332126] niks3-server[957]: 2026/09/21 09:51:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4837lb # [ 42.340550] niks3-server[957]: 2026/09/21 09:51:29 INFO Signed narinfos id=15 count=14838worker1 # [ 42.328490] nix-grpc-daemon[872]: time=2026-09-21T09:51:30.332Z level=INFO msg="Uploading 1 narinfos"4839lb # [ 42.345108] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4840lb # [ 42.358985] niks3-server[957]: 2026/09/21 09:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4841lb # [ 42.366825] niks3-server[957]: 2026/09/21 09:51:29 INFO Completed upload id=154842worker1 # [ 42.351939] nix-grpc-daemon[872]: time=2026-09-21T09:51:30.356Z level=INFO msg="Upload complete. (278ms)"4843lb # [ 42.370045] niks3-server[957]: 2026/09/21 09:51:29 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4844worker1 # [ 42.355732] nix-grpc-daemon[779]: ts=2026-09-21T09:51:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=7524845worker1 # [ 42.362189] nix-grpc-daemon[779]: ts=2026-09-21T09:51:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357344846worker1 # [ 42.373132] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4847worker1 # [ 42.380114] nix-daemon[857]: reaped child process 1164, status = succeeded4848worker1 # [ 42.383159] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4849worker1 # [ 43.418970] nix-daemon[857]: reaped child process 1178, status = succeeded4850worker1 # [ 43.503291] nix-daemon[857]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)4851worker1 # [ 43.517137] nix-daemon[857]: reaped child process 1197, status = succeeded4852lb # [ 43.539719] niks3-server[957]: 2026/09/21 09:51:30 INFO Received uploads request method=POST path=/api/pending_closures4853worker1 # [ 43.533169] nix-grpc-daemon[872]: time=2026-09-21T09:51:31.537Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4854worker1 # [ 43.534657] nix-grpc-daemon[872]: time=2026-09-21T09:51:31.539Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4855lb # [ 43.582926] niks3-server[957]: 2026/09/21 09:51:30 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4856lb # [ 43.588605] niks3-server[957]: 2026/09/21 09:51:30 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4857lb # [ 43.598137] niks3-server[957]: 2026/09/21 09:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4858worker1 # [ 43.589744] nix-grpc-daemon[872]: time=2026-09-21T09:51:31.594Z level=INFO msg="Uploading 1 narinfos"4859lb # [ 43.605428] niks3-server[957]: 2026/09/21 09:51:30 INFO Signed narinfos id=16 count=14860lb # [ 43.608884] niks3-server[957]: 2026/09/21 09:51:30 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4861lb # [ 43.621295] niks3-server[957]: 2026/09/21 09:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4862lb # [ 43.627580] niks3-server[957]: 2026/09/21 09:51:30 INFO Completed upload id=164863lb # [ 43.630898] niks3-server[957]: 2026/09/21 09:51:31 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4864worker1 # [ 43.614243] nix-grpc-daemon[872]: time=2026-09-21T09:51:31.618Z level=INFO msg="Upload complete. (221ms)"4865worker1 # [ 43.622554] nix-grpc-daemon[779]: ts=2026-09-21T09:51:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14866worker2 # [ 43.644575] nix-grpc-daemon[785]: ts=2026-09-21T09:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412464867worker2 # [ 43.652947] nix-grpc-daemon[785]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4868worker1 # [ 43.636378] nix-daemon[857]: reaped child process 1176, status = succeeded4869worker2 # [ 43.675089] nix-grpc-daemon[785]: ts=2026-09-21T09:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:412464870worker2 # [ 43.680077] nix-grpc-daemon[785]: ts=2026-09-21T09:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=1 nar_bytes_out=2324871client # [ 43.801482] nix-daemon[946]: reaped child process 948, status = succeeded4872client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.30 seconds)4873client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4874client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)4875client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4876client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)4877client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4878client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)4879(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.36 seconds)4880subtest: an input only one worker has still reaches the builder4881client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only4882client # this derivation will be built:4883client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4884client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4885client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.31 seconds)4886client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4887client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds)4888worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4889worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)4890worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4891worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4892client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&24893client # [ 44.541934] nix-daemon[946]: accepted connection from pid 1029, user root (trusted)4894client # this derivation will be built:4895client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4896worker1 # [ 44.639592] nix-grpc-daemon[779]: ts=2026-09-21T09:51:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:357344897client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...4898worker2 # [ 44.662575] nix-grpc-daemon[785]: ts=2026-09-21T09:51:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464899client # copying 1 paths...4900client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...4901worker1 # [ 44.655337] nix-grpc-daemon[779]: ts=2026-09-21T09:51:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357344902worker1 # [ 44.663863] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4903worker1 # [ 44.766357] nix-daemon[857]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)4904worker1 # [ 44.781165] nix-daemon[857]: reaped child process 1209, status = succeeded4905lb # [ 44.802686] niks3-server[957]: 2026/09/21 09:51:32 INFO Received uploads request method=POST path=/api/pending_closures4906worker1 # [ 44.796231] nix-grpc-daemon[872]: time=2026-09-21T09:51:32.800Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4907worker1 # [ 44.798047] nix-grpc-daemon[872]: time=2026-09-21T09:51:32.802Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4908lb # [ 44.833998] niks3-server[957]: 2026/09/21 09:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4909lb # [ 44.841850] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4910lb # [ 44.848139] niks3-server[957]: 2026/09/21 09:51:32 INFO Signed narinfos id=17 count=14911worker1 # [ 44.836667] nix-grpc-daemon[872]: time=2026-09-21T09:51:32.841Z level=INFO msg="Uploading 1 narinfos"4912lb # [ 44.852453] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4913lb # [ 44.866263] niks3-server[957]: 2026/09/21 09:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4914lb # [ 44.874052] niks3-server[957]: 2026/09/21 09:51:32 INFO Completed upload id=174915worker1 # [ 44.861735] nix-grpc-daemon[872]: time=2026-09-21T09:51:32.866Z level=INFO msg="Upload complete. (185ms)"4916lb # [ 44.877720] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4917worker1 # [ 44.866494] nix-grpc-daemon[779]: ts=2026-09-21T09:51:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_in=1204918worker1 # [ 44.879611] nix-grpc-daemon[779]: ts=2026-09-21T09:51:32Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04919worker1 # [ 44.885203] nix-grpc-daemon[779]: ts=2026-09-21T09:51:32Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14920worker1 # [ 44.890869] nix-daemon[857]: reaped child process 1204, status = succeeded4921worker2 # [ 44.942987] nix-grpc-daemon[785]: ts=2026-09-21T09:51:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412464922worker2 # [ 44.955746] nix-grpc-daemon[785]: ts=2026-09-21T09:51:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412464923client # copying 2 paths to 'lb:50051'...4924client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4925worker2 # [ 44.972681] nix-grpc-daemon[785]: ts=2026-09-21T09:51:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412464926client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4927worker2 # [ 44.986995] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4928worker2 # [ 45.096203] nix-daemon[849]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4929worker2 # [ 45.110739] nix-daemon[849]: reaped child process 961, status = succeeded4930worker2 # [ 45.188153] nix-daemon[849]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4931worker2 # [ 45.200970] nix-daemon[849]: reaped child process 966, status = succeeded4932lb # [ 45.202796] niks3-server[957]: 2026/09/21 09:51:32 INFO Received uploads request method=POST path=/api/pending_closures4933lb # [ 45.211882] niks3-server[957]: 2026/09/21 09:51:32 INFO Received uploads request method=POST path=/api/pending_closures4934worker2 # [ 45.224195] nix-grpc-daemon[877]: time=2026-09-21T09:51:33.287Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4935worker2 # [ 45.226579] nix-grpc-daemon[877]: time=2026-09-21T09:51:33.289Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4936worker2 # [ 45.228249] nix-grpc-daemon[877]: time=2026-09-21T09:51:33.289Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4937lb # [ 45.246344] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst4938lb # [ 45.257698] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4939lb # [ 45.272131] niks3-server[957]: 2026/09/21 09:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4940lb # [ 45.275555] niks3-server[957]: 2026/09/21 09:51:32 INFO Signed narinfos id=18 count=14941lb # [ 45.277515] niks3-server[957]: 2026/09/21 09:51:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4942lb # [ 45.279306] niks3-server[957]: 2026/09/21 09:51:32 INFO Signed narinfos id=19 count=24943lb # [ 45.280706] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls4944worker2 # [ 45.285306] nix-grpc-daemon[877]: time=2026-09-21T09:51:33.347Z level=INFO msg="Uploading 3 narinfos"4945lb # [ 45.291362] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4946lb # [ 45.296694] niks3-server[957]: 2026/09/21 09:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4947lb # [ 45.300690] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo4948lb # [ 45.303539] niks3-server[957]: 2026/09/21 09:51:32 INFO Completed upload id=184949lb # [ 45.305047] niks3-server[957]: 2026/09/21 09:51:32 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4950lb # [ 45.307657] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4951lb # [ 45.310449] niks3-server[957]: 2026/09/21 09:51:32 INFO Completed upload id=194952worker2 # [ 45.314314] nix-grpc-daemon[877]: time=2026-09-21T09:51:33.377Z level=INFO msg="Upload complete. (314ms)"4953lb # [ 45.313461] niks3-server[957]: 2026/09/21 09:51:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo4954worker2 # [ 45.320125] nix-grpc-daemon[785]: ts=2026-09-21T09:51:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=2 nar_bytes_in=9764955worker2 # [ 45.325644] nix-grpc-daemon[785]: ts=2026-09-21T09:51:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412464956worker2 # [ 45.335113] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4957worker2 # [ 45.345201] nix-daemon[849]: reaped child process 956, status = succeeded4958worker2 # [ 45.349815] nix-grpc-daemon[785]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...4959worker2 # [ 45.356233] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4960client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...4961client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4962client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4963worker2 # [ 46.181405] nix-daemon[849]: reaped child process 971, status = succeeded4964worker2 # [ 46.262904] nix-daemon[849]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4965worker2 # [ 46.274850] nix-daemon[849]: reaped child process 990, status = succeeded4966lb # [ 46.280989] niks3-server[957]: 2026/09/21 09:51:33 INFO Received uploads request method=POST path=/api/pending_closures4967worker2 # [ 46.293189] nix-grpc-daemon[877]: time=2026-09-21T09:51:34.356Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4968lb # [ 46.313145] niks3-server[957]: 2026/09/21 09:51:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4969lb # [ 46.319511] niks3-server[957]: 2026/09/21 09:51:33 INFO Signed narinfos id=20 count=14970worker2 # [ 46.326584] nix-grpc-daemon[877]: time=2026-09-21T09:51:34.389Z level=INFO msg="Uploading 1 narinfos"4971lb # [ 46.325110] niks3-server[957]: 2026/09/21 09:51:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls4972lb # [ 46.339276] niks3-server[957]: 2026/09/21 09:51:33 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4973lb # [ 46.345831] niks3-server[957]: 2026/09/21 09:51:33 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4974lb # [ 46.350353] niks3-server[957]: 2026/09/21 09:51:33 INFO Completed upload id=204975worker2 # [ 46.356885] nix-grpc-daemon[877]: time=2026-09-21T09:51:34.420Z level=INFO msg="Upload complete. (192ms)"4976lb # [ 46.355567] niks3-server[957]: 2026/09/21 09:51:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo4977worker2 # [ 46.362342] nix-grpc-daemon[785]: ts=2026-09-21T09:51:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=14978worker2 # [ 46.372677] nix-daemon[849]: reaped child process 968, status = succeeded4979client # copying 1 paths...4980worker1 # [ 46.389942] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:357344981worker1 # [ 46.403059] nix-grpc-daemon[779]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...4982client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...4983worker2 # [ 46.455253] nix-grpc-daemon[785]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:412464984worker2 # [ 46.460189] nix-grpc-daemon[785]: ts=2026-09-21T09:51:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=1 nar_bytes_out=1204985client # [ 46.535505] nix-daemon[946]: reaped child process 1032, status = succeeded4986client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.12 seconds)4987client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&24988client # [ 46.638095] nix-daemon[946]: accepted connection from pid 1050, user root (trusted)4989client # this derivation will be built:4990client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv4991lb # [ 46.717475] envoy[914]: conn peer=[2001:db8:1::1]:42076 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3403 tx=1828 ms=16104992lb # [ 46.721386] envoy[914]: conn peer=[2001:db8:1::1]:42080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=544993lb # [ 46.724873] envoy[914]: conn peer=[2001:db8:1::1]:42082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3906 tx=2066 ms=21974994lb # [ 46.728378] envoy[914]: conn peer=[2001:db8:1::1]:42088 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=184995lb # [ 46.732408] envoy[914]: conn peer=[2001:db8:1::1]:42104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4329 tx=2274 ms=17684996lb # [ 46.735867] envoy[914]: conn peer=[2001:db8:1::1]:41494 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=584997lb # [ 46.739285] envoy[914]: conn peer=[2001:db8:1::1]:41510 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1945 ms=18144998lb # [ 46.744210] envoy[914]: conn peer=[2001:db8:1::1]:41522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=184999worker2 # [ 46.758362] nix-grpc-daemon[785]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:412465000client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5001worker2 # [ 46.764604] nix-grpc-daemon[785]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412465002worker2 # [ 46.769671] nix-grpc-daemon[877]: time=2026-09-21T09:51:34.833Z level=INFO msg="All 1 paths already cached"5003worker1 # [ 46.765896] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05004worker1 # [ 46.772394] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15005worker1 # [ 46.778331] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357345006worker1 # [ 46.790617] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357345007client # copying 1 paths to 'lb:50051'...5008client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5009worker1 # [ 46.808842] nix-grpc-daemon[779]: ts=2026-09-21T09:51:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:357345010worker1 # [ 46.821091] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5011worker1 # [ 46.835950] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)5012worker1 # [ 46.853741] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 2/5)5013worker1 # [ 46.892623] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 281 ms (attempt 3/5)5014worker1 # [ 47.175766] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 465 ms (attempt 4/5)5015worker1 # [ 47.643893] nix-grpc-daemon[779]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5016worker1 # [ 47.648154] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5017worker1 # [ 47.653302] nix-grpc-daemon[779]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5018worker1 # [ 47.772108] nix-daemon[857]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5019worker1 # [ 47.788802] nix-daemon[857]: reaped child process 1226, status = succeeded5020worker1 # [ 47.868893] nix-daemon[857]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5021worker1 # [ 47.885329] nix-daemon[857]: reaped child process 1231, status = succeeded5022lb # [ 47.906769] niks3-server[957]: 2026/09/21 09:51:35 INFO Received uploads request method=POST path=/api/pending_closures5023worker1 # [ 47.904463] nix-grpc-daemon[872]: time=2026-09-21T09:51:35.908Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5024worker1 # [ 47.905973] nix-grpc-daemon[872]: time=2026-09-21T09:51:35.908Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5025lb # [ 47.936170] niks3-server[957]: 2026/09/21 09:51:35 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5026lb # [ 47.947638] niks3-server[957]: 2026/09/21 09:51:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5027worker1 # [ 47.939573] nix-grpc-daemon[872]: time=2026-09-21T09:51:35.944Z level=INFO msg="Uploading 1 narinfos"5028lb # [ 47.954703] niks3-server[957]: 2026/09/21 09:51:35 INFO Signed narinfos id=21 count=15029lb # [ 47.962098] niks3-server[957]: 2026/09/21 09:51:35 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5030lb # [ 47.971713] niks3-server[957]: 2026/09/21 09:51:35 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5031worker1 # [ 47.962872] nix-grpc-daemon[872]: time=2026-09-21T09:51:35.967Z level=INFO msg="Upload complete. (289ms)"5032lb # [ 47.977921] niks3-server[957]: 2026/09/21 09:51:35 INFO Completed upload id=215033worker1 # [ 47.964890] nix-grpc-daemon[779]: ts=2026-09-21T09:51:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 paths=1 nar_bytes_in=5605034lb # [ 47.986376] niks3-server[957]: 2026/09/21 09:51:35 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5035worker1 # [ 47.970092] nix-grpc-daemon[779]: ts=2026-09-21T09:51:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:357345036worker1 # [ 47.978306] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5037worker1 # [ 47.986183] nix-daemon[857]: reaped child process 1216, status = succeeded5038worker1 # [ 47.990132] nix-daemon[857]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5039client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5040client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5041client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5042worker1 # [ 49.225436] nix-daemon[857]: reaped child process 1235, status = succeeded5043worker1 # [ 49.303971] nix-daemon[857]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5044worker1 # [ 49.317880] nix-daemon[857]: reaped child process 1254, status = succeeded5045lb # [ 49.340901] niks3-server[957]: 2026/09/21 09:51:36 INFO Received uploads request method=POST path=/api/pending_closures5046worker1 # [ 49.348832] nix-grpc-daemon[872]: time=2026-09-21T09:51:37.353Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5047lb # [ 49.387148] niks3-server[957]: 2026/09/21 09:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5048lb # [ 49.395195] niks3-server[957]: 2026/09/21 09:51:36 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5049lb # [ 49.401605] niks3-server[957]: 2026/09/21 09:51:36 INFO Signed narinfos id=22 count=15050worker1 # [ 49.386334] nix-grpc-daemon[872]: time=2026-09-21T09:51:37.391Z level=INFO msg="Uploading 1 narinfos"5051lb # [ 49.408361] niks3-server[957]: 2026/09/21 09:51:36 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5052lb # [ 49.419879] niks3-server[957]: 2026/09/21 09:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5053lb # [ 49.425810] niks3-server[957]: 2026/09/21 09:51:36 INFO Completed upload id=225054worker1 # [ 49.410786] nix-grpc-daemon[872]: time=2026-09-21T09:51:37.415Z level=INFO msg="Upload complete. (211ms)"5055lb # [ 49.429243] niks3-server[957]: 2026/09/21 09:51:36 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5056worker1 # [ 49.418623] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15057worker1 # [ 49.432944] nix-daemon[857]: reaped child process 1233, status = succeeded5058client # copying 1 paths...5059worker1 # [ 49.462499] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:357345060client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5061worker1 # [ 49.479278] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:357345062worker1 # [ 49.483595] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_out=1205063client # [ 49.612569] nix-daemon[946]: reaped child process 1053, status = succeeded5064client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.08 seconds)5065client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25066client # [ 49.715708] nix-daemon[946]: accepted connection from pid 1072, user root (trusted)5067client # this derivation will be built:5068client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5069worker1 # [ 49.809848] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:357345070client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5071worker1 # [ 49.816528] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:357345072worker1 # [ 49.822413] nix-grpc-daemon[872]: time=2026-09-21T09:51:37.827Z level=INFO msg="All 1 paths already cached"5073worker1 # [ 49.837681] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05074worker1 # [ 49.842608] nix-grpc-daemon[779]: ts=2026-09-21T09:51:37Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15075worker2 # [ 49.866097] nix-grpc-daemon[785]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412465076worker2 # [ 49.877691] nix-grpc-daemon[785]: ts=2026-09-21T09:51:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:412465077worker2 # [ 49.937869] nix-grpc-daemon[877]: time=2026-09-21T09:51:38.000Z level=INFO msg="All 1 paths already cached"5078client # copying 1 paths to 'lb:50051'...5079client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5080worker2 # [ 49.946395] nix-grpc-daemon[785]: ts=2026-09-21T09:51:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:412465081worker2 # [ 49.957173] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5082worker2 # [ 50.065890] nix-daemon[849]: accepted connection from pid 994, user nix-grpc-daemon (trusted)5083worker2 # [ 50.078414] nix-daemon[849]: reaped child process 997, status = succeeded5084worker2 # [ 50.163071] nix-daemon[849]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5085worker2 # [ 50.174557] nix-daemon[849]: reaped child process 1002, status = succeeded5086lb # [ 50.179639] niks3-server[957]: 2026/09/21 09:51:37 INFO Received uploads request method=POST path=/api/pending_closures5087worker2 # [ 50.193039] nix-grpc-daemon[877]: time=2026-09-21T09:51:38.256Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5088worker2 # [ 50.194431] nix-grpc-daemon[877]: time=2026-09-21T09:51:38.256Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5089lb # [ 50.210062] niks3-server[957]: 2026/09/21 09:51:37 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5090lb # [ 50.219404] niks3-server[957]: 2026/09/21 09:51:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5091worker2 # [ 50.228582] nix-grpc-daemon[877]: time=2026-09-21T09:51:38.291Z level=INFO msg="Uploading 1 narinfos"5092lb # [ 50.224921] niks3-server[957]: 2026/09/21 09:51:37 INFO Signed narinfos id=23 count=15093lb # [ 50.230647] niks3-server[957]: 2026/09/21 09:51:37 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5094lb # [ 50.243242] niks3-server[957]: 2026/09/21 09:51:37 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5095lb # [ 50.250417] niks3-server[957]: 2026/09/21 09:51:37 INFO Completed upload id=235096worker2 # [ 50.254188] nix-grpc-daemon[877]: time=2026-09-21T09:51:38.317Z level=INFO msg="Upload complete. (285ms)"5097worker2 # [ 50.257432] nix-grpc-daemon[785]: ts=2026-09-21T09:51:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=0 paths=1 nar_bytes_in=5605098lb # [ 50.256976] niks3-server[957]: 2026/09/21 09:51:37 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5099worker2 # [ 50.263316] nix-grpc-daemon[785]: ts=2026-09-21T09:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:412465100worker2 # [ 50.271878] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5101worker2 # [ 50.277460] nix-daemon[849]: reaped child process 992, status = succeeded5102worker2 # [ 50.282497] nix-daemon[849]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5103client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5104client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5105client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5106worker2 # [ 51.117427] nix-daemon[849]: reaped child process 1006, status = succeeded5107worker2 # [ 51.201699] nix-daemon[849]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)5108worker2 # [ 51.212442] nix-daemon[849]: reaped child process 1025, status = succeeded5109lb # [ 51.218311] niks3-server[957]: 2026/09/21 09:51:38 INFO Received uploads request method=POST path=/api/pending_closures5110worker2 # [ 51.230722] nix-grpc-daemon[877]: time=2026-09-21T09:51:39.293Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5111lb # [ 51.252866] niks3-server[957]: 2026/09/21 09:51:38 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5112lb # [ 51.256419] niks3-server[957]: 2026/09/21 09:51:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5113lb # [ 51.262542] niks3-server[957]: 2026/09/21 09:51:38 INFO Signed narinfos id=24 count=15114worker2 # [ 51.269790] nix-grpc-daemon[877]: time=2026-09-21T09:51:39.333Z level=INFO msg="Uploading 1 narinfos"5115lb # [ 51.274358] niks3-server[957]: 2026/09/21 09:51:38 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5116lb # [ 51.285879] niks3-server[957]: 2026/09/21 09:51:38 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5117lb # [ 51.292758] niks3-server[957]: 2026/09/21 09:51:38 INFO Completed upload id=245118worker2 # [ 51.299344] nix-grpc-daemon[877]: time=2026-09-21T09:51:39.362Z level=INFO msg="Upload complete. (210ms)"5119lb # [ 51.296490] niks3-server[957]: 2026/09/21 09:51:38 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5120client # copying 1 paths...5121worker2 # [ 51.305578] nix-grpc-daemon[785]: ts=2026-09-21T09:51:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41246 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15122worker2 # [ 51.312609] nix-grpc-daemon[785]: ts=2026-09-21T09:51:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:412465123client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5124worker2 # [ 51.323228] nix-daemon[849]: reaped child process 1004, status = succeeded5125worker1 # [ 51.309271] nix-grpc-daemon[779]: ts=2026-09-21T09:51:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:357345126worker1 # [ 51.318448] nix-grpc-daemon[779]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5127worker1 # [ 51.331094] nix-grpc-daemon[779]: ts=2026-09-21T09:51:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35734 duration_s=0 paths=1 nar_bytes_out=1205128client # [ 51.466323] nix-daemon[946]: reaped child process 1075, status = succeeded5129client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.85 seconds)5130(finished: subtest: an input only one worker has still reaches the builder, in 7.61 seconds)5131subtest: upload whose reference the worker lacks is completed from the cache5132client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5133client # this derivation will be built:5134client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5135client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5136client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.36 seconds)5137client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5138client # this derivation will be built:5139client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5140client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5141client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.26 seconds)5142client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5143client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5144worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5145worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5146worker2 # [ 52.357301] systemd[1]: Stopping Nix worker-protocol over gRPC...5147worker2 # [ 52.914276] nix-grpc-daemon[785]: ts=2026-09-21T09:51:40Z level=info event=signal_exit5148worker1 # [ 54.189359] nix-grpc-daemon[779]: ts=2026-09-21T09:51:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:357285149worker1 # [ 54.192973] nix-grpc-daemon[779]: ts=2026-09-21T09:51:42Z level=info event=worker_gone addr=192.168.1.4:500515150worker2 # [ 54.208716] nix-grpc-daemon[785]: ts=2026-09-21T09:51:42Z level=info event=scheduler_restarting5151worker2 # [ 54.222628] nix-grpc-daemon[785]: ts=2026-09-21T09:51:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5152worker2 # [ 54.247706] nix-daemon[849]: reaped child process 851, status = succeeded5153worker2 # [ 55.000421] nix-grpc-daemon[785]: E0000 00:00:1789984303.063699 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5154worker2 # [ 55.005951] nix-grpc-daemon[785]: E0000 00:00:1789984303.068988 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5155worker2 # [ 55.547991] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5156worker2 # [ 55.551425] systemd[1]: Stopped Nix worker-protocol over gRPC.5157worker2 # [ 55.556171] systemd[1]: nix-grpc-daemon.service: Consumed 1.184s CPU time over 38.781s wall clock time, 56.4M memory peak, 40.1K incoming IP traffic, 76.3K outgoing IP traffic.5158worker2 # [ 55.567105] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5159worker2 # [ 55.571409] systemd[1]: Closed Nix worker-protocol over gRPC.5160worker2 # [ 55.574478] systemd[1]: nix-grpc-daemon.socket: Consumed 305us CPU time over 46.869s wall clock time, 256K memory peak, 50.3K incoming IP traffic, 45.8K outgoing IP traffic.5161lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5162lb: must succeed: curl -sf localhost:9901/clusters5163lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5164lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5165client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25166worker1 # [ 55.767143] nix-grpc-daemon[779]: ts=2026-09-21T09:51:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:357345167client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5168worker1 # [ 55.772774] nix-grpc-daemon[779]: ts=2026-09-21T09:51:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:357345169client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5170worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5171worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5172worker2 # [ 55.874483] systemd[1]: Listening on Nix worker-protocol over gRPC.5173worker1 # [ 55.904463] systemd[1]: Stopping Nix worker-protocol over gRPC...5174worker2 # [ 56.544917] systemd[1]: Starting scheduler-token.service...5175worker2 # [ 56.605286] systemd[1]: scheduler-token.service: Deactivated successfully.5176worker2 # [ 56.608366] systemd[1]: Finished scheduler-token.service.5177worker2 # [ 56.610582] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 63ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5178worker2 # [ 56.619136] systemd[1]: Starting Nix worker-protocol over gRPC...5179worker2 # [ 56.696682] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5180worker2 # [ 56.701044] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5181worker2 # [ 56.706053] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15182worker2 # [ 56.708831] nix-grpc-daemon[1049]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5183worker2 # [ 56.712219] nix-grpc-daemon[1049]: E0000 00:00:1789984304.766988 1049 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5184worker2 # [ 56.714475] nix-grpc-daemon[1049]: E0000 00:00:1789984304.769157 1049 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5185worker2 # [ 56.716583] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5186worker2 # [ 56.718152] systemd[1]: Started Nix worker-protocol over gRPC.5187worker2 # [ 56.724072] nix-grpc-daemon[1049]: E0000 00:00:1789984304.787047 1090 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5188lb # [ 56.725088] envoy[914]: conn peer=[2001:db8:1::1]:41532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535189lb # [ 56.729383] envoy[914]: conn peer=[2001:db8:1::1]:41530 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3758 tx=1898 ms=27965190lb # [ 56.732907] envoy[914]: conn peer=[2001:db8:1::1]:41542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3733 tx=1878 ms=15745191worker2 # [ 56.738540] nix-grpc-daemon[1049]: E0000 00:00:1789984304.801935 1090 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5192worker2 # [ 56.742139] nix-grpc-daemon[1049]: E0000 00:00:1789984304.804156 1090 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5193lb # [ 56.737847] envoy[914]: conn peer=[2001:db8:1::1]:41552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=745194lb # [ 56.741397] envoy[914]: conn peer=[2001:db8:1::4]:38624 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=9011 tx=774 ms=372495195lb # [ 56.745134] envoy[914]: conn peer=[2001:db8:1::1]:45060 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165196worker2 # [ 56.752839] nix-grpc-daemon[1049]: E0000 00:00:1789984304.816219 1090 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5197worker2 # [ 56.761845] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:44Z level=info event=scheduler_connected addr=lb:500515198worker1 # [ 56.745164] nix-grpc-daemon[779]: ts=2026-09-21T09:51:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:357285199worker1 # [ 56.749847] nix-grpc-daemon[779]: ts=2026-09-21T09:51:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05200worker1 # [ 56.863657] nix-grpc-daemon[779]: ts=2026-09-21T09:51:44Z level=info event=signal_exit5201worker1 # [ 57.187250] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_yield5202lb # [ 57.205405] niks3-server[957]: 2026/09/21 09:51:44 INFO lead: released remote=192.168.1.3:335165203worker2 # [ 57.209938] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_restarting addr=lb:500515204worker1 # [ 57.194374] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:357285205worker1 # [ 57.198814] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=worker_gone addr=192.168.1.4:500515206worker1 # [ 57.203466] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_restarting addr=lb:500515207worker1 # [ 57.206470] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:357285208worker1 # [ 57.209822] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=worker_gone addr=192.168.1.3:500515209worker2 # [ 57.411709] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515210worker2 # [ 57.415832] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5211worker1 # [ 57.399911] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515212worker1 # [ 57.403969] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5213worker2 # [ 57.617229] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515214worker2 # [ 57.621417] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5215worker1 # [ 57.604731] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515216worker1 # [ 57.608809] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker2 # [ 57.823422] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 57.826970] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5219worker1 # [ 57.809787] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_connected addr=lb:500515220worker1 # [ 57.813754] nix-grpc-daemon[779]: ts=2026-09-21T09:51:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 58.028431] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515222worker2 # [ 58.032323] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5223worker1 # [ 58.016349] nix-grpc-daemon[779]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515224worker1 # [ 58.019586] nix-grpc-daemon[779]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5225worker1 # [ 58.191955] nix-grpc-daemon[779]: ts=2026-09-21T09:51:46Z level=info event=scheduler_restarting5226worker2 # [ 58.233343] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515227worker2 # [ 58.237194] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5228worker1 # [ 58.234310] nix-grpc-daemon[779]: ts=2026-09-21T09:51:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5229worker1 # [ 58.238918] nix-daemon[857]: reaped child process 859, status = succeeded5230worker2 # [ 58.438633] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515231worker2 # [ 58.442629] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232worker2 # [ 58.643887] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515233worker2 # [ 58.647671] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5234worker2 # [ 58.850331] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_connected addr=lb:500515235worker2 # [ 58.853450] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5236worker1 # [ 58.956932] nix-grpc-daemon[779]: E0000 00:00:1789984306.961412 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5237worker1 # [ 58.962167] nix-grpc-daemon[779]: E0000 00:00:1789984306.961561 824 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5238worker2 # [ 59.051620] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 59.055506] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker2 # [ 59.256842] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_connected addr=lb:500515241worker2 # [ 59.261244] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker2 # [ 59.462108] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_connected addr=lb:500515243worker2 # [ 59.465915] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5244worker1 # [ 59.486558] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5245worker1 # [ 59.491451] systemd[1]: Stopped Nix worker-protocol over gRPC.5246worker1 # [ 59.494341] systemd[1]: nix-grpc-daemon.service: Consumed 2.631s CPU time over 42.784s wall clock time, 69.1M memory peak, 89.2K incoming IP traffic, 198.3K outgoing IP traffic.5247worker1 # [ 59.505224] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5248worker1 # [ 59.509608] systemd[1]: Closed Nix worker-protocol over gRPC.5249worker1 # [ 59.512853] systemd[1]: nix-grpc-daemon.socket: Consumed 295us CPU time over 51.791s wall clock time, 256K memory peak, 87.3K incoming IP traffic, 71K outgoing IP traffic.5250lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5251lb: must succeed: curl -sf localhost:9901/clusters5252lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5253lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5254client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25255worker2 # [ 59.666962] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_connected addr=lb:500515256worker2 # [ 59.670842] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5257worker2 # [ 59.740190] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245258worker2 # [ 59.753202] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5259worker2 # [ 59.769173] nix-grpc-daemon[1049]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5260worker2 # [ 59.781303] nix-grpc-daemon[1049]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5261worker2 # [ 59.834712] nix-grpc-daemon[1049]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 399 ms (attempt 3/5)5262worker2 # [ 59.872552] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_connected addr=lb:500515263worker2 # [ 59.876271] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 60.077394] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_connected addr=lb:500515265worker2 # [ 60.081279] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5266worker2 # [ 60.235873] nix-grpc-daemon[1049]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 538 ms (attempt 4/5)5267worker2 # [ 60.282325] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_connected addr=lb:500515268worker2 # [ 60.286129] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5269worker2 # [ 60.487277] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_connected addr=lb:500515270worker2 # [ 60.491163] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5271worker2 # [ 60.692438] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_connected addr=lb:500515272worker2 # [ 60.696264] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker2 # [ 60.776176] nix-grpc-daemon[1049]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5274worker2 # [ 60.779790] nix-grpc-daemon[1049]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5275client # copying 2 paths...5276client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5277worker2 # [ 60.802537] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384245278client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5279worker2 # [ 60.813803] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5280worker2 # [ 60.838111] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=niks3_push_started pid=11095281worker2 # [ 60.897445] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_connected addr=lb:500515282worker2 # [ 60.899734] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5283worker2 # [ 60.941615] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.004Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5284worker2 # [ 61.023067] nix-daemon[849]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)5285worker2 # [ 61.035344] nix-daemon[849]: reaped child process 1125, status = succeeded5286lb # [ 61.039862] niks3-server[957]: 2026/09/21 09:51:48 INFO Received uploads request method=POST path=/api/pending_closures5287lb # [ 61.051183] niks3-server[957]: 2026/09/21 09:51:48 INFO Received uploads request method=POST path=/api/pending_closures5288worker2 # [ 61.062470] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.125Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5289worker2 # [ 61.063963] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.127Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5290worker2 # [ 61.065766] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.129Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5291worker2 # [ 61.099865] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=scheduler_connected addr=lb:500515292worker2 # [ 61.101875] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5293lb # [ 61.099565] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5294lb # [ 61.113982] niks3-server[957]: 2026/09/21 09:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5295lb # [ 61.120137] niks3-server[957]: 2026/09/21 09:51:48 INFO Signed narinfos id=25 count=15296lb # [ 61.127034] niks3-server[957]: 2026/09/21 09:51:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5297lb # [ 61.128542] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5298lb # [ 61.131099] niks3-server[957]: 2026/09/21 09:51:48 INFO Signed narinfos id=26 count=25299lb # [ 61.134534] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5300worker2 # [ 61.138156] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.200Z level=INFO msg="Uploading 3 narinfos"5301lb # [ 61.137057] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5302lb # [ 61.150390] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5303lb # [ 61.153893] niks3-server[957]: 2026/09/21 09:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5304lb # [ 61.156201] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5305lb # [ 61.158754] niks3-server[957]: 2026/09/21 09:51:48 INFO Completed upload id=265306lb # [ 61.160933] niks3-server[957]: 2026/09/21 09:51:48 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5307lb # [ 61.163641] niks3-server[957]: 2026/09/21 09:51:48 INFO Completed upload id=255308worker2 # [ 61.167708] nix-grpc-daemon[1109]: time=2026-09-21T09:51:49.231Z level=INFO msg="Upload complete. (223ms)"5309lb # [ 61.165826] niks3-server[957]: 2026/09/21 09:51:48 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5310worker2 # [ 61.172704] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=0 paths=2 nar_bytes_in=3125311worker2 # [ 61.184915] nix-daemon[849]: reaped child process 1107, status = succeeded5312client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.61 seconds)5313worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5314worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5315worker1 # [ 61.251640] systemd[1]: Listening on Nix worker-protocol over gRPC.5316worker1 # [ 61.258843] systemd[1]: Starting Nix worker-protocol over gRPC...5317worker2 # [ 61.303202] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=scheduler_connected addr=lb:500515318worker2 # [ 61.307203] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker1 # [ 61.346983] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5320worker1 # [ 61.351480] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5321worker1 # [ 61.354219] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15322worker1 # [ 61.358199] nix-grpc-daemon[1280]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5323worker1 # [ 61.359583] nix-grpc-daemon[1280]: E0000 00:00:1789984309.361188 1280 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5324worker1 # [ 61.361713] nix-grpc-daemon[1280]: E0000 00:00:1789984309.362773 1280 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5325worker1 # [ 61.369503] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5326lb # [ 61.385672] niks3-server[957]: 2026/09/21 09:51:48 INFO lead: acquired remote=192.168.1.3:527085327worker1 # [ 61.372369] systemd[1]: Started Nix worker-protocol over gRPC.5328worker1 # [ 61.376130] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=scheduler_take_over5329worker1 # [ 61.378092] nix-grpc-daemon[1280]: E0000 00:00:1789984309.376496 1319 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5330worker1 # [ 61.388094] nix-grpc-daemon[1280]: E0000 00:00:1789984309.391894 1319 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5331worker1 # [ 61.390194] nix-grpc-daemon[1280]: E0000 00:00:1789984309.392057 1319 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5332lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5333lb: must succeed: curl -sf localhost:9901/clusters5334worker1 # [ 61.402233] nix-grpc-daemon[1280]: E0000 00:00:1789984309.406916 1319 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5335worker1 # [ 61.413853] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=scheduler_connected addr=lb:500515336worker1 # [ 61.415445] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5338worker2 # [ 61.508282] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:49Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 61.498880] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=oidc_jwks provider=mock keys=15340worker1 # [ 61.501716] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:478325341worker1 # [ 61.504919] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05342worker1 # [ 61.616601] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 61.621698] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:478325344worker1 # [ 61.625313] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05345lb: must succeed: curl -sf localhost:9901/clusters5346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5347lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.09 seconds)5348(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.04 seconds)5349subtest: scheduler down: the other node takes the lock and keeps it5350lb: must succeed: curl -sf localhost:9901/clusters5351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5352lb: must succeed: curl -sf localhost:9901/clusters5353lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5354lb: must succeed: curl -sf localhost:9901/clusters5355lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5356lb: must succeed: curl -sf localhost:9901/clusters5357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5358worker1 # [ 62.617581] systemd[1]: Stopping Nix worker-protocol over gRPC...5359worker1 # [ 63.367252] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=signal_exit5360worker1 # [ 63.374126] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_yield5361lb # [ 63.392964] niks3-server[957]: 2026/09/21 09:51:50 INFO lead: released remote=192.168.1.3:527085362worker1 # [ 63.381742] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_restarting addr=lb:500515363worker1 # [ 63.386198] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:478325364worker2 # [ 63.406997] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:51Z level=info event=scheduler_restarting addr=lb:500515365worker1 # [ 63.390419] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=worker_gone addr=192.168.1.3:500515366worker1 # [ 63.393580] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:478325367worker1 # [ 63.397082] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=worker_gone addr=192.168.1.4:500515368worker1 # [ 63.582845] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_connected addr=lb:500515369worker1 # [ 63.587107] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370worker2 # [ 63.607459] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:51Z level=info event=scheduler_connected addr=lb:500515371worker2 # [ 63.611261] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker1 # [ 63.787646] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 63.810233] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:51Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 63.791788] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker2 # [ 63.813530] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 64.011602] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515377worker1 # [ 63.992715] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:51Z level=info event=scheduler_connected addr=lb:500515378worker2 # [ 64.015682] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker1 # [ 63.997126] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker2 # [ 64.216810] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515381worker1 # [ 64.198156] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515382worker2 # [ 64.220937] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker1 # [ 64.202371] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 64.381642] nix-grpc-daemon[1280]: ts=2026-09-21T09:51:52Z level=info event=scheduler_restarting5385worker2 # [ 64.423433] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515386worker2 # [ 64.426628] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker1 # [ 64.493259] nix-grpc-daemon[1280]: E0000 00:00:1789984312.497570 1325 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5388worker1 # [ 64.498464] nix-grpc-daemon[1280]: E0000 00:00:1789984312.498169 1325 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5389worker2 # [ 64.624804] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515390worker2 # [ 64.628735] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker2 # [ 64.829707] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_connected addr=lb:500515392worker2 # [ 64.833665] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5393worker2 # [ 65.034763] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_connected addr=lb:500515394worker2 # [ 65.038883] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5395worker2 # [ 65.240122] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_connected addr=lb:500515396worker2 # [ 65.243997] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5397worker1 # [ 65.372427] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5398worker1 # [ 65.376980] systemd[1]: Stopped Nix worker-protocol over gRPC.5399worker1 # [ 65.381753] systemd[1]: nix-grpc-daemon.service: Consumed 154ms CPU time over 4.118s wall clock time, 12.6M memory peak, 9.5K incoming IP traffic, 8.7K outgoing IP traffic.5400worker1 # [ 65.393177] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5401worker1 # [ 65.397946] systemd[1]: Closed Nix worker-protocol over gRPC.5402worker1 # [ 65.401914] systemd[1]: nix-grpc-daemon.socket: Consumed 520us CPU time over 4.150s wall clock time, 256K memory peak, 13.6K incoming IP traffic, 10.4K outgoing IP traffic.5403worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5404worker2 # [ 65.446189] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_connected addr=lb:500515405worker2 # [ 65.451467] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5406worker2 # [ 65.648870] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_connected addr=lb:500515407worker2 # [ 65.652854] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5408worker2 # [ 65.853948] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_connected addr=lb:500515409worker2 # [ 65.858220] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5410worker2 # [ 66.059212] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_connected addr=lb:500515411worker2 # [ 66.063154] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5412worker2 # [ 66.264278] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_connected addr=lb:500515413worker2 # [ 66.268222] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker2 # [ 66.470179] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 66.473978] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker2 # [ 66.675178] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_connected addr=lb:500515417worker2 # [ 66.678995] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418lb # [ 66.721276] envoy[914]: conn peer=[2001:db8:1::3]:39782 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4902 tx=1395 ms=413175419worker2 # [ 66.730592] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_take_over5420lb # [ 66.726731] envoy[914]: conn peer=[2001:db8:1::1]:45070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1009 tx=282 ms=14455421lb # [ 66.732149] envoy[914]: conn peer=[2001:db8:1::3]:45662 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1547 tx=400 ms=29975422lb # [ 66.736494] niks3-server[957]: 2026/09/21 09:51:54 INFO lead: acquired remote=192.168.1.4:516385423worker2 # [ 66.880377] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=scheduler_connected addr=lb:500515424worker2 # [ 66.899926] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=oidc_jwks provider=mock keys=15425worker2 # [ 66.903337] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:463405426worker2 # [ 66.907200] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05427worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 2.16 seconds)5428worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5429worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5430client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25431worker2 # [ 67.763438] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:384245432client # don't know how to build these paths:5433client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5434worker2 # [ 67.774432] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245435worker2 # [ 67.797726] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05436worker2 # [ 67.802820] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15437worker2 # [ 67.841875] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245438worker2 # [ 67.854951] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245439client # copying 1 paths to 'lb:50051'...5440client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5441worker2 # [ 67.867528] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384245442worker2 # [ 67.878389] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5443worker2 # [ 67.976198] nix-daemon[849]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5444worker2 # [ 67.988629] nix-daemon[849]: reaped child process 1158, status = succeeded5445worker2 # [ 68.067703] nix-daemon[849]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5446worker2 # [ 68.079418] nix-daemon[849]: reaped child process 1163, status = succeeded5447lb # [ 68.083642] niks3-server[957]: 2026/09/21 09:51:55 INFO Received uploads request method=POST path=/api/pending_closures5448worker2 # [ 68.095945] nix-grpc-daemon[1109]: time=2026-09-21T09:51:56.158Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5449worker2 # [ 68.097348] nix-grpc-daemon[1109]: time=2026-09-21T09:51:56.158Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5450lb # [ 68.118279] niks3-server[957]: 2026/09/21 09:51:55 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5451lb # [ 68.126454] niks3-server[957]: 2026/09/21 09:51:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5452worker2 # [ 68.136765] nix-grpc-daemon[1109]: time=2026-09-21T09:51:56.200Z level=INFO msg="Uploading 1 narinfos"5453lb # [ 68.132909] niks3-server[957]: 2026/09/21 09:51:55 INFO Signed narinfos id=27 count=15454lb # [ 68.141322] niks3-server[957]: 2026/09/21 09:51:55 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5455lb # [ 68.150239] niks3-server[957]: 2026/09/21 09:51:55 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5456lb # [ 68.164495] niks3-server[957]: 2026/09/21 09:51:55 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5457lb # [ 68.170493] niks3-server[957]: 2026/09/21 09:51:55 INFO Completed upload id=275458worker2 # [ 68.174466] nix-grpc-daemon[1109]: time=2026-09-21T09:51:56.237Z level=INFO msg="Upload complete. (284ms)"5459worker2 # [ 68.179340] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=0 paths=1 nar_bytes_in=5205460worker2 # [ 68.185401] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245461worker2 # [ 68.196237] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5462worker2 # [ 68.206500] nix-daemon[849]: reaped child process 1153, status = succeeded5463client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5464worker2 # [ 68.213856] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5465client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5466client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5467client # farm-a> LOG-farm-a-on-standby5468worker2 # [ 69.451723] nix-daemon[849]: reaped child process 1167, status = succeeded5469worker2 # [ 69.531238] nix-daemon[849]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5470worker2 # [ 69.542395] nix-daemon[849]: reaped child process 1186, status = succeeded5471lb # [ 69.547217] niks3-server[957]: 2026/09/21 09:51:56 INFO Received uploads request method=POST path=/api/pending_closures5472worker2 # [ 69.563189] nix-grpc-daemon[1109]: time=2026-09-21T09:51:57.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5473worker2 # [ 69.564652] nix-grpc-daemon[1109]: time=2026-09-21T09:51:57.628Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5474lb # [ 69.591254] niks3-server[957]: 2026/09/21 09:51:56 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5475lb # [ 69.598605] niks3-server[957]: 2026/09/21 09:51:56 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5476lb # [ 69.608120] niks3-server[957]: 2026/09/21 09:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5477lb # [ 69.612114] niks3-server[957]: 2026/09/21 09:51:56 INFO Signed narinfos id=28 count=15478worker2 # [ 69.619167] nix-grpc-daemon[1109]: time=2026-09-21T09:51:57.681Z level=INFO msg="Uploading 1 narinfos"5479lb # [ 69.617825] niks3-server[957]: 2026/09/21 09:51:56 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5480lb # [ 69.624531] niks3-server[957]: 2026/09/21 09:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5481lb # [ 69.629872] niks3-server[957]: 2026/09/21 09:51:56 INFO Completed upload id=285482lb # [ 69.631636] niks3-server[957]: 2026/09/21 09:51:57 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5483worker2 # [ 69.635275] nix-grpc-daemon[1109]: time=2026-09-21T09:51:57.698Z level=INFO msg="Upload complete. (212ms)"5484worker2 # [ 69.650729] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15485worker2 # [ 69.655613] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05486worker2 # [ 69.659180] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=15487worker2 # [ 69.668611] nix-daemon[849]: reaped child process 1165, status = succeeded5488worker2 # [ 69.694549] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245489worker2 # [ 69.707131] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245490worker2 # [ 69.721637] nix-grpc-daemon[1109]: time=2026-09-21T09:51:57.784Z level=INFO msg="All 1 paths already cached"5491client # copying 1 paths to 'lb:50051'...5492client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5493worker2 # [ 69.730502] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384245494worker2 # [ 69.738889] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5495worker2 # [ 69.833041] nix-daemon[849]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5496worker2 # [ 69.846637] nix-daemon[849]: reaped child process 1195, status = succeeded5497worker2 # [ 69.924530] nix-daemon[849]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5498worker2 # [ 69.936737] nix-daemon[849]: reaped child process 1200, status = succeeded5499lb # [ 69.939613] niks3-server[957]: 2026/09/21 09:51:57 INFO Received uploads request method=POST path=/api/pending_closures5500worker2 # [ 69.951513] nix-grpc-daemon[1109]: time=2026-09-21T09:51:58.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5501worker2 # [ 69.953439] nix-grpc-daemon[1109]: time=2026-09-21T09:51:58.016Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5502lb # [ 69.970145] niks3-server[957]: 2026/09/21 09:51:57 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5503lb # [ 69.979874] niks3-server[957]: 2026/09/21 09:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5504lb # [ 69.985172] niks3-server[957]: 2026/09/21 09:51:57 INFO Signed narinfos id=29 count=15505worker2 # [ 69.992091] nix-grpc-daemon[1109]: time=2026-09-21T09:51:58.055Z level=INFO msg="Uploading 1 narinfos"5506lb # [ 69.992120] niks3-server[957]: 2026/09/21 09:51:57 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5507lb # [ 70.003447] niks3-server[957]: 2026/09/21 09:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5508worker2 # [ 70.013412] nix-grpc-daemon[1109]: time=2026-09-21T09:51:58.076Z level=INFO msg="Upload complete. (262ms)"5509lb # [ 70.009506] niks3-server[957]: 2026/09/21 09:51:57 INFO Completed upload id=295510worker2 # [ 70.015869] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=0 paths=1 nar_bytes_in=6405511lb # [ 70.015307] niks3-server[957]: 2026/09/21 09:51:57 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5512worker2 # [ 70.021267] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245513worker2 # [ 70.030471] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5514worker2 # [ 70.035795] nix-daemon[849]: reaped child process 1190, status = succeeded5515worker2 # [ 70.040646] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5516client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5517client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5518client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5519client # farm-b> LOG-farm-b-on-standby5520worker2 # [ 71.281524] nix-daemon[849]: reaped child process 1204, status = succeeded5521worker2 # [ 71.354134] nix-daemon[849]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5522worker2 # [ 71.366433] nix-daemon[849]: reaped child process 1223, status = succeeded5523lb # [ 71.370252] niks3-server[957]: 2026/09/21 09:51:58 INFO Received uploads request method=POST path=/api/pending_closures5524worker2 # [ 71.382739] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.445Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5525worker2 # [ 71.384299] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.447Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5526lb # [ 71.411595] niks3-server[957]: 2026/09/21 09:51:58 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5527lb # [ 71.419496] niks3-server[957]: 2026/09/21 09:51:58 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5528lb # [ 71.431104] niks3-server[957]: 2026/09/21 09:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5529lb # [ 71.437518] niks3-server[957]: 2026/09/21 09:51:58 INFO Signed narinfos id=30 count=15530worker2 # [ 71.443958] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.507Z level=INFO msg="Uploading 1 narinfos"5531lb # [ 71.441295] niks3-server[957]: 2026/09/21 09:51:58 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5532lb # [ 71.452113] niks3-server[957]: 2026/09/21 09:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5533lb # [ 71.455285] niks3-server[957]: 2026/09/21 09:51:58 INFO Completed upload id=305534worker2 # [ 71.459351] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.522Z level=INFO msg="Upload complete. (211ms)"5535lb # [ 71.460064] niks3-server[957]: 2026/09/21 09:51:58 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5536worker2 # [ 71.465512] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15537worker2 # [ 71.478663] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05538worker2 # [ 71.482500] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=15539worker2 # [ 71.491640] nix-daemon[849]: reaped child process 1202, status = succeeded5540worker2 # [ 71.523199] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245541worker2 # [ 71.535936] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245542worker2 # [ 71.549178] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.612Z level=INFO msg="All 2 paths already cached"5543client # copying 1 paths to 'lb:50051'...5544client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5545worker2 # [ 71.557365] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384245546worker2 # [ 71.565590] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5547worker2 # [ 71.659692] nix-daemon[849]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5548worker2 # [ 71.672102] nix-daemon[849]: reaped child process 1232, status = succeeded5549worker2 # [ 71.752294] nix-daemon[849]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5550worker2 # [ 71.764735] nix-daemon[849]: reaped child process 1237, status = succeeded5551lb # [ 71.767412] niks3-server[957]: 2026/09/21 09:51:59 INFO Received uploads request method=POST path=/api/pending_closures5552worker2 # [ 71.780645] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.844Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5553worker2 # [ 71.782181] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.845Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5554lb # [ 71.799780] niks3-server[957]: 2026/09/21 09:51:59 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5555lb # [ 71.809068] niks3-server[957]: 2026/09/21 09:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5556lb # [ 71.814289] niks3-server[957]: 2026/09/21 09:51:59 INFO Signed narinfos id=31 count=15557worker2 # [ 71.821133] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.884Z level=INFO msg="Uploading 1 narinfos"5558lb # [ 71.827972] niks3-server[957]: 2026/09/21 09:51:59 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5559lb # [ 71.836100] niks3-server[957]: 2026/09/21 09:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5560lb # [ 71.843230] niks3-server[957]: 2026/09/21 09:51:59 INFO Completed upload id=315561worker2 # [ 71.847352] nix-grpc-daemon[1109]: time=2026-09-21T09:51:59.910Z level=INFO msg="Upload complete. (269ms)"5562lb # [ 71.845391] niks3-server[957]: 2026/09/21 09:51:59 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5563worker2 # [ 71.851118] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=0 paths=1 nar_bytes_in=7685564worker2 # [ 71.857518] nix-grpc-daemon[1049]: ts=2026-09-21T09:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245565worker2 # [ 71.866106] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5566worker2 # [ 71.874351] nix-daemon[849]: reaped child process 1227, status = succeeded5567worker2 # [ 71.879446] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5568client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5569client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5570client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5571client # farm-top> LOG-farm-top-on-standby5572worker2 # [ 72.716469] nix-daemon[849]: reaped child process 1241, status = succeeded5573worker2 # [ 72.796488] nix-daemon[849]: accepted connection from pid 1257, user nix-grpc-daemon (trusted)5574worker2 # [ 72.809546] nix-daemon[849]: reaped child process 1260, status = succeeded5575lb # [ 72.812509] niks3-server[957]: 2026/09/21 09:52:00 INFO Received uploads request method=POST path=/api/pending_closures5576worker2 # [ 72.825565] nix-grpc-daemon[1109]: time=2026-09-21T09:52:00.888Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5577worker2 # [ 72.827044] nix-grpc-daemon[1109]: time=2026-09-21T09:52:00.890Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5578lb # [ 72.851189] niks3-server[957]: 2026/09/21 09:52:00 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5579lb # [ 72.858265] niks3-server[957]: 2026/09/21 09:52:00 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5580lb # [ 72.871664] niks3-server[957]: 2026/09/21 09:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5581lb # [ 72.876645] niks3-server[957]: 2026/09/21 09:52:00 INFO Signed narinfos id=32 count=15582worker2 # [ 72.880502] nix-grpc-daemon[1109]: time=2026-09-21T09:52:00.943Z level=INFO msg="Uploading 1 narinfos"5583lb # [ 72.884085] niks3-server[957]: 2026/09/21 09:52:00 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5584lb # [ 72.891140] niks3-server[957]: 2026/09/21 09:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5585lb # [ 72.901814] niks3-server[957]: 2026/09/21 09:52:00 INFO Completed upload id=325586worker2 # [ 72.905340] nix-grpc-daemon[1109]: time=2026-09-21T09:52:00.968Z level=INFO msg="Upload complete. (219ms)"5587lb # [ 72.907812] niks3-server[957]: 2026/09/21 09:52:00 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5588worker2 # [ 72.912487] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15589worker2 # [ 72.927115] nix-daemon[849]: reaped child process 1239, status = succeeded5590client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.33 seconds)5591client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath5592client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)5593worker1 # [ 73.096716] systemd[1]: Listening on Nix worker-protocol over gRPC.5594worker1 # [ 73.105454] systemd[1]: Starting Nix worker-protocol over gRPC...5595worker1 # [ 73.197232] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5596worker1 # [ 73.199326] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5597worker1 # [ 73.202789] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15598worker1 # [ 73.208054] nix-grpc-daemon[1347]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5599worker1 # [ 73.210855] nix-grpc-daemon[1347]: E0000 00:00:1789984321.212084 1347 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5600worker1 # [ 73.215484] nix-grpc-daemon[1347]: E0000 00:00:1789984321.215344 1347 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5601worker1 # [ 73.221401] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5602worker1 # [ 73.225140] systemd[1]: Started Nix worker-protocol over gRPC.5603worker1 # [ 73.229207] nix-grpc-daemon[1347]: E0000 00:00:1789984321.228667 1388 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5604worker1 # [ 73.240059] nix-grpc-daemon[1347]: E0000 00:00:1789984321.243754 1388 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5605worker1 # [ 73.243229] nix-grpc-daemon[1347]: E0000 00:00:1789984321.243991 1388 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5606lb: must succeed: curl -sf localhost:9901/clusters5607worker1 # [ 73.255060] nix-grpc-daemon[1347]: E0000 00:00:1789984321.259717 1388 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5608worker1 # [ 73.267649] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=info event=scheduler_connected addr=lb:500515609worker2 # [ 73.287676] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:463405610worker2 # [ 73.292260] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5612worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5613worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5618worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5619worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5620(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.97 seconds)5621subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5622client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs5623client # [ 73.530510] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.5624client # Running as unit: rs.service; invocation ID: f7ca0d0ef4e5488bb0b4817e0b9874a15625client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.07 seconds)5626worker1 # [ 73.642959] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:337005627worker1 # [ 73.656164] nix-daemon[857]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5628client # [ 73.691592] nix[1175]: don't know how to build these paths:5629client # [ 73.692538] nix[1175]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5630worker2 # [ 73.698266] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05631worker2 # [ 73.702329] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=15632worker2 # [ 73.752299] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245633worker2 # [ 73.764621] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384245634client # [ 73.778147] nix[1175]: copying 1 paths to 'lb:50051'...5635client # [ 73.779150] nix[1175]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5636worker2 # [ 73.775891] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384245637worker2 # [ 73.789133] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5638worker2 # [ 73.886314] nix-daemon[849]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5639worker2 # [ 73.899556] nix-daemon[849]: reaped child process 1276, status = succeeded5640worker2 # [ 73.978488] nix-daemon[849]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5641worker2 # [ 73.990349] nix-daemon[849]: reaped child process 1281, status = succeeded5642lb # [ 73.994652] niks3-server[957]: 2026/09/21 09:52:01 INFO Received uploads request method=POST path=/api/pending_closures5643worker2 # [ 74.007836] nix-grpc-daemon[1109]: time=2026-09-21T09:52:02.070Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5644worker2 # [ 74.009367] nix-grpc-daemon[1109]: time=2026-09-21T09:52:02.070Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5645lb # [ 74.026723] niks3-server[957]: 2026/09/21 09:52:01 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5646lb # [ 74.030114] niks3-server[957]: 2026/09/21 09:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5647worker2 # [ 74.039457] nix-grpc-daemon[1109]: time=2026-09-21T09:52:02.102Z level=INFO msg="Uploading 1 narinfos"5648lb # [ 74.035119] niks3-server[957]: 2026/09/21 09:52:01 INFO Signed narinfos id=33 count=15649lb # [ 74.049234] niks3-server[957]: 2026/09/21 09:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5650lb # [ 74.053278] niks3-server[957]: 2026/09/21 09:52:01 INFO Completed upload id=335651worker2 # [ 74.057138] nix-grpc-daemon[1109]: time=2026-09-21T09:52:02.120Z level=INFO msg="Upload complete. (256ms)"5652lb # [ 74.055071] niks3-server[957]: 2026/09/21 09:52:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5653worker2 # [ 74.061058] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=0 paths=1 nar_bytes_in=4165654worker2 # [ 74.066247] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384245655lb # [ 74.066215] niks3-server[957]: 2026/09/21 09:52:01 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5656worker2 # [ 74.070470] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5657worker2 # [ 74.072672] nix-daemon[849]: reaped child process 1271, status = succeeded5658worker2 # [ 74.077642] nix-daemon[849]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)5659client # [ 74.080784] nix[1175]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5660lb: must succeed: curl -sf localhost:9901/clusters5661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5662lb: must succeed: curl -sf localhost:9901/clusters5663lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5664lb: must succeed: curl -sf localhost:9901/clusters5665lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5666lb: must succeed: curl -sf localhost:9901/clusters5667lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5668worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5669worker1 # [ 76.164709] systemd-journald[283]: Received client request to rotate journal, rotating.5670worker1 # [ 76.185511] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/89e0bf61779042ad8141e5c7366dd85d.5671worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5672worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5673worker2 # [ 76.115383] systemd[1]: Stopping Nix worker-protocol over gRPC...5674worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)5675client: must succeed: sleep 5.05676worker2 # [ 76.713381] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:04Z level=info event=draining5677lb # [ 76.722677] envoy[914]: conn peer=[2001:db8:1::1]:55968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9573 tx=4068 ms=51675678client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5679worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5680worker2 # [ 81.520806] nix-daemon[849]: reaped child process 1285, status = succeeded5681worker2 # [ 81.604403] nix-daemon[849]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5682worker2 # [ 81.617254] nix-daemon[849]: reaped child process 1329, status = succeeded5683lb # [ 81.620512] niks3-server[957]: 2026/09/21 09:52:08 INFO Received uploads request method=POST path=/api/pending_closures5684worker2 # [ 81.632490] nix-grpc-daemon[1109]: time=2026-09-21T09:52:09.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5685worker2 # [ 81.634085] nix-grpc-daemon[1109]: time=2026-09-21T09:52:09.697Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5686lb # [ 81.665745] niks3-server[957]: 2026/09/21 09:52:09 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5687lb # [ 81.674294] niks3-server[957]: 2026/09/21 09:52:09 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5688lb # [ 81.684172] niks3-server[957]: 2026/09/21 09:52:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5689lb # [ 81.687929] niks3-server[957]: 2026/09/21 09:52:09 INFO Signed narinfos id=34 count=15690worker2 # [ 81.694587] nix-grpc-daemon[1109]: time=2026-09-21T09:52:09.757Z level=INFO msg="Uploading 1 narinfos"5691lb # [ 81.698211] niks3-server[957]: 2026/09/21 09:52:09 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5692lb # [ 81.702763] niks3-server[957]: 2026/09/21 09:52:09 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5693lb # [ 81.706779] niks3-server[957]: 2026/09/21 09:52:09 INFO Completed upload id=345694worker2 # [ 81.710844] nix-grpc-daemon[1109]: time=2026-09-21T09:52:09.774Z level=INFO msg="Upload complete. (219ms)"5695lb # [ 81.708597] niks3-server[957]: 2026/09/21 09:52:09 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5696worker2 # [ 81.720070] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38424 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15697worker2 # [ 81.734237] nix-daemon[849]: reaped child process 1283, status = succeeded5698client # [ 81.752205] systemd[1]: rs.service: Deactivated successfully.5699client # [ 81.755857] systemd[1]: rs.service: Consumed 127ms CPU time over 8.224s wall clock time, 43.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5700worker2 # [ 82.715366] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=signal_exit5701worker2 # [ 82.729889] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=scheduler_yield5702lb # [ 82.728400] niks3-server[957]: 2026/09/21 09:52:10 INFO lead: released remote=192.168.1.4:516385703worker1 # [ 82.714642] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:10Z level=info event=scheduler_restarting addr=lb:500515704worker2 # [ 82.736394] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:463405705worker2 # [ 82.742330] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=worker_gone addr=192.168.1.3:500515706worker2 # [ 82.745420] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=scheduler_restarting addr=lb:500515707worker2 # [ 82.748471] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:463405708worker2 # [ 82.751789] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:10Z level=info event=worker_gone addr=192.168.1.4:500515709worker1 # [ 82.915070] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:10Z level=info event=scheduler_connected addr=lb:500515710worker1 # [ 82.919888] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5711worker2 # [ 82.943369] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515712worker2 # [ 82.947254] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5713worker1 # [ 83.120589] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515714worker1 # [ 83.125330] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5715worker2 # [ 83.145479] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515716worker2 # [ 83.149264] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5717lb # [ 83.241842] niks3-server[957]: 2026/09/21 09:52:10 INFO lead: acquired remote=192.168.1.3:503965718worker1 # [ 83.230342] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_take_over5719worker1 # [ 83.326456] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515720worker2 # [ 83.347608] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515721worker1 # [ 83.331293] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5722worker2 # [ 83.351633] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5723worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)5724client: waiting for success: ! systemctl is-active rs5725client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)5726client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5727client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)5728client: must fail: journalctl -u rs | grep -q 'reconnecting'5729client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)5730lb: must succeed: curl -sf localhost:9901/clusters5731worker2 # [ 83.549923] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515732worker1 # [ 83.531951] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515733worker2 # [ 83.553995] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5734worker1 # [ 83.536588] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5736worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5737worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5738worker2 # [ 83.738268] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=scheduler_restarting5739worker1 # [ 83.738503] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515740worker1 # [ 83.743143] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5741worker2 # [ 83.768659] nix-grpc-daemon[1049]: ts=2026-09-21T09:52:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5742worker2 # [ 83.779917] nix-daemon[849]: reaped child process 1101, status = succeeded5743worker2 # [ 83.797921] nix-grpc-daemon[1049]: E0000 00:00:1789984331.861212 1094 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5744worker2 # [ 83.803412] nix-grpc-daemon[1049]: E0000 00:00:1789984331.866230 1094 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5745worker1 # [ 83.940701] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_connected addr=lb:500515746worker1 # [ 83.945363] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5747worker1 # [ 84.146318] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=scheduler_connected addr=lb:500515748worker1 # [ 84.150940] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5749worker1 # [ 84.352111] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=scheduler_connected addr=lb:500515750worker1 # [ 84.356685] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5751worker1 # [ 84.557595] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=scheduler_connected addr=lb:500515752worker1 # [ 84.565178] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:491505753worker1 # [ 84.568788] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05754lb: must succeed: curl -sf localhost:9901/clusters5755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5756worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5757worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5758worker2 # [ 85.399888] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5759worker2 # [ 85.403517] systemd[1]: Stopped Nix worker-protocol over gRPC.5760worker2 # [ 85.405650] systemd[1]: nix-grpc-daemon.service: Consumed 1.430s CPU time over 28.780s wall clock time, 39.9M memory peak, 61.3K incoming IP traffic, 149.4K outgoing IP traffic.5761worker2 # [ 85.420540] systemd[1]: Starting scheduler-token.service...5762worker2 # [ 85.466260] systemd[1]: scheduler-token.service: Deactivated successfully.5763worker2 # [ 85.468135] systemd[1]: Finished scheduler-token.service.5764worker2 # [ 85.469557] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 47ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5765worker2 # [ 85.473550] systemd[1]: Starting Nix worker-protocol over gRPC...5766worker2 # [ 85.549659] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5767worker2 # [ 85.551659] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5768worker2 # [ 85.554785] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15769worker2 # [ 85.559054] nix-grpc-daemon[1339]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5770worker2 # [ 85.561152] nix-grpc-daemon[1339]: E0000 00:00:1789984333.622184 1339 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5771worker2 # [ 85.567042] nix-grpc-daemon[1339]: E0000 00:00:1789984333.627677 1339 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5772worker2 # [ 85.570036] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5773worker2 # [ 85.572363] systemd[1]: Started Nix worker-protocol over gRPC.5774worker2 # [ 85.575064] nix-grpc-daemon[1339]: E0000 00:00:1789984333.637898 1379 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5775worker2 # [ 85.586739] nix-grpc-daemon[1339]: E0000 00:00:1789984333.650129 1379 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5776worker2 # [ 85.591057] nix-grpc-daemon[1339]: E0000 00:00:1789984333.652497 1379 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5777worker2 # [ 85.601056] nix-grpc-daemon[1339]: E0000 00:00:1789984333.664473 1379 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5778worker2 # [ 85.610564] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:13Z level=info event=scheduler_connected addr=lb:500515779worker1 # [ 85.599808] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:13Z level=info event=oidc_jwks provider=mock keys=15780worker1 # [ 85.604119] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:491505781worker1 # [ 85.607987] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05782lb: must succeed: curl -sf localhost:9901/clusters5783lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5784worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5785worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5786(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.33 seconds)5787subtest: scheduler restart mid-build: build finishes, no double build5788client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid5789client # [ 85.863574] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.5790client # Running as unit: mid.service; invocation ID: c991e8ac10664cceb79969918ead7b375791client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)5792worker1 # [ 85.978343] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:337005793client # [ 86.007653] nix[1211]: don't know how to build these paths:5794client # [ 86.008569] nix[1211]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5795worker1 # [ 85.995640] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:13Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05796worker1 # [ 86.000921] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:14Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15797worker1 # [ 86.039646] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337005798worker1 # [ 86.050699] nix-grpc-daemon[1347]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)5799worker1 # [ 86.059923] nix-grpc-daemon[1347]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 2/5)5800worker1 # [ 86.153729] nix-grpc-daemon[1347]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 321 ms (attempt 3/5)5801worker1 # [ 86.476784] nix-grpc-daemon[1347]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 322 ms (attempt 4/5)5802lb # [ 86.721881] envoy[914]: conn peer=[2001:db8:1::1]:35732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1429 ms=80705803lb # [ 86.725561] envoy[914]: conn peer=[2001:db8:1::4]:60830 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36721 tx=1993 ms=270015804worker1 # [ 86.801814] nix-grpc-daemon[1347]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5805worker1 # [ 86.805633] nix-grpc-daemon[1347]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5806worker1 # [ 86.824149] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337005807client # [ 86.857223] nix[1211]: copying 1 paths to 'lb:50051'...5808worker1 # [ 86.836117] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337005809client # [ 86.861164] nix[1211]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5810worker1 # [ 86.845663] nix-daemon[857]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5811worker1 # [ 86.870196] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:14Z level=info event=niks3_push_started pid=14725812worker1 # [ 86.974278] nix-grpc-daemon[1472]: time=2026-09-21T09:52:14.976Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5813worker1 # [ 87.064149] nix-daemon[857]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5814worker1 # [ 87.076514] nix-daemon[857]: reaped child process 1491, status = succeeded5815worker1 # [ 87.157066] nix-daemon[857]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5816worker1 # [ 87.169554] nix-daemon[857]: reaped child process 1496, status = succeeded5817lb # [ 87.194754] niks3-server[957]: 2026/09/21 09:52:14 INFO Received uploads request method=POST path=/api/pending_closures5818worker1 # [ 87.186242] nix-grpc-daemon[1472]: time=2026-09-21T09:52:15.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5819worker1 # [ 87.189320] nix-grpc-daemon[1472]: time=2026-09-21T09:52:15.192Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5820lb # [ 87.233067] niks3-server[957]: 2026/09/21 09:52:14 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5821lb # [ 87.240438] niks3-server[957]: 2026/09/21 09:52:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5822lb # [ 87.247357] niks3-server[957]: 2026/09/21 09:52:14 INFO Signed narinfos id=35 count=15823worker1 # [ 87.237345] nix-grpc-daemon[1472]: time=2026-09-21T09:52:15.241Z level=INFO msg="Uploading 1 narinfos"5824lb # [ 87.261320] niks3-server[957]: 2026/09/21 09:52:14 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5825lb # [ 87.270240] niks3-server[957]: 2026/09/21 09:52:14 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5826lb # [ 87.275281] niks3-server[957]: 2026/09/21 09:52:14 INFO Completed upload id=355827worker1 # [ 87.260407] nix-grpc-daemon[1472]: time=2026-09-21T09:52:15.265Z level=INFO msg="Upload complete. (287ms)"5828lb # [ 87.278677] niks3-server[957]: 2026/09/21 09:52:14 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5829worker1 # [ 87.264382] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33700 duration_s=0 paths=1 nar_bytes_in=4245830worker1 # [ 87.270250] nix-grpc-daemon[1347]: ts=2026-09-21T09:52:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:337005831worker1 # [ 87.279107] nix-daemon[857]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5832worker1 # [ 87.285565] nix-daemon[857]: reaped child process 1470, status = succeeded5833worker1 # [ 87.289706] nix-daemon[857]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5834client # [ 87.312695] nix[1211]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5835lb: must succeed: curl -sf localhost:9901/clusters5836lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5837lb: must succeed: curl -sf localhost:9901/clusters5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5839worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5840worker1 # [ 89.472095] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1347 (nix-grpc-daemon) on client request.5841worker1 # [ 89.473736] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5842lb # [ 89.493778] niks3-server[957]: 2026/09/21 09:52:16 INFO lead: released remote=192.168.1.3:503965843worker1 # [ 89.486240] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5844worker1 # [ 89.488490] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5845client # [ 89.508464] nix[1211]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5846worker2 # [ 89.504314] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5847worker1 # [ 89.491149] systemd[1]: nix-grpc-daemon.service: Consumed 455ms CPU time over 16.381s wall clock time, 35.2M memory peak, 17K incoming IP traffic, 31.8K outgoing IP traffic.5848client # [ 89.513103] nix[1211]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5849worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5850client: must succeed: sleep 5.05851worker1 # [ 89.510528] nix-daemon[857]: reaped child process 1498, status = succeeded5852worker1 # [ 89.512170] nix-daemon[857]: reaped child process 1406, status = succeeded5853worker1 # [ 89.522241] nix-daemon[1500]: unexpected Nix daemon error: error: interrupted by the user5854worker1 # [ 89.526159] nix-daemon[857]: reaped child process 1500, status = failed with exit code 15855worker2 # [ 89.705183] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:17Z level=info event=scheduler_connected addr=lb:500515856worker1 # [ 89.689353] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5857worker1 # [ 89.697734] systemd[1]: Starting Nix worker-protocol over gRPC...5858worker1 # [ 89.800413] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5859worker1 # [ 89.802668] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5860worker1 # [ 89.805807] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15861worker1 # [ 89.810060] nix-grpc-daemon[1537]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5862worker1 # [ 89.811873] nix-grpc-daemon[1537]: E0000 00:00:1789984337.814437 1537 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5863worker1 # [ 89.817463] nix-grpc-daemon[1537]: E0000 00:00:1789984337.819896 1537 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5864worker1 # [ 89.821101] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5865worker1 # [ 89.824267] systemd[1]: Started Nix worker-protocol over gRPC.5866lb # [ 89.841172] niks3-server[957]: 2026/09/21 09:52:17 INFO lead: acquired remote=192.168.1.3:604885867worker1 # [ 89.827062] nix-grpc-daemon[1537]: E0000 00:00:1789984337.831252 1578 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5868worker1 # [ 89.830236] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=scheduler_take_over5869worker1 # [ 89.837694] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=oidc_jwks provider=mock keys=15870worker1 # [ 89.840605] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:491545871worker1 # [ 89.842216] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05872worker1 # [ 89.849049] nix-grpc-daemon[1537]: E0000 00:00:1789984337.853716 1578 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5873worker1 # [ 89.853055] nix-grpc-daemon[1537]: E0000 00:00:1789984337.856026 1578 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5874worker1 # [ 89.863057] nix-grpc-daemon[1537]: E0000 00:00:1789984337.867753 1578 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5875worker1 # [ 89.871767] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=scheduler_connected addr=lb:500515876worker1 # [ 89.873553] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:491545877worker1 # [ 89.875049] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05878worker1 # [ 89.995732] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:17Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05879worker1 # [ 90.000285] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:18Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15880worker2 # [ 90.064062] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564985881worker2 # [ 90.075655] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5882worker2 # [ 90.091969] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)5883worker2 # [ 90.193708] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5)5884worker2 # [ 90.375986] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 3/5)5885worker2 # [ 90.464869] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 259 ms (attempt 4/5)5886worker2 # [ 90.726843] nix-grpc-daemon[1339]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5887worker2 # [ 90.730617] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5888worker2 # [ 90.745220] nix-grpc-daemon[1339]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5889worker2 # [ 90.767417] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5890worker2 # [ 90.782843] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5891client # [ 90.786386] nix[1211]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5892client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5893client: waiting for success: ! systemctl is-active mid5894worker2 # [ 94.800496] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:22Z level=info event=niks3_push_started pid=14455895worker2 # [ 94.816674] nix-daemon[849]: reaped child process 1420, status = succeeded5896worker2 # [ 94.893633] nix-grpc-daemon[1445]: time=2026-09-21T09:52:22.955Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5897worker2 # [ 94.975265] nix-daemon[849]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)5898worker2 # [ 94.987770] nix-daemon[849]: reaped child process 1460, status = succeeded5899lb # [ 94.993301] niks3-server[957]: 2026/09/21 09:52:22 INFO Received uploads request method=POST path=/api/pending_closures5900worker2 # [ 95.009071] nix-grpc-daemon[1445]: time=2026-09-21T09:52:23.072Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5901worker2 # [ 95.010656] nix-grpc-daemon[1445]: time=2026-09-21T09:52:23.074Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5902lb # [ 95.048973] niks3-server[957]: 2026/09/21 09:52:22 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5903lb # [ 95.052894] niks3-server[957]: 2026/09/21 09:52:22 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5904lb # [ 95.055657] niks3-server[957]: 2026/09/21 09:52:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5905lb # [ 95.059058] niks3-server[957]: 2026/09/21 09:52:22 INFO Signed narinfos id=36 count=15906worker2 # [ 95.062877] nix-grpc-daemon[1445]: time=2026-09-21T09:52:23.126Z level=INFO msg="Uploading 1 narinfos"5907lb # [ 95.069104] niks3-server[957]: 2026/09/21 09:52:22 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5908lb # [ 95.075124] niks3-server[957]: 2026/09/21 09:52:22 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5909lb # [ 95.083534] niks3-server[957]: 2026/09/21 09:52:22 INFO Completed upload id=365910worker2 # [ 95.088723] nix-grpc-daemon[1445]: time=2026-09-21T09:52:23.152Z level=INFO msg="Upload complete. (196ms)"5911lb # [ 95.087175] niks3-server[957]: 2026/09/21 09:52:22 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5912worker2 # [ 95.092962] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56498 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15913worker2 # [ 95.105068] nix-daemon[849]: reaped child process 1418, status = succeeded5914client # [ 95.129310] systemd[1]: mid.service: Deactivated successfully.5915client # [ 95.132930] systemd[1]: mid.service: Consumed 145ms CPU time over 9.269s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.5916client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)5917client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5918client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)5919(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.03 seconds)5920subtest: interrupting the client stops the build on the worker5921client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr5922client # [ 95.866451] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.5923client # Running as unit: intr.service; invocation ID: 145ff8e718a6409590c741f7a253fa7c5924client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)5925client # [ 95.988206] nix[1247]: don't know how to build these paths:5926client # [ 95.989117] nix[1247]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5927worker2 # [ 95.982076] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564985928worker1 # [ 95.976668] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:23Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05929worker1 # [ 95.980858] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:23Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15930worker1 # [ 96.022894] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373725931worker1 # [ 96.034345] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)5932worker1 # [ 96.051792] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)5933worker1 # [ 96.074188] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 2/5)5934worker1 # [ 96.172474] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 214 ms (attempt 3/5)5935worker1 # [ 96.388722] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 774 ms (attempt 4/5)5936lb # [ 96.722452] envoy[914]: conn peer=[2001:db8:1::3]:51732 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2386 tx=618 ms=162245937lb # [ 96.726594] envoy[914]: conn peer=[2001:db8:1::1]:36844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1827 ms=91125938worker1 # [ 97.165937] nix-grpc-daemon[1537]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5939worker1 # [ 97.170328] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5940worker1 # [ 97.187356] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373725941client # [ 97.221719] nix[1247]: copying 1 paths to 'lb:50051'...5942client # [ 97.224885] nix[1247]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5943worker1 # [ 97.202937] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373725944worker1 # [ 97.214206] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)5945worker1 # [ 97.236323] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:25Z level=info event=niks3_push_started pid=16175946worker1 # [ 97.338394] nix-grpc-daemon[1617]: time=2026-09-21T09:52:25.342Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5947worker1 # [ 97.422128] nix-daemon[857]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)5948worker1 # [ 97.435485] nix-daemon[857]: reaped child process 1633, status = succeeded5949worker1 # [ 97.518916] nix-daemon[857]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)5950worker1 # [ 97.531732] nix-daemon[857]: reaped child process 1638, status = succeeded5951lb # [ 97.556747] niks3-server[957]: 2026/09/21 09:52:24 INFO Received uploads request method=POST path=/api/pending_closures5952worker1 # [ 97.550026] nix-grpc-daemon[1617]: time=2026-09-21T09:52:25.554Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5953worker1 # [ 97.551612] nix-grpc-daemon[1617]: time=2026-09-21T09:52:25.556Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5954lb # [ 97.592191] niks3-server[957]: 2026/09/21 09:52:24 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5955lb # [ 97.597392] niks3-server[957]: 2026/09/21 09:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5956lb # [ 97.602707] niks3-server[957]: 2026/09/21 09:52:24 INFO Signed narinfos id=37 count=15957worker1 # [ 97.590842] nix-grpc-daemon[1617]: time=2026-09-21T09:52:25.595Z level=INFO msg="Uploading 1 narinfos"5958lb # [ 97.618227] niks3-server[957]: 2026/09/21 09:52:24 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5959lb # [ 97.622431] niks3-server[957]: 2026/09/21 09:52:24 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5960lb # [ 97.633073] niks3-server[957]: 2026/09/21 09:52:25 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5961lb # [ 97.637745] niks3-server[957]: 2026/09/21 09:52:25 INFO Completed upload id=375962worker1 # [ 97.622619] nix-grpc-daemon[1617]: time=2026-09-21T09:52:25.627Z level=INFO msg="Upload complete. (281ms)"5963worker1 # [ 97.626124] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=4245964worker1 # [ 97.630770] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373725965worker1 # [ 97.641058] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)5966worker1 # [ 97.645222] nix-daemon[857]: reaped child process 1615, status = succeeded5967worker1 # [ 97.650122] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)5968client # [ 97.672417] nix[1247]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5969client: must succeed: systemctl kill -s INT intr5970client # [ 99.286445] systemd[1]: intr.service: Sent signal SIGINT to main process 1247 (nix) on client request.5971client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)5972worker1 # [ 99.270628] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:27Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5973client # [ 99.294661] nix[1247]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5974client # [ 99.300843] nix[1247]: error: interrupted by the user5975client # [ 99.327525] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5976client # [ 99.329188] systemd[1]: intr.service: Failed with result 'exit-code'.5977client # [ 99.331053] systemd[1]: intr.service: Consumed 120ms CPU time over 3.462s wall clock time, 43.6M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5978worker1 # [ 99.471472] nix-daemon[1642]: unexpected Nix daemon error: error: interrupted by the user5979worker1 # [ 99.473086] nix-daemon[857]: reaped child process 1640, status = succeeded5980worker1 # [ 99.476900] nix-daemon[857]: reaped child process 1642, status = failed with exit code 15981(finished: subtest: interrupting the client stops the build on the worker, in 4.63 seconds)5982subtest: a deploy mid-build drains: the build finishes, the new generation takes over5983client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw5984client # [ 100.523584] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.5985client # Running as unit: sw.service; invocation ID: 29d8b10c3247481ea856e0b1865a007d5986client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)5987worker2 # [ 100.655878] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564985988client # [ 100.665687] nix[1268]: don't know how to build these paths:5989client # [ 100.666647] nix[1268]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5990worker1 # [ 100.654861] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:28Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05991worker1 # [ 100.659095] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:28Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=15992worker2 # [ 100.718165] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564985993worker2 # [ 100.731168] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564985994client # [ 100.745200] nix[1268]: copying 1 paths to 'lb:50051'...5995worker2 # [ 100.742580] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564985996client # [ 100.747145] nix[1268]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...5997worker2 # [ 100.755393] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5998worker2 # [ 100.851741] nix-daemon[849]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)5999worker2 # [ 100.864041] nix-daemon[849]: reaped child process 1496, status = succeeded6000worker2 # [ 100.944365] nix-daemon[849]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6001worker2 # [ 100.956703] nix-daemon[849]: reaped child process 1501, status = succeeded6002lb # [ 100.959860] niks3-server[957]: 2026/09/21 09:52:28 INFO Received uploads request method=POST path=/api/pending_closures6003worker2 # [ 100.976580] nix-grpc-daemon[1445]: time=2026-09-21T09:52:29.039Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6004worker2 # [ 100.978057] nix-grpc-daemon[1445]: time=2026-09-21T09:52:29.039Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6005lb # [ 100.996377] niks3-server[957]: 2026/09/21 09:52:28 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6006lb # [ 101.004566] niks3-server[957]: 2026/09/21 09:52:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6007lb # [ 101.008589] niks3-server[957]: 2026/09/21 09:52:28 INFO Signed narinfos id=38 count=16008worker2 # [ 101.015325] nix-grpc-daemon[1445]: time=2026-09-21T09:52:29.078Z level=INFO msg="Uploading 1 narinfos"6009lb # [ 101.015534] niks3-server[957]: 2026/09/21 09:52:28 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6010lb # [ 101.023117] niks3-server[957]: 2026/09/21 09:52:28 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6011lb # [ 101.026510] niks3-server[957]: 2026/09/21 09:52:28 INFO Completed upload id=386012worker2 # [ 101.030375] nix-grpc-daemon[1445]: time=2026-09-21T09:52:29.093Z level=INFO msg="Upload complete. (263ms)"6013lb # [ 101.029270] niks3-server[957]: 2026/09/21 09:52:28 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6014worker2 # [ 101.033605] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56498 duration_s=0 paths=1 nar_bytes_in=4166015worker2 # [ 101.038539] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564986016worker2 # [ 101.046949] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6017worker2 # [ 101.051838] nix-daemon[849]: reaped child process 1490, status = succeeded6018worker2 # [ 101.056719] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6019client # [ 101.060092] nix[1268]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6020worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26021worker2 # Checking switch inhibitors... done6022worker2 # [ 102.126452] nixos[1533]: switching to system configuration /nix/store/qqd2wgcapb7a8xhvsp7j1jcc9pc91hnq-nixos-system-worker2-test6023worker2 # [ 102.129409] systemd[1]: Stopped target Remote File Systems.6024worker2 # activating the configuration...6025worker2 # [ 102.132382] systemd[1]: Stopped target Local File Systems.6026worker2 # setting up /etc...6027worker2 # [ 102.550724] systemd[1]: Reload requested from client PID 1533 ('.switch-to-conf') (unit backdoor.service)...6028worker2 # [ 102.552525] systemd[1]: Reloading...6029worker2 # [ 103.899317] systemd[1]: Reloading finished in 1343 ms.6030worker2 # restarting sysinit-reactivation.target6031worker2 # [ 103.955169] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6032worker2 # [ 104.059380] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6033worker2 # [ 104.061530] systemd[1]: Reached target Reactivate sysinit units.6034worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6035worker2 # [ 104.067134] systemd[1]: Reloading D-Bus System Message Bus...6036worker2 # [ 104.068376] dbus-broker-launch[520]: Caught SIGHUP, trigger reload.6037worker2 # [ 104.072103] systemd[1]: Reloading Nix worker-protocol over gRPC...6038worker2 # [ 104.085515] dbus-broker-launch[520]: Looking up NSS user entry for 'systemd-timesync'...6039worker2 # [ 104.090230] dbus-broker-launch[520]: NSS returned no entry for 'systemd-timesync'6040worker2 # [ 104.096070] dbus-broker-launch[520]: Invalid user-name in /nix/store/9f31zf0wixjfmyvwg6m53sif40vrkysg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6041worker2 # [ 104.107060] systemd[1]: Reloaded Nix worker-protocol over gRPC.6042worker2 # [ 104.113516] systemd[1]: Reloaded D-Bus System Message Bus.6043worker2 # [ 104.117800] systemd[1]: Reached target Remote File Systems.6044worker2 # [ 104.120440] systemd[1]: Starting linger-users.service...6045worker2 # [ 104.121710] systemd[1]: Reached target Local File Systems.6046worker2 # [ 104.124105] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6047worker2 # [ 104.131422] systemd[1]: Starting Create SUID/SGID Wrappers...6048worker2 # [ 104.132295] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6049worker2 # [ 104.133478] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676050worker2 # [ 104.135493] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6051worker2 # [ 104.136975] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6052worker2 # [ 104.141570] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6053worker2 # [ 104.152174] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6054worker2 # [ 104.153469] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6055worker2 # [ 104.182161] systemd[1]: linger-users.service: Deactivated successfully.6056worker2 # [ 104.183155] systemd[1]: Finished linger-users.service.6057worker2 # [ 104.391899] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6058worker2 # [ 104.396079] systemd[1]: Finished Create SUID/SGID Wrappers.6059worker2 # [ 104.572191] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:32Z level=info event=draining6060worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6061worker2 # [ 104.649813] nixos[1533]: finished switching to system configuration /nix/store/qqd2wgcapb7a8xhvsp7j1jcc9pc91hnq-nixos-system-worker2-test6062worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.85 seconds)6063worker2: must succeed: pgrep -f 'read -t [9]0 x'6064worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6065client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26066worker1 # [ 104.821253] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:373726067client # don't know how to build these paths:6068client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6069worker2 # [ 104.849744] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564986070worker1 # [ 104.850862] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06071worker1 # [ 104.855768] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16072worker1 # [ 104.900672] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726073worker1 # [ 104.913607] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726074client # copying 1 paths to 'lb:50051'...6075worker1 # [ 104.926355] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726076client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6077worker1 # [ 104.953472] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6078worker1 # [ 105.054058] nix-daemon[857]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6079worker1 # [ 105.067180] nix-daemon[857]: reaped child process 1696, status = succeeded6080worker1 # [ 105.147970] nix-daemon[857]: accepted connection from pid 1698, user nix-grpc-daemon (trusted)6081worker1 # [ 105.161156] nix-daemon[857]: reaped child process 1701, status = succeeded6082lb # [ 105.183742] niks3-server[957]: 2026/09/21 09:52:32 INFO Received uploads request method=POST path=/api/pending_closures6083worker1 # [ 105.179542] nix-grpc-daemon[1617]: time=2026-09-21T09:52:33.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6084worker1 # [ 105.180986] nix-grpc-daemon[1617]: time=2026-09-21T09:52:33.183Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6085lb # [ 105.212574] niks3-server[957]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6086lb # [ 105.225470] niks3-server[957]: 2026/09/21 09:52:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6087worker1 # [ 105.217961] nix-grpc-daemon[1617]: time=2026-09-21T09:52:33.222Z level=INFO msg="Uploading 1 narinfos"6088lb # [ 105.233831] niks3-server[957]: 2026/09/21 09:52:32 INFO Signed narinfos id=39 count=16089lb # [ 105.238849] niks3-server[957]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6090lb # [ 105.251130] niks3-server[957]: 2026/09/21 09:52:32 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6091lb # [ 105.257929] niks3-server[957]: 2026/09/21 09:52:32 INFO Completed upload id=396092worker1 # [ 105.242713] nix-grpc-daemon[1617]: time=2026-09-21T09:52:33.247Z level=INFO msg="Upload complete. (277ms)"6093lb # [ 105.261489] niks3-server[957]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6094worker1 # [ 105.245187] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=5286095worker1 # [ 105.251164] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726096worker1 # [ 105.262109] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6097worker1 # [ 105.266551] nix-daemon[857]: reaped child process 1691, status = succeeded6098worker1 # [ 105.271369] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6099client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6100client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6101client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6102client # farm-a> LOG-farm-a-during-drain6103worker1 # [ 106.086311] nix-daemon[857]: reaped child process 1705, status = succeeded6104worker1 # [ 106.158878] nix-daemon[857]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)6105worker1 # [ 106.173607] nix-daemon[857]: reaped child process 1724, status = succeeded6106lb # [ 106.193864] niks3-server[957]: 2026/09/21 09:52:33 INFO Received uploads request method=POST path=/api/pending_closures6107worker1 # [ 106.187604] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.192Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6108worker1 # [ 106.189299] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.193Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6109lb # [ 106.229654] niks3-server[957]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6110lb # [ 106.252262] niks3-server[957]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6111lb # [ 106.265622] niks3-server[957]: 2026/09/21 09:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6112lb # [ 106.268299] niks3-server[957]: 2026/09/21 09:52:33 INFO Signed narinfos id=40 count=16113worker1 # [ 106.255074] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.259Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 106.271763] niks3-server[957]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6115lb # [ 106.276072] niks3-server[957]: 2026/09/21 09:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6116lb # [ 106.279926] niks3-server[957]: 2026/09/21 09:52:33 INFO Completed upload id=406117worker1 # [ 106.264787] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.269Z level=INFO msg="Upload complete. (187ms)"6118lb # [ 106.282500] niks3-server[957]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6119worker1 # [ 106.272799] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16120worker1 # [ 106.281734] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06121worker1 # [ 106.285129] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16122worker1 # [ 106.293309] nix-daemon[857]: reaped child process 1703, status = succeeded6123worker1 # [ 106.326124] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726124worker1 # [ 106.339101] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726125worker1 # [ 106.353251] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.357Z level=INFO msg="All 1 paths already cached"6126client # copying 1 paths to 'lb:50051'...6127worker1 # [ 106.361292] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726128client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6129worker1 # [ 106.381139] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6130worker1 # [ 106.478966] nix-daemon[857]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)6131worker1 # [ 106.492453] nix-daemon[857]: reaped child process 1733, status = succeeded6132worker1 # [ 106.572764] nix-daemon[857]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)6133worker1 # [ 106.586237] nix-daemon[857]: reaped child process 1738, status = succeeded6134lb # [ 106.607488] niks3-server[957]: 2026/09/21 09:52:33 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 106.600625] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.605Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6136worker1 # [ 106.602642] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.606Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6137lb # [ 106.640968] niks3-server[957]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6138lb # [ 106.645222] niks3-server[957]: 2026/09/21 09:52:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6139lb # [ 106.650472] niks3-server[957]: 2026/09/21 09:52:34 INFO Signed narinfos id=41 count=16140worker1 # [ 106.640238] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.644Z level=INFO msg="Uploading 1 narinfos"6141lb # [ 106.658550] niks3-server[957]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6142lb # [ 106.674503] niks3-server[957]: 2026/09/21 09:52:34 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6143lb # [ 106.681046] niks3-server[957]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6144lb # [ 106.685604] niks3-server[957]: 2026/09/21 09:52:34 INFO Completed upload id=416145worker1 # [ 106.670407] nix-grpc-daemon[1617]: time=2026-09-21T09:52:34.675Z level=INFO msg="Upload complete. (276ms)"6146worker1 # [ 106.674666] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=6406147worker1 # [ 106.681330] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726148worker1 # [ 106.691654] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6149worker1 # [ 106.700346] nix-daemon[857]: reaped child process 1728, status = succeeded6150worker1 # [ 106.705276] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6151client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6152client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6153lb # [ 106.722900] envoy[914]: conn peer=[2001:db8:1::1]:45206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=827 ms=33256154client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6155client # farm-b> LOG-farm-b-during-drain6156worker1 # [ 107.947110] nix-daemon[857]: reaped child process 1742, status = succeeded6157worker1 # [ 108.022843] nix-daemon[857]: accepted connection from pid 1758, user nix-grpc-daemon (trusted)6158worker1 # [ 108.036072] nix-daemon[857]: reaped child process 1761, status = succeeded6159lb # [ 108.059534] niks3-server[957]: 2026/09/21 09:52:35 INFO Received uploads request method=POST path=/api/pending_closures6160worker1 # [ 108.053535] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.058Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6161worker1 # [ 108.055896] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.059Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6162lb # [ 108.098511] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6163lb # [ 108.107146] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6164lb # [ 108.114773] niks3-server[957]: 2026/09/21 09:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6165lb # [ 108.122471] niks3-server[957]: 2026/09/21 09:52:35 INFO Signed narinfos id=42 count=16166worker1 # [ 108.107761] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.112Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 108.127114] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6168lb # [ 108.138270] niks3-server[957]: 2026/09/21 09:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6169lb # [ 108.144191] niks3-server[957]: 2026/09/21 09:52:35 INFO Completed upload id=426170worker1 # [ 108.129343] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.134Z level=INFO msg="Upload complete. (216ms)"6171lb # [ 108.146257] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6172worker1 # [ 108.135612] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16173worker1 # [ 108.147930] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06174worker1 # [ 108.152352] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16175worker1 # [ 108.162272] nix-daemon[857]: reaped child process 1740, status = succeeded6176worker1 # [ 108.191780] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726177worker1 # [ 108.205950] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726178worker1 # [ 108.218667] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.223Z level=INFO msg="All 2 paths already cached"6179client # copying 1 paths to 'lb:50051'...6180client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6181worker1 # [ 108.227097] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726182worker1 # [ 108.235564] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6183worker1 # [ 108.330223] nix-daemon[857]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6184worker1 # [ 108.343482] nix-daemon[857]: reaped child process 1770, status = succeeded6185worker1 # [ 108.424932] nix-daemon[857]: accepted connection from pid 1772, user nix-grpc-daemon (trusted)6186worker1 # [ 108.438411] nix-daemon[857]: reaped child process 1775, status = succeeded6187lb # [ 108.460235] niks3-server[957]: 2026/09/21 09:52:35 INFO Received uploads request method=POST path=/api/pending_closures6188worker1 # [ 108.452925] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.457Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6189worker1 # [ 108.454706] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.459Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6190lb # [ 108.487858] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6191lb # [ 108.498773] niks3-server[957]: 2026/09/21 09:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6192worker1 # [ 108.491435] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.496Z level=INFO msg="Uploading 1 narinfos"6193lb # [ 108.506407] niks3-server[957]: 2026/09/21 09:52:35 INFO Signed narinfos id=43 count=16194lb # [ 108.514900] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6195lb # [ 108.524963] niks3-server[957]: 2026/09/21 09:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6196lb # [ 108.531896] niks3-server[957]: 2026/09/21 09:52:35 INFO Completed upload id=436197lb # [ 108.534953] niks3-server[957]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6198worker1 # [ 108.519389] nix-grpc-daemon[1617]: time=2026-09-21T09:52:36.524Z level=INFO msg="Upload complete. (271ms)"6199worker1 # [ 108.523479] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=7686200worker1 # [ 108.528292] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726201worker1 # [ 108.540059] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6202worker1 # [ 108.543625] nix-daemon[857]: reaped child process 1765, status = succeeded6203worker1 # [ 108.548724] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6204client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6205client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6206client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6207client # farm-top> LOG-farm-top-during-drain6208worker1 # [ 109.982700] nix-daemon[857]: reaped child process 1779, status = succeeded6209worker1 # [ 110.067814] nix-daemon[857]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6210worker1 # [ 110.081337] nix-daemon[857]: reaped child process 1798, status = succeeded6211lb # [ 110.104067] niks3-server[957]: 2026/09/21 09:52:37 INFO Received uploads request method=POST path=/api/pending_closures6212worker1 # [ 110.098101] nix-grpc-daemon[1617]: time=2026-09-21T09:52:38.102Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6213worker1 # [ 110.101047] nix-grpc-daemon[1617]: time=2026-09-21T09:52:38.104Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6214lb # [ 110.135781] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6215lb # [ 110.151288] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6216lb # [ 110.160947] niks3-server[957]: 2026/09/21 09:52:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6217lb # [ 110.165965] niks3-server[957]: 2026/09/21 09:52:37 INFO Signed narinfos id=44 count=16218worker1 # [ 110.151305] nix-grpc-daemon[1617]: time=2026-09-21T09:52:38.155Z level=INFO msg="Uploading 1 narinfos"6219lb # [ 110.171719] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6220lb # [ 110.181453] niks3-server[957]: 2026/09/21 09:52:37 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6221lb # [ 110.185655] niks3-server[957]: 2026/09/21 09:52:37 INFO Completed upload id=446222worker1 # [ 110.170676] nix-grpc-daemon[1617]: time=2026-09-21T09:52:38.175Z level=INFO msg="Upload complete. (211ms)"6223lb # [ 110.188459] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6224worker1 # [ 110.178518] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16225worker1 # [ 110.192590] nix-daemon[857]: reaped child process 1777, status = succeeded6226client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.54 seconds)6227client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6228client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6229worker2: must succeed: pkill -f 'read -t [9]0 x'6230worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6231client: waiting for success: ! systemctl is-active sw6232worker2 # [ 110.501177] nix-daemon[849]: reaped child process 1505, status = succeeded6233worker2 # [ 110.567301] nix-daemon[849]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6234worker2 # [ 110.579728] nix-daemon[849]: reaped child process 1677, status = succeeded6235lb # [ 110.583202] niks3-server[957]: 2026/09/21 09:52:37 INFO Received uploads request method=POST path=/api/pending_closures6236worker2 # [ 110.593675] nix-grpc-daemon[1445]: time=2026-09-21T09:52:38.656Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6237worker2 # [ 110.595318] nix-grpc-daemon[1445]: time=2026-09-21T09:52:38.658Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6238lb # [ 110.610768] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6239lb # [ 110.614107] niks3-server[957]: 2026/09/21 09:52:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6240lb # [ 110.617230] niks3-server[957]: 2026/09/21 09:52:37 INFO Signed narinfos id=45 count=16241worker2 # [ 110.621284] nix-grpc-daemon[1445]: time=2026-09-21T09:52:38.684Z level=INFO msg="Uploading 1 narinfos"6242lb # [ 110.620184] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6243lb # [ 110.622728] niks3-server[957]: 2026/09/21 09:52:37 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6244lb # [ 110.627127] niks3-server[957]: 2026/09/21 09:52:37 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6245lb # [ 110.630820] niks3-server[957]: 2026/09/21 09:52:38 INFO Completed upload id=456246worker2 # [ 110.635852] nix-grpc-daemon[1445]: time=2026-09-21T09:52:38.699Z level=INFO msg="Upload complete. (162ms)"6247lb # [ 110.632228] niks3-server[957]: 2026/09/21 09:52:38 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6248worker2 # [ 110.639154] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56498 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16249worker2 # [ 110.645208] nix-daemon[849]: reaped child process 1503, status = succeeded6250client # [ 110.673639] systemd[1]: sw.service: Deactivated successfully.6251client # [ 110.678091] systemd[1]: sw.service: Consumed 130ms CPU time over 10.153s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6252client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6253client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6254client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6255worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6256worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6257lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6260worker2 # [ 111.577854] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:39Z level=info event=signal_exit6261lb: must succeed: curl -sf localhost:9901/clusters6262worker1 # [ 112.565351] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:491546263worker1 # [ 112.569084] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:40Z level=info event=worker_gone addr=192.168.1.4:500516264worker2 # [ 112.583541] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:40Z level=info event=scheduler_restarting6265worker2 # [ 112.597529] nix-grpc-daemon[1339]: ts=2026-09-21T09:52:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6266lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6267worker2 # [ 112.617160] nix-daemon[849]: reaped child process 1411, status = succeeded6268worker2 # [ 113.458584] nix-grpc-daemon[1339]: E0000 00:00:1789984361.521805 1384 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6269worker2 # [ 113.463395] nix-grpc-daemon[1339]: E0000 00:00:1789984361.521941 1384 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6270worker2 # [ 113.600530] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6271worker2 # [ 113.603863] systemd[1]: nix-grpc-daemon.service: Consumed 612ms CPU time over 28.126s wall clock time, 35.8M memory peak, 21K incoming IP traffic, 28.9K outgoing IP traffic.6272lb: must succeed: curl -sf localhost:9901/clusters6273lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6274worker2 # [ 114.596893] systemd[1]: Starting scheduler-token.service...6275worker2 # [ 114.653545] systemd[1]: scheduler-token.service: Deactivated successfully.6276worker2 # [ 114.656514] systemd[1]: Finished scheduler-token.service.6277worker2 # [ 114.659176] systemd[1]: scheduler-token.service: Consumed 33ms CPU time over 59ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6278worker2 # [ 114.666868] systemd[1]: Starting Nix worker-protocol over gRPC...6279lb: must succeed: curl -sf localhost:9901/clusters6280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6281worker2 # [ 114.744410] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6282worker2 # [ 114.746150] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6283worker2 # [ 114.749375] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16284worker2 # [ 114.753660] nix-grpc-daemon[1692]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6285worker2 # [ 114.754983] nix-grpc-daemon[1692]: E0000 00:00:1789984362.816703 1692 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6286worker2 # [ 114.761067] nix-grpc-daemon[1692]: E0000 00:00:1789984362.821970 1692 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6287worker2 # [ 114.764116] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6288worker2 # [ 114.765651] systemd[1]: Started Nix worker-protocol over gRPC.6289worker2 # [ 114.772062] nix-grpc-daemon[1692]: E0000 00:00:1789984362.834031 1734 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6290worker2 # [ 114.785134] nix-grpc-daemon[1692]: E0000 00:00:1789984362.848527 1734 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6291worker2 # [ 114.789054] nix-grpc-daemon[1692]: E0000 00:00:1789984362.850736 1734 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6292worker2 # [ 114.799031] nix-grpc-daemon[1692]: E0000 00:00:1789984362.862411 1734 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6293worker2 # [ 114.807900] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:42Z level=info event=scheduler_connected addr=lb:500516294worker1 # [ 114.792520] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:491546295worker1 # [ 114.796778] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06296lb: must succeed: curl -sf localhost:9901/clusters6297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6298lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.24 seconds)6299lb: must succeed: curl -sf localhost:9901/clusters6300lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6301worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6302worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6303(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.36 seconds)6304subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6305client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16306worker2 # [ 115.989213] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579286307worker2 # [ 115.999967] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6308worker1 # [ 115.998244] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726309worker1 # [ 116.036248] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06310worker1 # [ 116.040279] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16311worker1 # [ 116.044564] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06312worker1 # [ 116.049812] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06313worker1 # [ 116.056475] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726314worker1 # [ 116.067921] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726315worker1 # [ 116.079651] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726316worker1 # [ 116.090286] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6317worker1 # [ 116.185204] nix-daemon[857]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6318worker1 # [ 116.198768] nix-daemon[857]: reaped child process 1810, status = succeeded6319worker1 # [ 116.284181] nix-daemon[857]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6320worker1 # [ 116.296904] nix-daemon[857]: reaped child process 1815, status = succeeded6321lb # [ 116.321500] niks3-server[957]: 2026/09/21 09:52:43 INFO Received uploads request method=POST path=/api/pending_closures6322worker1 # [ 116.317444] nix-grpc-daemon[1617]: time=2026-09-21T09:52:44.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6323worker1 # [ 116.318953] nix-grpc-daemon[1617]: time=2026-09-21T09:52:44.321Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6324lb # [ 116.355819] niks3-server[957]: 2026/09/21 09:52:43 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6325lb # [ 116.364579] niks3-server[957]: 2026/09/21 09:52:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6326worker1 # [ 116.356567] nix-grpc-daemon[1617]: time=2026-09-21T09:52:44.361Z level=INFO msg="Uploading 1 narinfos"6327lb # [ 116.371566] niks3-server[957]: 2026/09/21 09:52:43 INFO Signed narinfos id=46 count=16328lb # [ 116.381492] niks3-server[957]: 2026/09/21 09:52:43 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6329lb # [ 116.391746] niks3-server[957]: 2026/09/21 09:52:43 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6330lb # [ 116.397286] niks3-server[957]: 2026/09/21 09:52:43 INFO Completed upload id=466331worker1 # [ 116.382384] nix-grpc-daemon[1617]: time=2026-09-21T09:52:44.387Z level=INFO msg="Upload complete. (282ms)"6332lb # [ 116.403045] niks3-server[957]: 2026/09/21 09:52:43 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6333worker1 # [ 116.387141] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=5046334worker1 # [ 116.391889] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726335worker1 # [ 116.404578] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6336worker1 # [ 116.408125] nix-daemon[857]: reaped child process 1805, status = succeeded6337worker1 # [ 116.411904] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6338lb # [ 116.724573] envoy[914]: conn peer=[2001:db8:1::1]:35496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4029 ms=53776339lb # [ 116.728356] envoy[914]: conn peer=[2001:db8:1::1]:45210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1430 ms=99986340lb # [ 116.731884] envoy[914]: conn peer=[2001:db8:1::4]:43774 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2366 tx=448 ms=269846341worker1 # [ 117.446710] nix-daemon[857]: reaped child process 1819, status = succeeded6342worker1 # [ 117.537743] nix-daemon[857]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6343worker1 # [ 117.549685] nix-daemon[857]: reaped child process 1838, status = succeeded6344lb # [ 117.573599] niks3-server[957]: 2026/09/21 09:52:44 INFO Received uploads request method=POST path=/api/pending_closures6345worker1 # [ 117.566554] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6346worker1 # [ 117.568102] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.572Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6347lb # [ 117.619350] niks3-server[957]: 2026/09/21 09:52:44 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6348lb # [ 117.626532] niks3-server[957]: 2026/09/21 09:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6349lb # [ 117.630879] niks3-server[957]: 2026/09/21 09:52:45 INFO Signed narinfos id=47 count=16350worker1 # [ 117.621401] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.625Z level=INFO msg="Uploading 1 narinfos"6351lb # [ 117.638156] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6352lb # [ 117.643864] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6353lb # [ 117.652314] niks3-server[957]: 2026/09/21 09:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6354lb # [ 117.658329] niks3-server[957]: 2026/09/21 09:52:45 INFO Completed upload id=476355worker1 # [ 117.643326] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.648Z level=INFO msg="Upload complete. (221ms)"6356lb # [ 117.660138] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6357worker1 # [ 117.654381] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16358worker1 # [ 117.660442] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16359worker1 # [ 117.664424] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726360worker1 # [ 117.668264] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06361worker1 # [ 117.678063] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726362worker1 # [ 117.680353] nix-daemon[857]: reaped child process 1817, status = succeeded6363worker1 # [ 117.683879] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726364worker1 # [ 117.687707] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6365worker1 # [ 117.774550] nix-daemon[857]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6366worker1 # [ 117.788881] nix-daemon[857]: reaped child process 1847, status = succeeded6367worker1 # [ 117.873520] nix-daemon[857]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6368worker1 # [ 117.887424] nix-daemon[857]: reaped child process 1852, status = succeeded6369lb # [ 117.909112] niks3-server[957]: 2026/09/21 09:52:45 INFO Received uploads request method=POST path=/api/pending_closures6370worker1 # [ 117.902033] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.906Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6371worker1 # [ 117.903609] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.908Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6372lb # [ 117.941430] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6373lb # [ 117.949886] niks3-server[957]: 2026/09/21 09:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6374lb # [ 117.955813] niks3-server[957]: 2026/09/21 09:52:45 INFO Signed narinfos id=48 count=16375worker1 # [ 117.944343] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.949Z level=INFO msg="Uploading 1 narinfos"6376lb # [ 117.962958] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6377lb # [ 117.975776] niks3-server[957]: 2026/09/21 09:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6378lb # [ 117.983982] niks3-server[957]: 2026/09/21 09:52:45 INFO Completed upload id=486379worker1 # [ 117.971653] nix-grpc-daemon[1617]: time=2026-09-21T09:52:45.976Z level=INFO msg="Upload complete. (278ms)"6380lb # [ 117.987485] niks3-server[957]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6381worker1 # [ 117.974626] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=5046382worker1 # [ 117.979741] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726383worker1 # [ 117.991107] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6384worker1 # [ 117.995673] nix-daemon[857]: reaped child process 1842, status = succeeded6385worker1 # [ 118.000225] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6386worker1 # [ 119.036566] nix-daemon[857]: reaped child process 1856, status = succeeded6387worker1 # [ 119.118094] nix-daemon[857]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6388worker1 # [ 119.131238] nix-daemon[857]: reaped child process 1875, status = succeeded6389lb # [ 119.154219] niks3-server[957]: 2026/09/21 09:52:46 INFO Received uploads request method=POST path=/api/pending_closures6390worker1 # [ 119.147300] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6391worker1 # [ 119.150722] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.153Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6392lb # [ 119.194577] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6393lb # [ 119.200449] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6394lb # [ 119.215800] niks3-server[957]: 2026/09/21 09:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6395lb # [ 119.219737] niks3-server[957]: 2026/09/21 09:52:46 INFO Signed narinfos id=49 count=16396worker1 # [ 119.207381] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.211Z level=INFO msg="Uploading 1 narinfos"6397lb # [ 119.225297] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6398lb # [ 119.236566] niks3-server[957]: 2026/09/21 09:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6399lb # [ 119.239691] niks3-server[957]: 2026/09/21 09:52:46 INFO Completed upload id=496400worker1 # [ 119.224812] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.229Z level=INFO msg="Upload complete. (217ms)"6401lb # [ 119.242829] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6402worker1 # [ 119.233067] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16403worker1 # [ 119.239754] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16404worker1 # [ 119.251361] nix-daemon[857]: reaped child process 1854, status = succeeded6405worker1 # [ 119.275698] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726406worker1 # [ 119.281715] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06407worker1 # [ 119.293857] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726408worker1 # [ 119.304877] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726409worker1 # [ 119.311717] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6410worker1 # [ 119.405178] nix-daemon[857]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6411worker1 # [ 119.418680] nix-daemon[857]: reaped child process 1884, status = succeeded6412worker1 # [ 119.505449] nix-daemon[857]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6413worker1 # [ 119.519673] nix-daemon[857]: reaped child process 1889, status = succeeded6414lb # [ 119.541412] niks3-server[957]: 2026/09/21 09:52:46 INFO Received uploads request method=POST path=/api/pending_closures6415worker1 # [ 119.534703] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.539Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6416worker1 # [ 119.536092] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.539Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6417lb # [ 119.571150] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6418lb # [ 119.580650] niks3-server[957]: 2026/09/21 09:52:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6419lb # [ 119.584823] niks3-server[957]: 2026/09/21 09:52:46 INFO Signed narinfos id=50 count=16420worker1 # [ 119.573106] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.577Z level=INFO msg="Uploading 1 narinfos"6421lb # [ 119.591115] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6422lb # [ 119.604113] niks3-server[957]: 2026/09/21 09:52:46 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6423worker1 # [ 119.595494] nix-grpc-daemon[1617]: time=2026-09-21T09:52:47.600Z level=INFO msg="Upload complete. (273ms)"6424lb # [ 119.610655] niks3-server[957]: 2026/09/21 09:52:46 INFO Completed upload id=506425lb # [ 119.614382] niks3-server[957]: 2026/09/21 09:52:46 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6426worker1 # [ 119.598198] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=5046427worker1 # [ 119.603453] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726428worker1 # [ 119.612117] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6429worker1 # [ 119.618424] nix-daemon[857]: reaped child process 1879, status = succeeded6430worker1 # [ 119.622506] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6431worker1 # [ 120.456733] nix-daemon[857]: reaped child process 1893, status = succeeded6432worker1 # [ 120.543180] nix-daemon[857]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6433worker1 # [ 120.556081] nix-daemon[857]: reaped child process 1912, status = succeeded6434lb # [ 120.579783] niks3-server[957]: 2026/09/21 09:52:47 INFO Received uploads request method=POST path=/api/pending_closures6435worker1 # [ 120.572882] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6436worker1 # [ 120.575341] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.579Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6437lb # [ 120.617306] niks3-server[957]: 2026/09/21 09:52:47 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6438lb # [ 120.627699] niks3-server[957]: 2026/09/21 09:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6439worker1 # [ 120.622044] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.626Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 120.637088] niks3-server[957]: 2026/09/21 09:52:48 INFO Signed narinfos id=51 count=16441lb # [ 120.640112] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6442lb # [ 120.651900] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6443lb # [ 120.654274] niks3-server[957]: 2026/09/21 09:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6444lb # [ 120.657339] niks3-server[957]: 2026/09/21 09:52:48 INFO Completed upload id=516445lb # [ 120.659202] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6446worker1 # [ 120.643543] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.648Z level=INFO msg="Upload complete. (214ms)"6447worker1 # [ 120.653634] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16448worker1 # [ 120.661208] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16449worker1 # [ 120.666366] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726450worker1 # [ 120.669515] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06451worker1 # [ 120.679300] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726452worker1 # [ 120.682500] nix-daemon[857]: reaped child process 1891, status = succeeded6453worker1 # [ 120.687162] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.691Z level=INFO msg="All 1 paths already cached"6454worker1 # [ 120.691425] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726455worker1 # [ 120.695156] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6456worker1 # [ 120.782355] nix-daemon[857]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6457worker1 # [ 120.795526] nix-daemon[857]: reaped child process 1921, status = succeeded6458worker1 # [ 120.882807] nix-daemon[857]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)6459worker1 # [ 120.897179] nix-daemon[857]: reaped child process 1926, status = succeeded6460lb # [ 120.918315] niks3-server[957]: 2026/09/21 09:52:48 INFO Received uploads request method=POST path=/api/pending_closures6461worker1 # [ 120.911447] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.916Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6462worker1 # [ 120.912995] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.917Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6463lb # [ 120.950388] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6464lb # [ 120.959868] niks3-server[957]: 2026/09/21 09:52:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6465worker1 # [ 120.950625] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.955Z level=INFO msg="Uploading 1 narinfos"6466lb # [ 120.966140] niks3-server[957]: 2026/09/21 09:52:48 INFO Signed narinfos id=52 count=16467lb # [ 120.973126] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6468lb # [ 120.984321] niks3-server[957]: 2026/09/21 09:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6469lb # [ 120.992894] niks3-server[957]: 2026/09/21 09:52:48 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6470lb # [ 120.998900] niks3-server[957]: 2026/09/21 09:52:48 INFO Completed upload id=526471worker1 # [ 120.983987] nix-grpc-daemon[1617]: time=2026-09-21T09:52:48.988Z level=INFO msg="Upload complete. (283ms)"6472worker1 # [ 120.987965] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=6246473worker1 # [ 120.994698] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726474worker1 # [ 121.004218] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6475worker1 # [ 121.014318] nix-daemon[857]: reaped child process 1916, status = succeeded6476worker1 # [ 121.020600] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6477worker1 # [ 122.460416] nix-daemon[857]: reaped child process 1930, status = succeeded6478worker1 # [ 122.547136] nix-daemon[857]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6479worker1 # [ 122.559584] nix-daemon[857]: reaped child process 1949, status = succeeded6480lb # [ 122.583755] niks3-server[957]: 2026/09/21 09:52:49 INFO Received uploads request method=POST path=/api/pending_closures6481worker1 # [ 122.577727] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.582Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6482worker1 # [ 122.580419] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.583Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6483lb # [ 122.622542] niks3-server[957]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6484lb # [ 122.629895] niks3-server[957]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6485lb # [ 122.639139] niks3-server[957]: 2026/09/21 09:52:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6486lb # [ 122.645406] niks3-server[957]: 2026/09/21 09:52:50 INFO Signed narinfos id=53 count=16487worker1 # [ 122.630444] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.635Z level=INFO msg="Uploading 1 narinfos"6488lb # [ 122.650840] niks3-server[957]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6489lb # [ 122.654535] niks3-server[957]: 2026/09/21 09:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6490lb # [ 122.658415] niks3-server[957]: 2026/09/21 09:52:50 INFO Completed upload id=536491worker1 # [ 122.643543] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.648Z level=INFO msg="Upload complete. (215ms)"6492lb # [ 122.661800] niks3-server[957]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6493worker1 # [ 122.649773] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16494worker1 # [ 122.656527] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16495worker1 # [ 122.661997] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726496worker1 # [ 122.667125] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06497worker1 # [ 122.673277] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726498worker1 # [ 122.675870] nix-daemon[857]: reaped child process 1928, status = succeeded6499worker1 # [ 122.679754] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.684Z level=INFO msg="All 1 paths already cached"6500worker1 # [ 122.683512] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726501worker1 # [ 122.688201] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6502worker1 # [ 122.773257] nix-daemon[857]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6503worker1 # [ 122.787939] nix-daemon[857]: reaped child process 1958, status = succeeded6504worker1 # [ 122.872428] nix-daemon[857]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6505worker1 # [ 122.886129] nix-daemon[857]: reaped child process 1963, status = succeeded6506lb # [ 122.908135] niks3-server[957]: 2026/09/21 09:52:50 INFO Received uploads request method=POST path=/api/pending_closures6507worker1 # [ 122.903555] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.908Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6508worker1 # [ 122.904975] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.908Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6509lb # [ 122.935085] niks3-server[957]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6510lb # [ 122.946893] niks3-server[957]: 2026/09/21 09:52:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6511lb # [ 122.952783] niks3-server[957]: 2026/09/21 09:52:50 INFO Signed narinfos id=54 count=16512worker1 # [ 122.939950] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.944Z level=INFO msg="Uploading 1 narinfos"6513lb # [ 122.959051] niks3-server[957]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6514lb # [ 122.968706] niks3-server[957]: 2026/09/21 09:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6515lb # [ 122.971785] niks3-server[957]: 2026/09/21 09:52:50 INFO Completed upload id=546516worker1 # [ 122.956860] nix-grpc-daemon[1617]: time=2026-09-21T09:52:50.961Z level=INFO msg="Upload complete. (264ms)"6517lb # [ 122.974842] niks3-server[957]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6518worker1 # [ 122.960769] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=6246519worker1 # [ 122.966259] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726520worker1 # [ 122.972854] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6521worker1 # [ 122.979651] nix-daemon[857]: reaped child process 1953, status = succeeded6522worker1 # [ 122.983792] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6523worker1 # [ 124.221267] nix-daemon[857]: reaped child process 1967, status = succeeded6524worker1 # [ 124.297657] nix-daemon[857]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)6525worker1 # [ 124.312128] nix-daemon[857]: reaped child process 1986, status = succeeded6526lb # [ 124.334788] niks3-server[957]: 2026/09/21 09:52:51 INFO Received uploads request method=POST path=/api/pending_closures6527worker1 # [ 124.327876] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.332Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6528worker1 # [ 124.330258] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.334Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6529lb # [ 124.370080] niks3-server[957]: 2026/09/21 09:52:51 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6530lb # [ 124.386852] niks3-server[957]: 2026/09/21 09:52:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6531lb # [ 124.398847] niks3-server[957]: 2026/09/21 09:52:51 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6532lb # [ 124.403176] niks3-server[957]: 2026/09/21 09:52:51 INFO Signed narinfos id=55 count=16533worker1 # [ 124.390865] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.395Z level=INFO msg="Uploading 1 narinfos"6534lb # [ 124.409868] niks3-server[957]: 2026/09/21 09:52:51 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6535lb # [ 124.416157] niks3-server[957]: 2026/09/21 09:52:51 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6536lb # [ 124.419597] niks3-server[957]: 2026/09/21 09:52:51 INFO Completed upload id=556537worker1 # [ 124.404702] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.409Z level=INFO msg="Upload complete. (212ms)"6538lb # [ 124.422452] niks3-server[957]: 2026/09/21 09:52:51 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6539worker1 # [ 124.412539] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16540worker1 # [ 124.422731] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16541worker1 # [ 124.427383] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726542worker1 # [ 124.431205] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06543worker1 # [ 124.442338] nix-daemon[857]: reaped child process 1965, status = succeeded6544worker1 # [ 124.456643] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726545worker1 # [ 124.467702] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.472Z level=INFO msg="All 1 paths already cached"6546worker1 # [ 124.473972] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726547worker1 # [ 124.482130] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6548worker1 # [ 124.570038] nix-daemon[857]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6549worker1 # [ 124.584987] nix-daemon[857]: reaped child process 1995, status = succeeded6550worker1 # [ 124.667467] nix-daemon[857]: accepted connection from pid 1997, user nix-grpc-daemon (trusted)6551worker1 # [ 124.680976] nix-daemon[857]: reaped child process 2000, status = succeeded6552lb # [ 124.703833] niks3-server[957]: 2026/09/21 09:52:52 INFO Received uploads request method=POST path=/api/pending_closures6553worker1 # [ 124.696721] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.701Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6554worker1 # [ 124.698645] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.703Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6555lb # [ 124.735845] niks3-server[957]: 2026/09/21 09:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6556worker1 # [ 124.728591] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.733Z level=INFO msg="Uploading 1 narinfos"6557lb # [ 124.743550] niks3-server[957]: 2026/09/21 09:52:52 INFO Signed narinfos id=56 count=16558lb # [ 124.753078] niks3-server[957]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6559lb # [ 124.760154] niks3-server[957]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6560lb # [ 124.774389] niks3-server[957]: 2026/09/21 09:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6561lb # [ 124.781067] niks3-server[957]: 2026/09/21 09:52:52 INFO Completed upload id=566562worker1 # [ 124.766387] nix-grpc-daemon[1617]: time=2026-09-21T09:52:52.770Z level=INFO msg="Upload complete. (278ms)"6563lb # [ 124.784637] niks3-server[957]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6564worker1 # [ 124.771328] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=6246565worker1 # [ 124.777385] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726566worker1 # [ 124.787485] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6567worker1 # [ 124.799702] nix-daemon[857]: reaped child process 1990, status = succeeded6568worker1 # [ 124.802594] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6569worker1 # [ 126.040583] nix-daemon[857]: reaped child process 2004, status = succeeded6570worker1 # [ 126.111194] nix-daemon[857]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)6571worker1 # [ 126.124709] nix-daemon[857]: reaped child process 2023, status = succeeded6572lb # [ 126.148686] niks3-server[957]: 2026/09/21 09:52:53 INFO Received uploads request method=POST path=/api/pending_closures6573worker1 # [ 126.142304] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.146Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6574worker1 # [ 126.143758] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.148Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6575lb # [ 126.191189] niks3-server[957]: 2026/09/21 09:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6576lb # [ 126.200136] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6577worker1 # [ 126.191559] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.196Z level=INFO msg="Uploading 1 narinfos"6578lb # [ 126.206997] niks3-server[957]: 2026/09/21 09:52:53 INFO Signed narinfos id=57 count=16579lb # [ 126.212752] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6580lb # [ 126.219447] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6581lb # [ 126.227796] niks3-server[957]: 2026/09/21 09:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6582lb # [ 126.234545] niks3-server[957]: 2026/09/21 09:52:53 INFO Completed upload id=576583worker1 # [ 126.219726] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.224Z level=INFO msg="Upload complete. (208ms)"6584lb # [ 126.236505] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6585worker1 # [ 126.229484] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16586worker1 # [ 126.238562] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16587worker1 # [ 126.242565] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726588worker1 # [ 126.246206] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06589worker1 # [ 126.255539] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726590worker1 # [ 126.260101] nix-daemon[857]: reaped child process 2002, status = succeeded6591worker1 # [ 126.263642] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.268Z level=INFO msg="All 2 paths already cached"6592worker1 # [ 126.268285] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726593worker1 # [ 126.277333] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6594worker1 # [ 126.364091] nix-daemon[857]: accepted connection from pid 2029, user nix-grpc-daemon (trusted)6595worker1 # [ 126.378185] nix-daemon[857]: reaped child process 2032, status = succeeded6596worker1 # [ 126.462463] nix-daemon[857]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6597worker1 # [ 126.475986] nix-daemon[857]: reaped child process 2037, status = succeeded6598lb # [ 126.498992] niks3-server[957]: 2026/09/21 09:52:53 INFO Received uploads request method=POST path=/api/pending_closures6599worker1 # [ 126.491866] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.496Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6600worker1 # [ 126.493223] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.496Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6601lb # [ 126.524871] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6602lb # [ 126.536941] niks3-server[957]: 2026/09/21 09:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6603worker1 # [ 126.526659] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.531Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 126.542171] niks3-server[957]: 2026/09/21 09:52:53 INFO Signed narinfos id=58 count=16605lb # [ 126.549120] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6606lb # [ 126.562663] niks3-server[957]: 2026/09/21 09:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6607worker1 # [ 126.553422] nix-grpc-daemon[1617]: time=2026-09-21T09:52:54.558Z level=INFO msg="Upload complete. (270ms)"6608lb # [ 126.568823] niks3-server[957]: 2026/09/21 09:52:53 INFO Completed upload id=586609worker1 # [ 126.555976] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=7526610lb # [ 126.572994] niks3-server[957]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6611worker1 # [ 126.561787] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726612worker1 # [ 126.570234] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6613worker1 # [ 126.576206] nix-daemon[857]: reaped child process 2027, status = succeeded6614worker1 # [ 126.580891] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6615worker1 # [ 127.804839] nix-daemon[857]: reaped child process 2041, status = succeeded6616worker1 # [ 127.888228] nix-daemon[857]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6617worker1 # [ 127.901642] nix-daemon[857]: reaped child process 2060, status = succeeded6618lb # [ 127.925803] niks3-server[957]: 2026/09/21 09:52:55 INFO Received uploads request method=POST path=/api/pending_closures6619worker1 # [ 127.920231] nix-grpc-daemon[1617]: time=2026-09-21T09:52:55.924Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6620worker1 # [ 127.921743] nix-grpc-daemon[1617]: time=2026-09-21T09:52:55.926Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6621lb # [ 127.966740] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6622lb # [ 127.976319] niks3-server[957]: 2026/09/21 09:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6623lb # [ 127.979859] niks3-server[957]: 2026/09/21 09:52:55 INFO Signed narinfos id=59 count=16624worker1 # [ 127.967380] nix-grpc-daemon[1617]: time=2026-09-21T09:52:55.971Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 127.984523] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6626lb # [ 127.995565] niks3-server[957]: 2026/09/21 09:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6627lb # [ 127.998262] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6628lb # [ 128.001725] niks3-server[957]: 2026/09/21 09:52:55 INFO Completed upload id=596629worker1 # [ 127.986708] nix-grpc-daemon[1617]: time=2026-09-21T09:52:55.991Z level=INFO msg="Upload complete. (196ms)"6630lb # [ 128.004106] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6631worker1 # [ 127.993820] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16632worker1 # [ 128.000466] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:55Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16633worker1 # [ 128.011185] nix-daemon[857]: reaped child process 2039, status = succeeded6634worker1 # [ 128.036885] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726635worker1 # [ 128.049115] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726636worker1 # [ 128.061981] nix-grpc-daemon[1617]: time=2026-09-21T09:52:56.066Z level=INFO msg="All 2 paths already cached"6637worker1 # [ 128.069392] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726638worker1 # [ 128.079167] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6639worker1 # [ 128.175072] nix-daemon[857]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6640worker1 # [ 128.188952] nix-daemon[857]: reaped child process 2067, status = succeeded6641worker1 # [ 128.275421] nix-daemon[857]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6642worker1 # [ 128.289607] nix-daemon[857]: reaped child process 2072, status = succeeded6643lb # [ 128.311378] niks3-server[957]: 2026/09/21 09:52:55 INFO Received uploads request method=POST path=/api/pending_closures6644worker1 # [ 128.304497] nix-grpc-daemon[1617]: time=2026-09-21T09:52:56.308Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6645worker1 # [ 128.305966] nix-grpc-daemon[1617]: time=2026-09-21T09:52:56.309Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6646lb # [ 128.337173] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6647lb # [ 128.350519] niks3-server[957]: 2026/09/21 09:52:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6648worker1 # [ 128.341911] nix-grpc-daemon[1617]: time=2026-09-21T09:52:56.346Z level=INFO msg="Uploading 1 narinfos"6649lb # [ 128.356795] niks3-server[957]: 2026/09/21 09:52:55 INFO Signed narinfos id=60 count=16650lb # [ 128.363590] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6651lb # [ 128.375246] niks3-server[957]: 2026/09/21 09:52:55 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6652lb # [ 128.380939] niks3-server[957]: 2026/09/21 09:52:55 INFO Completed upload id=606653worker1 # [ 128.366261] nix-grpc-daemon[1617]: time=2026-09-21T09:52:56.370Z level=INFO msg="Upload complete. (276ms)"6654lb # [ 128.384312] niks3-server[957]: 2026/09/21 09:52:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6655worker1 # [ 128.368553] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=7526656worker1 # [ 128.373336] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726657worker1 # [ 128.383755] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6658worker1 # [ 128.388420] nix-daemon[857]: reaped child process 2062, status = succeeded6659worker1 # [ 128.392926] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6660worker1 # [ 129.636161] nix-daemon[857]: reaped child process 2076, status = succeeded6661worker1 # [ 129.713203] nix-daemon[857]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)6662worker1 # [ 129.727787] nix-daemon[857]: reaped child process 2095, status = succeeded6663lb # [ 129.749995] niks3-server[957]: 2026/09/21 09:52:57 INFO Received uploads request method=POST path=/api/pending_closures6664worker1 # [ 129.743223] nix-grpc-daemon[1617]: time=2026-09-21T09:52:57.747Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6665worker1 # [ 129.744730] nix-grpc-daemon[1617]: time=2026-09-21T09:52:57.749Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6666lb # [ 129.788455] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6667lb # [ 129.804417] niks3-server[957]: 2026/09/21 09:52:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6668lb # [ 129.812469] niks3-server[957]: 2026/09/21 09:52:57 INFO Signed narinfos id=61 count=16669worker1 # [ 129.799909] nix-grpc-daemon[1617]: time=2026-09-21T09:52:57.804Z level=INFO msg="Uploading 1 narinfos"6670lb # [ 129.815899] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6671lb # [ 129.821872] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6672lb # [ 129.831318] niks3-server[957]: 2026/09/21 09:52:57 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6673lb # [ 129.835709] niks3-server[957]: 2026/09/21 09:52:57 INFO Completed upload id=616674lb # [ 129.837739] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6675worker1 # [ 129.822128] nix-grpc-daemon[1617]: time=2026-09-21T09:52:57.826Z level=INFO msg="Upload complete. (221ms)"6676worker1 # [ 129.830545] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16677worker1 # [ 129.838315] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:57Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16678worker1 # [ 129.849481] nix-daemon[857]: reaped child process 2074, status = succeeded6679worker1 # [ 129.872367] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726680worker1 # [ 129.884707] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726681worker1 # [ 129.897789] nix-grpc-daemon[1617]: time=2026-09-21T09:52:57.902Z level=INFO msg="All 2 paths already cached"6682worker1 # [ 129.906113] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:373726683worker1 # [ 129.913696] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6684worker1 # [ 130.010537] nix-daemon[857]: accepted connection from pid 2099, user nix-grpc-daemon (trusted)6685worker1 # [ 130.023592] nix-daemon[857]: reaped child process 2102, status = succeeded6686worker1 # [ 130.110385] nix-daemon[857]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)6687worker1 # [ 130.123652] nix-daemon[857]: reaped child process 2107, status = succeeded6688lb # [ 130.146987] niks3-server[957]: 2026/09/21 09:52:57 INFO Received uploads request method=POST path=/api/pending_closures6689worker1 # [ 130.139988] nix-grpc-daemon[1617]: time=2026-09-21T09:52:58.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6690worker1 # [ 130.141955] nix-grpc-daemon[1617]: time=2026-09-21T09:52:58.146Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6691lb # [ 130.174350] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6692lb # [ 130.182555] niks3-server[957]: 2026/09/21 09:52:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6693lb # [ 130.186998] niks3-server[957]: 2026/09/21 09:52:57 INFO Signed narinfos id=62 count=16694worker1 # [ 130.175473] nix-grpc-daemon[1617]: time=2026-09-21T09:52:58.180Z level=INFO msg="Uploading 1 narinfos"6695lb # [ 130.201479] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6696lb # [ 130.208542] niks3-server[957]: 2026/09/21 09:52:57 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6697worker1 # [ 130.199538] nix-grpc-daemon[1617]: time=2026-09-21T09:52:58.204Z level=INFO msg="Upload complete. (273ms)"6698lb # [ 130.214727] niks3-server[957]: 2026/09/21 09:52:57 INFO Completed upload id=626699worker1 # [ 130.202479] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=0 paths=1 nar_bytes_in=7526700lb # [ 130.217564] niks3-server[957]: 2026/09/21 09:52:57 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6701worker1 # [ 130.209349] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:373726702worker1 # [ 130.211162] nix-daemon[857]: reaped child process 2097, status = succeeded6703worker1 # [ 130.214131] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6704worker1 # [ 130.220859] nix-daemon[857]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6705worker1 # [ 131.236321] nix-daemon[857]: reaped child process 2111, status = succeeded6706worker1 # [ 131.316091] nix-daemon[857]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)6707worker1 # [ 131.328610] nix-daemon[857]: reaped child process 2130, status = succeeded6708lb # [ 131.352587] niks3-server[957]: 2026/09/21 09:52:58 INFO Received uploads request method=POST path=/api/pending_closures6709worker1 # [ 131.347317] nix-grpc-daemon[1617]: time=2026-09-21T09:52:59.351Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6710worker1 # [ 131.348902] nix-grpc-daemon[1617]: time=2026-09-21T09:52:59.353Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6711lb # [ 131.386423] niks3-server[957]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6712lb # [ 131.399577] niks3-server[957]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6713lb # [ 131.410153] niks3-server[957]: 2026/09/21 09:52:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6714lb # [ 131.415553] niks3-server[957]: 2026/09/21 09:52:58 INFO Signed narinfos id=63 count=16715worker1 # [ 131.403482] nix-grpc-daemon[1617]: time=2026-09-21T09:52:59.408Z level=INFO msg="Uploading 1 narinfos"6716lb # [ 131.422441] niks3-server[957]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6717lb # [ 131.432791] niks3-server[957]: 2026/09/21 09:52:58 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6718lb # [ 131.436967] niks3-server[957]: 2026/09/21 09:52:58 INFO Completed upload id=636719worker1 # [ 131.421883] nix-grpc-daemon[1617]: time=2026-09-21T09:52:59.426Z level=INFO msg="Upload complete. (192ms)"6720lb # [ 131.439111] niks3-server[957]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6721worker1 # [ 131.427869] nix-grpc-daemon[1537]: ts=2026-09-21T09:52:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37372 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16722worker1 # [ 131.442618] nix-daemon[857]: reaped child process 2109, status = succeeded6723client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.66 seconds)6724worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6725worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6726client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'6727client # [ 131.589716] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6728client # Running as unit: gpu.service; invocation ID: e85ea2f6bc014919b7547b9c8fd7ab906729client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)6730lb: must succeed: curl -sf localhost:9901/clusters6731lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6732lb: must succeed: curl -sf localhost:9901/clusters6733lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6734worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6735worker2 # [ 131.713811] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579286736client # [ 131.723184] nix[1348]: don't know how to build these paths:6737client # [ 131.724105] nix[1348]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6738worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6739worker2 # [ 131.723336] nix-grpc-daemon[1692]: ts=2026-09-21T09:52:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579286740worker2 # [ 131.733696] nix-grpc-daemon[1692]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)6741worker2 # [ 131.807189] nix-grpc-daemon[1692]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5)6742worker2 # [ 131.964332] nix-grpc-daemon[1692]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 108 ms (attempt 3/5)6743worker2 # [ 132.079174] nix-grpc-daemon[1692]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 568 ms (attempt 4/5)6744worker2 # [ 132.650193] nix-grpc-daemon[1692]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6745worker2 # [ 132.653108] nix-grpc-daemon[1692]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6746worker1 # [ 132.661798] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:00Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06747client # [ 132.684504] nix[1348]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6748lb: must succeed: curl -sf localhost:9901/clusters6749lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6750lb: must succeed: curl -sf localhost:9901/clusters6751lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6752worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6753worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6754client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6755lb # [ 136.724411] envoy[914]: conn peer=[2001:db8:1::1]:51226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27094 tx=11344 ms=154816756client # [ 162.687272] nix[1348]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6757client # [ 162.694495] nix[1348]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6758client # [ 162.720810] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6759client # [ 162.724431] systemd[1]: gpu.service: Failed with result 'exit-code'.6760client # [ 162.729180] systemd[1]: gpu.service: Consumed 142ms CPU time over 31.135s wall clock time, 35.6M memory peak, 5.4K incoming IP traffic, 5.1K outgoing IP traffic.6761client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 30.51 seconds)6762client: must succeed: journalctl -u gpu -o cat6763client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6764lb: must succeed: curl -sf localhost:9901/clusters6765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6766lb: must succeed: curl -sf localhost:9901/clusters6767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6768worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6769worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6770(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.66 seconds)6771subtest: low disk drains a worker and builds go to the other6772worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6773worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)6774worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6775worker1 # [ 164.852952] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:32Z level=info event=unhealthy reason=min_free6776worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)6777client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&26778worker2 # [ 165.677209] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579286779client # don't know how to build these paths:6780client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6781worker1 # [ 165.668215] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:373726782worker1 # [ 165.689613] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:33Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06783worker1 # [ 165.693747] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:33Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16784worker2 # [ 165.752973] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286785worker2 # [ 165.765871] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579286786client # copying 1 paths to 'lb:50051'...6787client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6788worker2 # [ 165.778643] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579286789worker2 # [ 165.790647] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6790worker2 # [ 165.810176] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:33Z level=info event=niks3_push_started pid=17526791worker2 # [ 165.915646] nix-grpc-daemon[1752]: time=2026-09-21T09:53:33.977Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6792worker2 # [ 165.997328] nix-daemon[849]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6793worker2 # [ 166.008764] nix-daemon[849]: reaped child process 1768, status = succeeded6794worker2 # [ 166.088574] nix-daemon[849]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6795worker2 # [ 166.101065] nix-daemon[849]: reaped child process 1773, status = succeeded6796lb # [ 166.103947] niks3-server[957]: 2026/09/21 09:53:33 INFO Received uploads request method=POST path=/api/pending_closures6797worker2 # [ 166.117723] nix-grpc-daemon[1752]: time=2026-09-21T09:53:34.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6798worker2 # [ 166.120681] nix-grpc-daemon[1752]: time=2026-09-21T09:53:34.182Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6799lb # [ 166.142400] niks3-server[957]: 2026/09/21 09:53:33 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6800lb # [ 166.150794] niks3-server[957]: 2026/09/21 09:53:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6801lb # [ 166.156484] niks3-server[957]: 2026/09/21 09:53:33 INFO Signed narinfos id=64 count=16802worker2 # [ 166.163321] nix-grpc-daemon[1752]: time=2026-09-21T09:53:34.226Z level=INFO msg="Uploading 1 narinfos"6803lb # [ 166.169707] niks3-server[957]: 2026/09/21 09:53:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6804lb # [ 166.178791] niks3-server[957]: 2026/09/21 09:53:33 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6805lb # [ 166.184622] niks3-server[957]: 2026/09/21 09:53:33 INFO Completed upload id=646806worker2 # [ 166.188599] nix-grpc-daemon[1752]: time=2026-09-21T09:53:34.251Z level=INFO msg="Upload complete. (274ms)"6807worker2 # [ 166.192197] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=0 paths=1 nar_bytes_in=5126808lb # [ 166.191725] niks3-server[957]: 2026/09/21 09:53:33 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6809worker2 # [ 166.198292] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286810worker2 # [ 166.206876] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6811worker2 # [ 166.214757] nix-daemon[849]: reaped child process 1750, status = succeeded6812worker2 # [ 166.218867] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6813client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6814client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6815lb # [ 166.726727] envoy[914]: conn peer=[2001:db8:1::1]:36188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=912 tx=539 ms=309836816client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6817client # farm-a> LOG-farm-a-drain6818worker2 # [ 167.254749] nix-daemon[849]: reaped child process 1777, status = succeeded6819worker2 # [ 167.338894] nix-daemon[849]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6820worker2 # [ 167.349931] nix-daemon[849]: reaped child process 1796, status = succeeded6821lb # [ 167.354122] niks3-server[957]: 2026/09/21 09:53:34 INFO Received uploads request method=POST path=/api/pending_closures6822worker2 # [ 167.368444] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.431Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6823worker2 # [ 167.370172] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.433Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6824lb # [ 167.399169] niks3-server[957]: 2026/09/21 09:53:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6825lb # [ 167.407303] niks3-server[957]: 2026/09/21 09:53:34 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6826worker2 # [ 167.416840] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.479Z level=INFO msg="Uploading 1 narinfos"6827lb # [ 167.413316] niks3-server[957]: 2026/09/21 09:53:34 INFO Signed narinfos id=65 count=16828lb # [ 167.415385] niks3-server[957]: 2026/09/21 09:53:34 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6829lb # [ 167.419964] niks3-server[957]: 2026/09/21 09:53:34 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6830lb # [ 167.427626] niks3-server[957]: 2026/09/21 09:53:34 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6831lb # [ 167.431889] niks3-server[957]: 2026/09/21 09:53:34 INFO Completed upload id=656832worker2 # [ 167.435961] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.499Z level=INFO msg="Upload complete. (210ms)"6833lb # [ 167.433621] niks3-server[957]: 2026/09/21 09:53:34 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6834worker2 # [ 167.440708] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16835worker1 # [ 167.426494] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:35Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06836worker1 # [ 167.428429] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:35Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16837worker2 # [ 167.453910] nix-daemon[849]: reaped child process 1775, status = succeeded6838worker2 # [ 167.489199] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286839worker2 # [ 167.500232] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579286840worker2 # [ 167.513312] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.576Z level=INFO msg="All 1 paths already cached"6841client # copying 1 paths to 'lb:50051'...6842client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6843worker2 # [ 167.521350] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579286844worker2 # [ 167.529477] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6845worker2 # [ 167.623971] nix-daemon[849]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6846worker2 # [ 167.636377] nix-daemon[849]: reaped child process 1803, status = succeeded6847worker2 # [ 167.716037] nix-daemon[849]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6848worker2 # [ 167.728646] nix-daemon[849]: reaped child process 1808, status = succeeded6849lb # [ 167.730761] niks3-server[957]: 2026/09/21 09:53:35 INFO Received uploads request method=POST path=/api/pending_closures6850worker2 # [ 167.743350] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6851worker2 # [ 167.744836] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.808Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6852lb # [ 167.763360] niks3-server[957]: 2026/09/21 09:53:35 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6853lb # [ 167.772597] niks3-server[957]: 2026/09/21 09:53:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6854worker2 # [ 167.781074] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.844Z level=INFO msg="Uploading 1 narinfos"6855lb # [ 167.777114] niks3-server[957]: 2026/09/21 09:53:35 INFO Signed narinfos id=66 count=16856lb # [ 167.784511] niks3-server[957]: 2026/09/21 09:53:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6857lb # [ 167.796233] niks3-server[957]: 2026/09/21 09:53:35 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6858lb # [ 167.803819] niks3-server[957]: 2026/09/21 09:53:35 INFO Completed upload id=666859worker2 # [ 167.807682] nix-grpc-daemon[1752]: time=2026-09-21T09:53:35.871Z level=INFO msg="Upload complete. (265ms)"6860lb # [ 167.807290] niks3-server[957]: 2026/09/21 09:53:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6861worker2 # [ 167.812130] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=0 paths=1 nar_bytes_in=6246862worker2 # [ 167.817320] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286863worker2 # [ 167.825646] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6864worker2 # [ 167.832617] nix-daemon[849]: reaped child process 1798, status = succeeded6865worker2 # [ 167.837506] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6866client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6867client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6868client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6869client # farm-b> LOG-farm-b-drain6870worker2 # [ 168.473248] nix-daemon[849]: reaped child process 1812, status = succeeded6871worker2 # [ 168.550253] nix-daemon[849]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6872worker2 # [ 168.561374] nix-daemon[849]: reaped child process 1831, status = succeeded6873lb # [ 168.565260] niks3-server[957]: 2026/09/21 09:53:35 INFO Received uploads request method=POST path=/api/pending_closures6874worker2 # [ 168.577237] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.640Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6875worker2 # [ 168.578826] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.642Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6876lb # [ 168.596267] niks3-server[957]: 2026/09/21 09:53:35 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6877lb # [ 168.609231] niks3-server[957]: 2026/09/21 09:53:35 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6878lb # [ 168.623623] niks3-server[957]: 2026/09/21 09:53:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6879worker2 # [ 168.632733] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.696Z level=INFO msg="Uploading 1 narinfos"6880lb # [ 168.629629] niks3-server[957]: 2026/09/21 09:53:35 INFO Signed narinfos id=67 count=16881lb # [ 168.633602] niks3-server[957]: 2026/09/21 09:53:36 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6882lb # [ 168.642773] niks3-server[957]: 2026/09/21 09:53:36 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6883lb # [ 168.646679] niks3-server[957]: 2026/09/21 09:53:36 INFO Completed upload id=676884worker2 # [ 168.650519] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.713Z level=INFO msg="Upload complete. (206ms)"6885lb # [ 168.648489] niks3-server[957]: 2026/09/21 09:53:36 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6886worker2 # [ 168.657152] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16887worker1 # [ 168.643917] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:36Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06888worker2 # [ 168.665195] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286889worker1 # [ 168.648491] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:36Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16890worker2 # [ 168.677520] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579286891worker2 # [ 168.682490] nix-daemon[849]: reaped child process 1810, status = succeeded6892worker2 # [ 168.687935] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.751Z level=INFO msg="All 2 paths already cached"6893client # copying 1 paths to 'lb:50051'...6894client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6895worker2 # [ 168.694645] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579286896worker2 # [ 168.700895] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6897worker2 # [ 168.779932] nix-daemon[849]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6898worker2 # [ 168.793174] nix-daemon[849]: reaped child process 1838, status = succeeded6899worker2 # [ 168.865907] nix-daemon[849]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)6900worker2 # [ 168.879037] nix-daemon[849]: reaped child process 1843, status = succeeded6901lb # [ 168.879404] niks3-server[957]: 2026/09/21 09:53:36 INFO Received uploads request method=POST path=/api/pending_closures6902worker2 # [ 168.891600] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.954Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6903worker2 # [ 168.893135] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.954Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6904lb # [ 168.905507] niks3-server[957]: 2026/09/21 09:53:36 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6905lb # [ 168.916111] niks3-server[957]: 2026/09/21 09:53:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6906lb # [ 168.920684] niks3-server[957]: 2026/09/21 09:53:36 INFO Signed narinfos id=68 count=16907worker2 # [ 168.927484] nix-grpc-daemon[1752]: time=2026-09-21T09:53:36.990Z level=INFO msg="Uploading 1 narinfos"6908lb # [ 168.932169] niks3-server[957]: 2026/09/21 09:53:36 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6909lb # [ 168.941130] niks3-server[957]: 2026/09/21 09:53:36 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6910lb # [ 168.947454] niks3-server[957]: 2026/09/21 09:53:36 INFO Completed upload id=686911worker2 # [ 168.951362] nix-grpc-daemon[1752]: time=2026-09-21T09:53:37.014Z level=INFO msg="Upload complete. (242ms)"6912worker2 # [ 168.953501] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=0 paths=1 nar_bytes_in=7526913lb # [ 168.953479] niks3-server[957]: 2026/09/21 09:53:36 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6914worker2 # [ 168.957891] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579286915worker2 # [ 168.965485] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6916worker2 # [ 168.972529] nix-daemon[849]: reaped child process 1833, status = succeeded6917client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6918worker2 # [ 168.976161] nix-daemon[849]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6919client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6920client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6921client # farm-top> LOG-farm-top-drain6922worker2 # [ 170.210198] nix-daemon[849]: reaped child process 1847, status = succeeded6923worker2 # [ 170.274186] nix-daemon[849]: accepted connection from pid 1863, user nix-grpc-daemon (trusted)6924worker2 # [ 170.285281] nix-daemon[849]: reaped child process 1866, status = succeeded6925lb # [ 170.289226] niks3-server[957]: 2026/09/21 09:53:37 INFO Received uploads request method=POST path=/api/pending_closures6926worker2 # [ 170.301413] nix-grpc-daemon[1752]: time=2026-09-21T09:53:38.364Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6927worker2 # [ 170.303050] nix-grpc-daemon[1752]: time=2026-09-21T09:53:38.366Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6928lb # [ 170.324684] niks3-server[957]: 2026/09/21 09:53:37 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6929lb # [ 170.335196] niks3-server[957]: 2026/09/21 09:53:37 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6930lb # [ 170.344858] niks3-server[957]: 2026/09/21 09:53:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6931lb # [ 170.350579] niks3-server[957]: 2026/09/21 09:53:37 INFO Signed narinfos id=69 count=16932worker2 # [ 170.357218] nix-grpc-daemon[1752]: time=2026-09-21T09:53:38.420Z level=INFO msg="Uploading 1 narinfos"6933lb # [ 170.354942] niks3-server[957]: 2026/09/21 09:53:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6934lb # [ 170.367220] niks3-server[957]: 2026/09/21 09:53:37 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6935lb # [ 170.374567] niks3-server[957]: 2026/09/21 09:53:37 INFO Completed upload id=696936lb # [ 170.375945] niks3-server[957]: 2026/09/21 09:53:37 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6937worker2 # [ 170.378561] nix-grpc-daemon[1752]: time=2026-09-21T09:53:38.441Z level=INFO msg="Upload complete. (196ms)"6938worker2 # [ 170.385270] nix-grpc-daemon[1692]: ts=2026-09-21T09:53:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57928 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16939worker2 # [ 170.398562] nix-daemon[849]: reaped child process 1845, status = succeeded6940client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.89 seconds)6941client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath6942client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)6943worker1: must succeed: rm /nix/.rw-store/fill6944worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6945worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6946worker1 # [ 170.855785] nix-grpc-daemon[1537]: ts=2026-09-21T09:53:38Z level=info event=healthy reason=min_free6947worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)6948(finished: subtest: low disk drains a worker and builds go to the other, in 8.29 seconds)6949(finished: run the VM test script, in 172.75 seconds)6950test script finished in 172.93s6951cleanup6952kill QemuMachine (pid 49)6953client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)6954client # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6955client # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6956client # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6957kill QemuMachine (pid 50)6958lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)6959lb # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6960lb # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6961lb # [2026-09-21T09:53:40Z INFO virtiofsd] Client disconnected, shutting down6962kill QemuMachine (pid 51)6963worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)6964worker1 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6965worker1 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6966worker1 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6967kill QemuMachine (pid 52)6968worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)6969worker2 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6970worker2 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6971worker2 # [2026-09-21T09:53:41Z INFO virtiofsd] Client disconnected, shutting down6972(finished: cleanup, in 1.03 seconds)6973additionally exposed symbols:6974 client, lb, worker1, worker2,6975 vlan1,6976 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh